Recently, a company called “Semlat” has software bots crawling websites all over the world. They state that you can remove yourself from their list, but so far I’ve not heard of anyone actually getting off the list. They claim to be an SEO company, but their practices of getting to the top are somewhat black hat.
Even worse, they are no longer the only ones. The list of “companies” that do this has grown huge in just the past month! A partial list of these companies is:
There is some speculation that these bots are also looking for weaknesses on sites to exploit. Even if it’s not true, it’s still a good idea to block them because it messes up Google Analytics reports and puts an unnecessary load on your server.
There are a number of ways to block these, or filter out their traffic. But the best two methods both involve adding fields to the .httaccess file on your server. Do this by:
1.) Log into your host’s cPanel.
2.) In the Files section, click on the File Manager icon. (you may have multiples)
3.) Check the box for Document Root for and select the domain name you wish to access from the drop-down menu if there is more then one.
4.) Click the box that says “Show Hidden Files (dot files)” – this is the critical step
5.) Click Go.
6.) Scroll down to .htaccess in the list of files.
7.) Highlight and click copy at the top of the screen
8.) Type in .httaccess.back and hit enter (this creates a backup, just in case)
9.) Highlight .httaccess and click “edit” at the top of the screen
10.) Paste one of the two following sections of code into the file, typically at the bottom.
11.) Click Save
12.) Access your site in a separate browser or browser window to make sure you can still access the website. If you can, all is good. If not, edit .httaccess again and remove that parts you just added.
Code #1
– this is a bit less elegant but is very direct:
SetEnvIfNoCase Referer semalt.com spambot=yes
SetEnvIfNoCase Referer buttons-for-website.com spambot=yes
SetEnvIfNoCase Referer darodar.com spambot=yes
SetEnvIfNoCase Referer 7makemoneyonline.com spambot=yes
SetEnvIfNoCase Referer ilovevitaly.co spambot=yes
SetEnvIfNoCase Referer myftpupload.com spambot=yes
SetEnvIfNoCase Referer econom.co spambot=yes
SetEnvIfNoCase Referer iskalko.ru spambot=yes
SetEnvIfNoCase Referer ilovevitaly.ru spambot=yes
SetEnvIfNoCase Referer ilovevitaly.com spambot=yes
SetEnvIfNoCase Referer o-o-8-o-o.ru spambot=yes
SetEnvIfNoCase Referer o-o-6-o-o.ru spambot=yes
SetEnvIfNoCase Referer cenoval.ru spambot=yes
SetEnvIfNoCase Referer priceg.com spambot=yes
SetEnvIfNoCase Referer cenokos.ru spambot=yes
SetEnvIfNoCase Referer seoexperimenty.ru spambot=yes
SetEnvIfNoCase Referer gobongo.info spambot=yes
SetEnvIfNoCase Referer vodkoved.ru spambot=yes
SetEnvIfNoCase Referer adcash.com spambot=yes
SetEnvIfNoCase Referer websocial.me spambot=yes
SetEnvIfNoCase Referer cityadspix.com spambot=yes
SetEnvIfNoCase Referer luxup.ru spambot=yes
SetEnvIfNoCase Referer ykecwqlixx.ru spambot=yes
SetEnvIfNoCase Referer superiends.org spambot=yes
SetEnvIfNoCase Referer slftsdybbg.ru spambot=yes
SetEnvIfNoCase Referer edakgfvwql.ru spambot=yes
SetEnvIfNoCase Referer socialseet.ru spambot=yes
SetEnvIfNoCase Referer screentoolkit.com spambot=yes
SetEnvIfNoCase Referer savetubevideo.com spambot=yes
Order allow,deny
Allow from all
Deny from env=spambot
New SPAM bots can be added easily to this list by just cutting and pasting the domain name.
Code #2
– Little more elegant. This is the one I went with. It is provided by Perishable Press.
# 5G:[WordPress]
RedirectMatch 403 /\$\&
RedirectMatch 403 (?i)/\&(t|title)=
RedirectMatch 403 (?i)/\.(bash|git|hg|log|svn|swp|tar)
RedirectMatch 403 (?i)/(1|contact|i|index1|iprober|phpinfo|phpspy|product|signup|t|test|timthumb|tz|visit|webshell|wp-signup).php
RedirectMatch 403 (?i)/(author-panel|class|database|manage|phpMyAdmin|register|submit-articles|system|usage|webmaster)/?$
RedirectMatch 403 (?i)/(=|_mm|cgi|cvs|dbscripts|jsp|rnd|shadow|userfiles)
This second version should work a little better since instead of blocking the bots, it gives them an official 403 not found. That should keep the bots from hitting your site at all in the future and reduce load even more.
Since I’ve implemented this method, I have seen Semalt (and all the others,) go away completely. I still get an occasional iffy one but it’s not as bad as the 10-15 (per a site) I was getting before.
Do you have another solution that works better? Please comment below!