WebmasterWorld Bans Search Engine Bots from Crawling

Nov 21, 2005 • 11:19 am | comments (2) by twitter Google+ | Filed Under SEO & SEM Forum News

Danny reports that WebmasterWorld Bans Spiders From Crawling where he points to a thread Brett started which he named lets try this for a month or three...

Brett explains that the bots, mostly from unauthorized sources are taking a toll on the server. He explains; "We have pushed the limits of page delivery, banning, ip based, agent based, and down right cloaking to avoid the rogue bots - but it is becoming an increasingly difficult problem to control."

Members are worried about how they can search WebmasterWorld, since the internal search engine is sub-par. Brett hopes that they can allow the bots again within 60 days, but it depends. Best of luck Brett!

Forum discussion at WebmasterWorld.

Previous story: AdSense Turns More AdBright Like with Advertise Here Links



10/17/2008 12:17 am

i ban ALL search bots on first sight.Google is the worst info stealer of them all and the most brainless of them all too.i have emailed them several times telling the morons to stay out of my site


06/11/2013 10:13 pm

hehe, wow. How'd that work out :)

blog comments powered by Disqus