WebmasterWorld Bans Search Engine Bots from Crawling

Nov 21, 2005 - 11:19 am 2 by
Filed Under SEO Forum News

Danny reports that WebmasterWorld Bans Spiders From Crawling where he points to a thread Brett started which he named lets try this for a month or three...

Brett explains that the bots, mostly from unauthorized sources are taking a toll on the server. He explains; "We have pushed the limits of page delivery, banning, ip based, agent based, and down right cloaking to avoid the rogue bots - but it is becoming an increasingly difficult problem to control."

Members are worried about how they can search WebmasterWorld, since the internal search engine is sub-par. Brett hopes that they can allow the bots again within 60 days, but it depends. Best of luck Brett!

Forum discussion at WebmasterWorld.


Popular Categories

The Pulse of the search community


Search Video Recaps

Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Massive Volatility Reported - Google Search Ranking Algorithm Update?

Feb 24, 2024 - 7:43 pm
Search Forum Recap

Daily Search Forum Recap: February 23, 2024

Feb 23, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Helpful Content System, Reddit’s Google Deal, Product Variant Schema, Google Ads, Bing Search & Search Volume Predictions

Feb 23, 2024 - 8:01 am
Google Ads

Google Ad Strength For PMAX To Weigh Assets Quantity & Variety More As AI Expands

Feb 23, 2024 - 7:51 am
Google Search Engine Optimization

Google: You Decide What Old Content Is Helpful Or Not

Feb 23, 2024 - 7:41 am

No Joke: Google Is Paying Reddit For More Content, More Often

Feb 23, 2024 - 7:31 am
Previous Story: AdSense Turns More AdBright Like with Advertise Here Links