Tag: crawlers
Most Recent Entries
Google: SEOs Can Help Shape Client Policies & Decisions On AI Bots
Oct 3, 2024 - 7:31 am 4 — by Barry SchwartzSearch News Buzz Video Recap: Google Ranking Volatility Spikes, Search Console Bug, Search App Stalled, Google Ads News & Business Profile Suspensions
Sep 20, 2024 - 8:01 am 1 — by Barry SchwartzTwo New Googlebots: GoogleOther-Image & GoogleOther-Video
May 17, 2024 - 7:21 am 0 — by Barry SchwartzOpenAI's ChatGPT New Web Crawler - GPTBot
Aug 7, 2023 - 7:21 am 1 — by Barry SchwartzNew Googlebot: Google-InspectionTool Added To Crawler User Agents
May 17, 2023 - 7:13 am 0 — by Barry SchwartzSearch News Buzz Video Recap: Google Reviews Update Done, Page Experience, Site Name Fixes, Google Texting Businesses & Google / Microsoft Ad Revenues
Apr 28, 2023 - 8:01 am 0 — by Barry SchwartzOlder Entries
Google: Blocking Other Crawlers Does Not Impact Google Search
Mar 2, 2023Google AdsBot Mobile Web Crawler Gets New User Agent String
Jul 14, 2022Video Recap of Weekly Search Buzz :: February 12, 2016
Feb 12, 2016How To Quickly Unblock Google From CSS & JavaScript, What Google Looks At & Number Notified
Jul 29, 2015Video Recap of Weekly Search Buzz :: February 1, 2013
Feb 1, 2013DuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012Google: Robots.txt Files Must Be Smaller Than 500KB
Jan 30, 2012BingBot IP Ranges Updated
Nov 16, 2011Google: Remove The Robots.txt File Completely
Jan 6, 2011Bing's MSNBot Crawls Twice, Once For Compressed HTML & Again For Uncompressed
Dec 16, 2009MSNBot Crawl Delay Doesn't Delay
Nov 11, 2009Bing's MSNBot Crawl Happy?
Oct 23, 2009Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails
Sep 17, 2009Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot
Sep 9, 2009DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b)
Aug 3, 2009MSNBot Clicking on Bing adCenter Search Ads?
Jul 29, 2009New MSNBot Named adidxbot Causing Trouble
Jul 7, 2009Tracking Search Bot Activity
Apr 6, 2009Multiple Robots.txt Files for Single Domain
Apr 1, 2009Live Search Begins Crawling JavaScript with MSNBot-Media
Jan 2, 2009Did Ask.com Stop or Slow Crawling the Web?
Jul 18, 2008Popular Categories
Search Video Recaps
Most Recent Articles
Daily Search Forum Recap: May 22, 2025
May 22, 2025 - 10:00 am
0
-
by
Barry Schwartz
Google Ads Come To AI Mode & AI Overviews Desktop
May 22, 2025 - 7:51 am
6
-
by
Barry Schwartz
New Google Ads Smart Bidding Exploration: Bidding Update
May 22, 2025 - 7:41 am
0
-
by
Barry Schwartz
Google Ads New AI Tools & Agentic Capabilities
May 22, 2025 - 7:31 am
0
-
by
Barry Schwartz
New Google Docs On Using AI Generative Content & AI Search Features
May 22, 2025 - 7:21 am
0
-
by
Barry Schwartz
Bing Tests Double Border Search Box
May 22, 2025 - 7:11 am
0
-
by
Barry Schwartz