Tag: crawlers
Most Recent Entries
Search News Buzz Video Recap: Google December 2025 Core Update, Discover Alignment To Rankings, Search Console Features, AI Mode Updates & More
Dec 12, 2025 - 8:01 am 0 — by Barry SchwartzOpenAI Scales Up Crawling & Bots For The Holidays
Nov 28, 2025 - 7:51 am 3 — by Barry SchwartzGoogle: SEOs Can Help Shape Client Policies & Decisions On AI Bots
Oct 3, 2024 - 7:31 am 4 — by Barry SchwartzSearch News Buzz Video Recap: Google Ranking Volatility Spikes, Search Console Bug, Search App Stalled, Google Ads News & Business Profile Suspensions
Sep 20, 2024 - 8:01 am 1 — by Barry SchwartzTwo New Googlebots: GoogleOther-Image & GoogleOther-Video
May 17, 2024 - 7:21 am 0 — by Barry SchwartzOpenAI's ChatGPT New Web Crawler - GPTBot
Aug 7, 2023 - 7:21 am 1 — by Barry SchwartzOlder Entries
New Googlebot: Google-InspectionTool Added To Crawler User Agents
May 17, 2023Search News Buzz Video Recap: Google Reviews Update Done, Page Experience, Site Name Fixes, Google Texting Businesses & Google / Microsoft Ad Revenues
Apr 28, 2023Google: Blocking Other Crawlers Does Not Impact Google Search
Mar 2, 2023Google AdsBot Mobile Web Crawler Gets New User Agent String
Jul 14, 2022Video Recap of Weekly Search Buzz :: February 12, 2016
Feb 12, 2016How To Quickly Unblock Google From CSS & JavaScript, What Google Looks At & Number Notified
Jul 29, 2015Video Recap of Weekly Search Buzz :: February 1, 2013
Feb 1, 2013DuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012Google: Robots.txt Files Must Be Smaller Than 500KB
Jan 30, 2012BingBot IP Ranges Updated
Nov 16, 2011Google: Remove The Robots.txt File Completely
Jan 6, 2011Bing's MSNBot Crawls Twice, Once For Compressed HTML & Again For Uncompressed
Dec 16, 2009MSNBot Crawl Delay Doesn't Delay
Nov 11, 2009Bing's MSNBot Crawl Happy?
Oct 23, 2009Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails
Sep 17, 2009Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot
Sep 9, 2009DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b)
Aug 3, 2009MSNBot Clicking on Bing adCenter Search Ads?
Jul 29, 2009New MSNBot Named adidxbot Causing Trouble
Jul 7, 2009Tracking Search Bot Activity
Apr 6, 2009Multiple Robots.txt Files for Single Domain
Apr 1, 2009Popular Categories
Search Video Recaps
Most Recent Articles
Daily Search Forum Recap: January 23, 2026
Jan 23, 2026 - 10:00 am
0
-
by
Barry Schwartz
Search News Buzz Video Recap: Google Search Double Heated, Personal Intelligence in AI Mode, ChatGPT Ads & Apple Siri Updates Coming
Jan 23, 2026 - 8:01 am
0
-
by
Barry Schwartz
Google Search Gains Personal Intelligence In AI Mode
Jan 23, 2026 - 7:51 am
0
-
by
Barry Schwartz
Google Falls Back To Featured Snippets When AI Overviews Don't Generate
Jan 23, 2026 - 7:41 am
4
-
by
Barry Schwartz
Google Ads PMax Asset Group Editing Bug
Jan 23, 2026 - 7:31 am
0
-
by
Barry Schwartz
Gemini Local Results Insights Into How Google Interprets Your Business
Jan 23, 2026 - 7:21 am
0
-
by
Barry Schwartz
