Tag: crawlers
Most Recent Entries
Two New Googlebots: GoogleOther-Image & GoogleOther-Video
May 17, 2024 - 7:21 am 0 — by Barry SchwartzOpenAI's ChatGPT New Web Crawler - GPTBot
Aug 7, 2023 - 7:21 am 1 — by Barry SchwartzNew Googlebot: Google-InspectionTool Added To Crawler User Agents
May 17, 2023 - 7:13 am 0 — by Barry SchwartzSearch News Buzz Video Recap: Google Reviews Update Done, Page Experience, Site Name Fixes, Google Texting Businesses & Google / Microsoft Ad Revenues
Apr 28, 2023 - 8:01 am 0 — by Barry SchwartzGoogle: Blocking Other Crawlers Does Not Impact Google Search
Mar 2, 2023 - 7:11 am 0 — by Barry SchwartzGoogle AdsBot Mobile Web Crawler Gets New User Agent String
Jul 14, 2022 - 7:51 am 0 — by Barry SchwartzOlder Entries
Video Recap of Weekly Search Buzz :: February 12, 2016
Feb 12, 2016How To Quickly Unblock Google From CSS & JavaScript, What Google Looks At & Number Notified
Jul 29, 2015Video Recap of Weekly Search Buzz :: February 1, 2013
Feb 1, 2013DuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012Google: Robots.txt Files Must Be Smaller Than 500KB
Jan 30, 2012BingBot IP Ranges Updated
Nov 16, 2011Google: Remove The Robots.txt File Completely
Jan 6, 2011Bing's MSNBot Crawls Twice, Once For Compressed HTML & Again For Uncompressed
Dec 16, 2009MSNBot Crawl Delay Doesn't Delay
Nov 11, 2009Bing's MSNBot Crawl Happy?
Oct 23, 2009Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails
Sep 17, 2009Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot
Sep 9, 2009DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b)
Aug 3, 2009MSNBot Clicking on Bing adCenter Search Ads?
Jul 29, 2009New MSNBot Named adidxbot Causing Trouble
Jul 7, 2009Tracking Search Bot Activity
Apr 6, 2009Multiple Robots.txt Files for Single Domain
Apr 1, 2009Live Search Begins Crawling JavaScript with MSNBot-Media
Jan 2, 2009Did Ask.com Stop or Slow Crawling the Web?
Jul 18, 2008Can Google's First Click Free Program be Used for Web Search?
Jun 5, 2008Google Now Crawling Content Behind Forms
Apr 14, 2008Popular Categories
Search Video Recaps
Most Recent Articles
Daily Search Forum Recap: July 26, 2024
Jul 26, 2024 - 10:00 am
0
-
by
Barry Schwartz
Google Volatility, Bing Generative Search, Reddit Blocks Bing, Sticky Cookies, AI Overview Ads & SearchGPT
Jul 26, 2024 - 8:01 am
0
-
by
Barry Schwartz
Google Gemini Adds Related Content & Verification Links
Jul 26, 2024 - 7:51 am
0
-
by
Barry Schwartz
SearchGPT - OpenAI's AI Search Tool
Jul 26, 2024 - 7:41 am
2
-
by
Barry Schwartz
Google's John Mueller: Don't Use LLMs For SEO Advice
Jul 26, 2024 - 7:31 am
8
-
by
Barry Schwartz
Google Search With Related Images Carousel Below Image Box
Jul 26, 2024 - 7:21 am
0
-
by
Barry Schwartz