GoogleBot Does Not Use Nutch

Oct 5, 2020 - 7:39 am 0 by

Googlebot Nutch

Britney Muller spotted someone using Apache Nutch with a GoogleBot useragent name when crawling a site. Google has confirmed GoogleBot does not use Nutch in its useragent. Nutch is "highly extensible and scalable open source web crawler software project."

Here is what Britney posted:

John Mueller from Google confirmed Google does not use nutch at all:

He said "We don't use "nutch" at all in any of the Googlebot user-agents we use for search or for the other uses of the shared infrastructure."

So if you see this with GoogleBot, it is not a real GoogleBot and you can block it if it is causing you issues.

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Volatility, Indexing Bugs, Google Ads Broad Match Default & YouTube SEO Tips - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 12, 2024

Jul 12, 2024 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Volatility, Indexing Bugs, Google Ads Broad Match Default & YouTube SEO Tips

Jul 12, 2024 - 8:01 am
Google Maps

Remove Thumbtack Book Links In Google Business Profiles

Jul 12, 2024 - 7:51 am
Google Search Engine Optimization

New: Google Search Console Let's You Add Your Shipping & Return Information

Jul 12, 2024 - 7:41 am
Google

Google Search Tests Autoplaying Video Thumbnails In Results?

Jul 12, 2024 - 7:31 am
Bing Search

Bing Search Tests Voting On Search Results

Jul 12, 2024 - 7:21 am
Previous Story: Google Shares Details On The Canonicalization & Mobile-Indexing Bugs