GoogleBot Does Not Use Nutch

Oct 5, 2020 • 7:39 am | comments (0) by | Filed Under Google Search Engine Optimization

Googlebot Nutch

Britney Muller spotted someone using Apache Nutch with a GoogleBot useragent name when crawling a site. Google has confirmed GoogleBot does not use Nutch in its useragent. Nutch is "highly extensible and scalable open source web crawler software project."

Here is what Britney posted:

John Mueller from Google confirmed Google does not use nutch at all:

He said "We don't use "nutch" at all in any of the Googlebot user-agents we use for search or for the other uses of the shared infrastructure."

So if you see this with GoogleBot, it is not a real GoogleBot and you can block it if it is causing you issues.

Forum discussion at Twitter.

Previous story: Google Shares Details On The Canonicalization & Mobile-Indexing Bugs
Ninja Banner
blog comments powered by Disqus