Google has many flavors of crawlers from web search to AdWords to AdSense and app crawlers and more. But which are used for URL discovery to be used in web search?
Well, John Mueller of Google said Google AdsBot is not one of them that is used for URL discovery. He said so on Twitter when Dawn Anderson asked him, he responded, "as far as I know that's not the case."
Here are the tweets:
@dawnieando as far as I know that's not the case.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) February 6, 2017
If I had to guess, these bots are used for search URL discover; Googlebot, Googlebot-News, Googlebot-Image, and Googlebot-Video. These bots are not used for search URL discovery; Mediapartners-Google, Mediapartners, AdsBot-Google and AdsBot-Google-Mobile-Apps.
In short, Google tries hard to separate ads with organic even when it comes to crawling and URL discovery.
Forum discussion at Twitter.
