There was a somewhat fun conversation on Twitter around the new URL inspector tool. Some SEOs are saying that if Google provided an API for this tool, it would reduce scraping of the Google search results. As you know, scraping Google's search results is against their terms of service and Google is not a fan of those that do it.
But the truth is, Google has done very little to stop the popular tools from scraping their search results. There have been times where they caused problems for the tools but overall, scrapers still do what they do best, scrape.
Now when someone mentioned to John Mueller of Google that providing this API for the URL inspector tool would reduce scraping, John laughed. He said, "Hah, I doubt it." Adding also that it is not right to say we will break your TOS (terms of service) if you don't give us a feature we want, is not a good argument.
I find the argument of "we'll break your tos less if you give us what we want" somewhat unconvincing.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) June 25, 2018
The topic of scraping is very interesting because technically Google is one big scraper but of course, we can always use robots.txt or other methods to prevent Google from indexing or scraping our content.
Forum discussion at Twitter.
