Google: Providing APIs Won't Reduce Search Results Scraping

Jun 29, 2018 - 7:21 am 4 by

Google Scrape

There was a somewhat fun conversation on Twitter around the new URL inspector tool. Some SEOs are saying that if Google provided an API for this tool, it would reduce scraping of the Google search results. As you know, scraping Google's search results is against their terms of service and Google is not a fan of those that do it.

But the truth is, Google has done very little to stop the popular tools from scraping their search results. There have been times where they caused problems for the tools but overall, scrapers still do what they do best, scrape.

Now when someone mentioned to John Mueller of Google that providing this API for the URL inspector tool would reduce scraping, John laughed. He said, "Hah, I doubt it." Adding also that it is not right to say we will break your TOS (terms of service) if you don't give us a feature we want, is not a good argument.

The topic of scraping is very interesting because technically Google is one big scraper but of course, we can always use robots.txt or other methods to prevent Google from indexing or scraping our content.

Forum discussion at Twitter.


Popular Categories

The Pulse of the search community


Search Video Recaps

Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 24, 2024

May 24, 2024 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Ranking Volatility, Ads In Google AI Overviews, Sundar Pichai Interview, Heartfelt Helpful Content & More Ad News

May 24, 2024 - 8:01 am
Google Search Engine Optimization

Google: The Site Reputation Abuse Policy Enforcement Not Yet Algorithmic

May 24, 2024 - 7:51 am
Google Search Engine Optimization

Google Search Can Now Index Electronic Publication (EPUB)

May 24, 2024 - 7:41 am

Directory Of Embarrassing Google AI Overviews

May 24, 2024 - 7:31 am
Web Analytics

Google Analytics Real-Time Reports Adds Users In Last 5 Minutes

May 24, 2024 - 7:21 am
Previous Story: Google: Google Analytics Won't Hurt Or Benefit Your Search Rankings