On June 2, 2020, Google yet again confirmed it was having issues displaying new content in its search results. Google said it was an "indexing issue." Well, it was actually a crawling issue that led to Google not being able to show the new pages in the index.
Gary Illyes from Google said in episode two of that new podcast that the issue was actually a crawling issue. Gary added that the crawling was happening so fast that it was overwhelming the indexing side. He said indexing was overwhelmed by the crawlers.
This resulted in Google was not able to export the new documents to serving chars, so for the vast majority of sites, it was not able to get those pages in the index and serve up that new content.
You can listen to him say this at around the 11:50 mark into the audio podcast embedded here:
Gary added that it now set up monitors on Twitter to look for complaints about indexing with Google. This was alerted to Google early on because they have tools that look at Twitter for indexing issue reporting, that can find these issues earlier than some of their other dashboards use. Interesting.
Forum discussion at Twitter.
