For the past couple of weeks I've been noticing an uptick in complaints that Google is not indexing new URLs on their web sites. They also are complaining that they can't get the fetch as Google tool to index those URLs because of the new limits. I personally have not been experiencing any delays in new URLs being indexed but more and more publishers, webmasters and SEOs have been noticing it.
Here are some tweets from yesterday (I know many shared tweets with me over the past couple weeks, but they are long gone, I can't find them now):
@JohnMu There have been a number of reports of new urls published taking days to be indexed (when new urls used to be crawled and indexed in minutes). I've seen that first-hand with some clients. Does that have anything to do with the mobile-first index? Is that temporary?
— Glenn Gabe (@glenngabe) March 26, 2018
This was something I was talking about the other day.. and @lauralippay - I was struggling to get fetch & render to work and a conversation started about the fact that urls weren't indexing as quickly. Thanks for confirmation @JohnMu
— SnowWrite #techseo (@SnowWrite) March 26, 2018
I'm seeing it too across multiple sites. Some news publishers, some not. I'm suspecting it's a timestamp issue and Google has dialed down indexing new content super fast in order to deal with fake news.
— John Doherty 🤓 (@dohertyjf) March 26, 2018
I agree and am as well. Something definitely changed that Google is not being forthcoming about IMO. I've been seeing it for a month or so.
— John Doherty 🤓 (@dohertyjf) March 26, 2018
John Mueller from Google responded but I don't think he has all the context, either way, he said that Google never indexes all URLs and this is unrelated to the mobile-first indexing roll out.
We don't index all content with the same speed, that's pretty normal & not related to mobile-first indexing. If your mobile site allows normal crawling, you wouldn't see any change in crawl/index frequency when switched over.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) March 26, 2018
Have you noticed indexing issues with new URLs recently?
Forum discussion at Twitter.
Update: Google's John Mueller commented on this, I posted about it over here.
