It seems that even after Google said it fixed the indexing issues with Disqus that has been lingering on for a while, that Google still may not be able to index all Disqus comments. Glenn Gabe said he noticed issues after the fix was rolled out. He showed those issues to Martin Splitt of Google who said that can happen.
"Martin explained that Disqus & other 3rd-party scripts can run into crawl rate issues. If they are being requested heavily, then G might not be able to request them either. And the requests time out & Google doesn't see the comments," Glenn said.
Here is the issue, in short, Glenn noticed that Google was not picking up on the new comments on sites he manages:
After receiving more comments, I resubmitted some pages for indexing via GSC. Google recrawled those pages quickly & I noticed something strange. The comments on certain pages suddenly weren't indexed anymore. They were prior to that crawl. So what happened?? Was the bug back?? pic.twitter.com/cVpZqBHTJq
— Glenn Gabe (@glenngabe) June 29, 2020
Some third-party scripts can take too long to run and they can run into crawl rate issues, he said Martin told him:
I pinged @g33konaut about what I was seeing. Martin explained that Disqus & other 3rd-party scripts can run into crawl rate issues. If they are being requested heavily, then G might not be able to request them either. And the requests time out & Google doesn't see the comments! pic.twitter.com/qF3oPBVskP
— Glenn Gabe (@glenngabe) June 29, 2020
This is not just an issue with Disqus but any third party script:
That's a big deal and could cause comments to be indexed for a certain period of time, then not be indexed until Google can render the pages accurately. Martin said that's the downside of external dependencies. So just be aware that Disqus comments might be indexed, or not... pic.twitter.com/o4LcpNE07w
— Glenn Gabe (@glenngabe) June 29, 2020
So keep that in mind when it comes to analyzing pages that have third-party scripts on them. The content in those scripts, or produced by those scripts, may or may not show to Google on one day versus the next. Then of course, when analyzing content quality of a page, that needs to be considered.
Forum discussion at Twitter.
