On Friday afternoon, Twitter decided to block unregistered, signed-out users, from seeing public tweets. That meant that for Google's normal crawling purposes, it was unable to see some of these tweets. It seems that Google now has about 52% fewer Twitter URLs in its index today than it had on Friday, just a few days later.
While we know the site command is not the most accurate measure of what Google has indexed, we don't have many other free tools to measure how many pages Google indexed of a site that we do not have Search Console level access to. So I decided to screen capture how many pages Google indexed of Twitter.com on Friday, right after the block started.
On Friday, June 30th at about 1pm ET, Twitter had 471 million URLs indexed in Google Search according to this site command:
Then yesterday, July 2nd, I snapped another screenshot and Google had 34% fewer URLs indexed, 309 million, that is 162 million fewer URLs indexed:
Then this morning, I grabbed another screenshot, and it was down now to 227 million URLs indexed, that is about 52% less than what was indexed on Friday:
Now, we know Google has access to the Twitter firehose, which is why we see Google still showing recent tweets in the Twitter Google Search carousel:
But for normal indexing of these Twitter URLs, it seems like these tweets are dropping out of the sky.
Twitter URLs indexed by Friday to today went from about 471 million URLs to 227 million, a drop of about 52%. I wonder what type of impact this has on its traffic from Google Search, if any.
Forum discussion at Twitter.
Update: July 5th - it looks like Twitter made some changes and now Google is having an easier time crawling and indexing content on Twitter.com.
Glenn Gabe noted on Twitter, "Looks like changes are being implemented. Twitter is now using a modal window for users not logged in (and Google can see the tweet content below the modal). And search visibility is bouncing back (see @semrush and @sistrix below). Much better than the redirect errors..."
The site command now shows a big return in the number of pages being returned in Google Search for Twitter.com:
More details can be found in our new story over here.
