Early this month, we gave a short roundup of threads where people have reported Pages Coming In & Out of the Google.com Results & Google Index. Yesterday, GoogleGuy finally chimed in at WebmasterWorld on the issue, or a part of the issue, stating;
Just to give folks an update, we've been going through the feedback and noticed one thing. We've been refreshing some (but not all) of the supplemental results. One part of the supplemental indexing system didn't return any results for [site:domain.com] (that is, a site: search with no additional terms). So that would match with fewer results being reported for site: queries but traffic not changing much. The pages are available for queries matching the supplemental results, but just adding a term or stopword to site: wouldn't automatically access those supplemental results.I'm checking with the crawl/index folks if this might factor into what people are seeing, and I should hear back later today or tomorrow. In the mean time, interested folks might want to check if their search traffic has gone up/down by a major amount, and see if there are fewer/more supplemental results for a site: search for their domain. Since folks outside Google couldn't force the supplemental results to return site: results, it needed a crawl/index person to notice that fact based on the feedback that we've gotten.
Anyone that wants to send more info along those lines to bostonpubcon2006 [at] gmail.com with the subject line "crawlpages" is welcome to. So you might send something like "I originally wrote about domain.com. I looked at my logs and haven't seen a major decrease in traffic; my traffic is about the same. I used to have about X% supplemental results, and now I hardly see any supplemental results with a site:domain.com query."
Chew on that for a bit. GoogleGuy then came back to answer some more specific questions. He admits to perfectly clean sites, with lots of pages, dropping pages out of the index, but only slightly losing search referral traffic from Google.com. He also says that you should "expect a reindex of most of the supplemental results over time."
Forum discussion at WebmasterWorld and huge hat tip to Donna for spotting this.
