I question the usefulness of this Barry. I believe last modified dates are pretty much defunct these days, since most sites are dynamic and not aware of their last modified dates. PHP will not send a last modified date in its default configuration, as this would be disasterous for several reasons (The timestamp on the page is probably not at all related to the actual age of the content of the page, as well as the fact that page.php?id=X typically houses several thousand pages.

Yes that is true. But why wouldn't they store that info, why they are there.

The apparent upswing in HEAD requests came with the December 2004 update, so far as I have seen in my own log activity. Given that their crawling patterns have changed since Big Daddy, those requests may be proceeding at the same pace as before but now constitute a higher proportion of requests for many sites.

The dynamic content issue is a tough one to navigate now because so many people use mod_rewrite or their .htaccess file (or some other method) to assert static URLs for what is actually dynamic content.

Maybe Google should invest time in asking people to use and honor a new meta tag: name="static-content" description="no".

I've noticed the same behaviour now cropping up again. Looking through logs of various sites, I've seen Googlebot doing a "HEAD /" on the home page of the site. This then resulted in sites that had 302-redirected home pages showing up in the index, but the "/" page no longer showing up.

I saw this in July, but then not again until August 2nd and August 10th where this "HEAD /" pops up again here and there over many of my sites. Now, all of the sudden, 302-redirected home pages are showing up in the index again. Anyone else getting similar phenomena?