Forget mobile first, now it's HTTPS first. Google announced last night that they will be indexing HTTPS by default from now on. That means GoogleBot will start crawling HTTPS equivalents of HTTP pages, even when the former are not linked to from any page.
Google explained that "when two URLs from the same domain appear to have the same content but are served over different protocol schemes," Google will typically choose to index the HTTPS URL if:
- It doesn't contain insecure dependencies.
- It isn't blocked from crawling by robots.txt.
- It doesn't redirect users to or through an insecure HTTP page.
- It doesn't have a rel="canonical" link to the HTTP page.
- It doesn't contain a noindex robots meta tag.
- It doesn't have on-host outlinks to HTTP URLs.
- The sitemaps lists the HTTPS URL, or doesn’t list the HTTP version of the URL
- The server has a valid TLS certificate.
John Mueller made a comment on Twitter this morning:
HTTPS takes time (& sometimes money) to set up, but it's the best we have & where the web is moving; you can't hid from it forever.
— John Mueller (@JohnMu) December 18, 2015When I asked about the on-host outlinks, John said:
@pedrodias @rustybrick If a page only has links to HTTP, that's a bit different than a page with relative links (which continue on HTTPS)
— John Mueller (@JohnMu) December 18, 2015Zineb explained on Twitter that on-host means internal links.
So Google is going bigger with HTTPS, again.
Forum discussion at Google+.
