Unique: AppleBot Will Listen As If It Is GoogleBot

Dec 2, 2015 - 8:23 am 0 by
Filed Under Apple App Store

apple search algorithm

We covered the AppleBot directives documentation before, back in May. But what I didn't know is that AppleBot is unique in that if you do not make any mention of directives for AppleBot but you do for GoogleBot, it will follow the GoogleBot directives you set up. Bing, Yahoo, etc, do not do the same, they follow general directives or directives detailed for their bots but not for GoogleBot.

John Mueller said this on Google+:

If you have a well-crafted & complex robots.txt file (e.g., longer than two lines), keep in mind that Applebot will follow Googlebot directives unless you explicitly set up a user-agent section for it. I'm not aware of other crawlers doing something similar, so I thought it would be worth sharing.

I didn't know it was a unique thing but John would know and he mentioned it.

The official AppleBot page says so:

If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.

This is specifically for Universal Search and App Deep Linking I believe - at least for now.

Forum discussion at Google+.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Volatility, Helpful Content Update Gone, Dangerous Search Results & Ads Confusion - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Weekend Google Core Ranking Volatility Taxing Site Owners

Apr 14, 2024 - 8:15 am
Search Forum Recap

Daily Search Forum Recap: April 12, 2024

Apr 12, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Volatility, Helpful Content Update Gone, Dangerous Google Search Results & Google Ads Confusion

Apr 12, 2024 - 8:01 am
Google Search Engine Optimization

Google: Indexing & Algorithm Updates Are Independent

Apr 12, 2024 - 7:51 am
Google Search Engine Optimization

Google Structured Data Carousels Beta Docs Clarifies Feature Availability & Markup Location

Apr 12, 2024 - 7:41 am
Google

Google: Ranking In Shopping, Images & Other Verticals Doesn't Hurt Your Web Rankings

Apr 12, 2024 - 7:31 am
Previous Story: Google's Near Parameter Still Lets You See Results By Specific Location