When Google Allows You To A/B Test Using User-Agents

Mar 13, 2015 - 8:17 am 1 by

Google testingWhen Google posted a blog post on approved A/B testing methods, they specifically said do not use user-agents when determining what to A/B test. Google wrote, "make sure that you’re not deciding whether to serve the test, or which content variant to serve, based on user-agent."

So when someone in the Google Webmaster Help forums had a valid reason to do so, John Mueller from Google explained why it may be okay in his case.

In short, he wants to test conversions between an m-dot and responsive design for iPad users only. He wants to see if the conversions will drop or increase but is worried if he does user-agent target for iPad users, it may be against Google's guidelines.

John Mueller said it should be okay but added some caveats, which I will quote right there:

I'd like to say that if this is within a specific category of devices, then it's not a problem doing that kind of A/B testing. That said, there are some weird edge cases that you'd need to watch out for. For example, if you're A/B testing a mobile version on separate URLs vs a non-mobile-version on the same URL, then that can get a bit confusing if things fluctuate back and forth "semi-randomly." If these tests can be completed within a reasonable length of time, or if you're able to do A/B testing within the same category of content that uses the same configuration (for example, A/B testing different m.dot versions of the site), then that's less of an issue.

So be careful when doing A/B testing and the best bet, do it for a short period of time.

Forum discussion at Google Webmaster Help.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Video Recaps

Search News Buzz Video Recap: Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More

May 17, 2024 - 8:01 am
Google Search Engine Optimization

Remove Your Content From Google's AI Overviews

May 17, 2024 - 7:51 am
Google Ads

Google Ads AI Summaries Live For Some Advertisers

May 17, 2024 - 7:41 am
Google Maps

Order with Google For Food Delivery Going Away End Of June

May 17, 2024 - 7:31 am
Google Search Engine Optimization

Two New Googlebots: GoogleOther-Image & GoogleOther-Video

May 17, 2024 - 7:21 am
Google Search Engine Optimization

Google Locks Down Complaints Of Image Thumbnails Not Showing In Search

May 17, 2024 - 7:11 am
Previous Story: Google Maps New Report Problem Feature Adding Moved Locations