When Google Allows You To A/B Test Using User-Agents

Mar 13, 2015 - 8:17 am 1 by

Google testingWhen Google posted a blog post on approved A/B testing methods, they specifically said do not use user-agents when determining what to A/B test. Google wrote, "make sure that you’re not deciding whether to serve the test, or which content variant to serve, based on user-agent."

So when someone in the Google Webmaster Help forums had a valid reason to do so, John Mueller from Google explained why it may be okay in his case.

In short, he wants to test conversions between an m-dot and responsive design for iPad users only. He wants to see if the conversions will drop or increase but is worried if he does user-agent target for iPad users, it may be against Google's guidelines.

John Mueller said it should be okay but added some caveats, which I will quote right there:

I'd like to say that if this is within a specific category of devices, then it's not a problem doing that kind of A/B testing. That said, there are some weird edge cases that you'd need to watch out for. For example, if you're A/B testing a mobile version on separate URLs vs a non-mobile-version on the same URL, then that can get a bit confusing if things fluctuate back and forth "semi-randomly." If these tests can be completed within a reasonable length of time, or if you're able to do A/B testing within the same category of content that uses the same configuration (for example, A/B testing different m.dot versions of the site), then that's less of an issue.

So be careful when doing A/B testing and the best bet, do it for a short period of time.

Forum discussion at Google Webmaster Help.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 27, 2024

Mar 27, 2024 - 4:00 pm
Google Ads

Google Local Service Ads Sends Email Asking You To Upload Photos

Mar 27, 2024 - 7:51 am
Google Maps

European Hotels Can Remove Pricing Information From Google Hotel Search

Mar 27, 2024 - 7:41 am
Google Search Engine Optimization

SEOs, Please Don't Remove Contact Us & About Us Pages

Mar 27, 2024 - 7:31 am
Misc Google

Google Search Developer Docs Gain AI Generated Help Features

Mar 27, 2024 - 7:21 am
Google Maps

Google Local Reviews Reactions Notice

Mar 27, 2024 - 7:11 am
Previous Story: Google Maps New Report Problem Feature Adding Moved Locations