When Google Allows You To A/B Test Using User-Agents

Mar 13, 2015 • 8:17 am | comments (1) by | Filed Under Google Search Engine Optimization

Google testingWhen Google posted a blog post on approved A/B testing methods, they specifically said do not use user-agents when determining what to A/B test. Google wrote, "make sure that you’re not deciding whether to serve the test, or which content variant to serve, based on user-agent."

So when someone in the Google Webmaster Help forums had a valid reason to do so, John Mueller from Google explained why it may be okay in his case.

In short, he wants to test conversions between an m-dot and responsive design for iPad users only. He wants to see if the conversions will drop or increase but is worried if he does user-agent target for iPad users, it may be against Google's guidelines.

John Mueller said it should be okay but added some caveats, which I will quote right there:

I'd like to say that if this is within a specific category of devices, then it's not a problem doing that kind of A/B testing. That said, there are some weird edge cases that you'd need to watch out for. For example, if you're A/B testing a mobile version on separate URLs vs a non-mobile-version on the same URL, then that can get a bit confusing if things fluctuate back and forth "semi-randomly." If these tests can be completed within a reasonable length of time, or if you're able to do A/B testing within the same category of content that uses the same configuration (for example, A/B testing different m.dot versions of the site), then that's less of an issue.

So be careful when doing A/B testing and the best bet, do it for a short period of time.

Forum discussion at Google Webmaster Help.

Previous story: Google Maps New Report Problem Feature Adding Moved Locations
Ninja Banner
blog comments powered by Disqus