Google: Do Not Do Extended A/B Testing With The Same A/B Test

Jun 17, 2016 - 7:45 am 3 by

google test tubes

We've covered how Google handles A/B testing and that they are cool with it. But someone asked in a Google Hangout on Google+ how long is too long for an A/B test. John Mueller from Google wouldn't give a specific time frame but he said don't let it stick for too long.

This question came up at the 15:04 mark into the video:

I’m serving A/B tests to a big proportion of traffic using Adobe software. If these tests are served to a selection of UA but not Googlebot, is this considered cloaking and could it risk a penalty? What is the recommended duration of time for a A/B test?

John responded:

So from our point of view, you should treat GoogleBot like you would any other user. So if you are doing A/B testing then GoogleBot probably needs to fall into one of those A or B test, how many variations you are testing slots as well.

The other thing to keep in mind is you need to kind of treat GoogleBot like other users in that location. So if you treat users in a different way, if you do an A/B test specifically for users in the US and GoogleBot crawls from the US for your site then you need kind of need to include GoogleBot in that A/B test route. In other way, if you are doing an A/B test for users in France and GoogleBot is crawling from the US and everyone from the US sees the same version then that is nothing where you’d need to do anything special.

With regards for the duration, we just recommend to kind of keep it reasonable. In the sense that you shouldn’t be doing an A/B test for an extended period of time where you can say well, it is not an A/B test anymore, it is essentially the stable state for the site. So that is something where we don’t have any fixed time but kind of need to look at it on your site.

Some sites do regular A/B testing, that they go A/B and that they test it for a month and then they test other variations and next variations and they continually test and that is fine too. It is just we don’t want a situation where you do different content from what GoogleBot uses.

So he said don't do it for an "extended period of time" but you can continue to A/B test different variations all the time, like Google does.

It is more about a cloaking issue that you should avoid.

Here is the video embed:

Forum discussion at a href="">Google+.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Coming, Ranking Volatility, Bye Search Notes, AI Overviews, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 19, 2024

Jul 19, 2024 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Coming, Ranking Volatility, Bye Search Notes, AI Overviews, Ads & More

Jul 19, 2024 - 8:01 am
Google Search Engine Optimization

Billions Of Google URLs Will No Longer Work

Jul 19, 2024 - 7:51 am
Google Search Engine Optimization

Google: ccTLDs & Language Do Help You Rank A Little Better In Local Country Region

Jul 19, 2024 - 7:41 am
Google Search Engine Optimization

Google's On Knowing If Your SEO Team Is Doing Their Job

Jul 19, 2024 - 7:31 am
Google Ads

Google Merchant Center Next Gains Support For Supplemental Feeds

Jul 19, 2024 - 7:21 am
Previous Story: Google Mobile Search Dropping Custom Date Range Search Filter