Google: Do Not Do Extended A/B Testing With The Same A/B Test

Jun 17, 2016 - 7:45 am 3 by

google test tubes

We've covered how Google handles A/B testing and that they are cool with it. But someone asked in a Google Hangout on Google+ how long is too long for an A/B test. John Mueller from Google wouldn't give a specific time frame but he said don't let it stick for too long.

This question came up at the 15:04 mark into the video:

I’m serving A/B tests to a big proportion of traffic using Adobe software. If these tests are served to a selection of UA but not Googlebot, is this considered cloaking and could it risk a penalty? What is the recommended duration of time for a A/B test?

John responded:

So from our point of view, you should treat GoogleBot like you would any other user. So if you are doing A/B testing then GoogleBot probably needs to fall into one of those A or B test, how many variations you are testing slots as well.

The other thing to keep in mind is you need to kind of treat GoogleBot like other users in that location. So if you treat users in a different way, if you do an A/B test specifically for users in the US and GoogleBot crawls from the US for your site then you need kind of need to include GoogleBot in that A/B test route. In other way, if you are doing an A/B test for users in France and GoogleBot is crawling from the US and everyone from the US sees the same version then that is nothing where you’d need to do anything special.

With regards for the duration, we just recommend to kind of keep it reasonable. In the sense that you shouldn’t be doing an A/B test for an extended period of time where you can say well, it is not an A/B test anymore, it is essentially the stable state for the site. So that is something where we don’t have any fixed time but kind of need to look at it on your site.

Some sites do regular A/B testing, that they go A/B and that they test it for a month and then they test other variations and next variations and they continually test and that is fine too. It is just we don’t want a situation where you do different content from what GoogleBot uses.

So he said don't do it for an "extended period of time" but you can continue to A/B test different variations all the time, like Google does.

It is more about a cloaking issue that you should avoid.

Here is the video embed:

Forum discussion at a href="https://plus.google.com/u/0/events/c2vhkbnl9dnso7jacf2ncv7o098">Google+.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Google Urges Patience As The March 2024 Core Update Continues To Rollout

Mar 18, 2024 - 7:51 am
Google

Official: Google Replaces Perspective Filter With Forums Filter

Mar 18, 2024 - 7:41 am
Google Maps

Google Business Profiles Now Offers Additional Review After Appeal Is Denied

Mar 18, 2024 - 7:31 am
Google Maps

EU Searchers Complaining About Google Maps Features Changes Related To DMA

Mar 18, 2024 - 7:21 am
Google

Google Showing Fewer Sitelinks Within Search

Mar 18, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: March 15, 2024

Mar 15, 2024 - 4:00 pm
Previous Story: Google Mobile Search Dropping Custom Date Range Search Filter