A/A Testing

A/A testing involves driving traffic to two pages to see which one performs better. Now, this may sound familiar to A/B testing but there is a slight difference between the two. In A/A testing, two exactly the same pages are compared against each other whereas, in A/B testing, two different pages are compared (the control and the variation). The goal of the A/A test is to not see which version converts better but to find no difference between your control and variation versions.

Why do organizations practice A/A Testing?

• To check the accuracy of A/B Test tools: Whenever you want to go in for a new implementation or switch to a new testing software, then you need to run an A/A test to check the working of the software before you run any A/B test.
• Determine the baseline conversion rate: Before running any A/B test, you need to set a benchmark for the conversion rate. A/A test can help you to set the baseline conversion rate for your website. For example, you are running an A/A test where the control gives 50 conversions out of 1,000 visitors, and the identical variation B gives 55 out of 1,000 conversions. The conversion rate for A is 5% and that for B is 5.5% when there is no difference between the two variations. Therefore, the conversion rate range that can be set for the future A/B test can be set at a benchmark of 5-5.5%. Any uplifts from this range after you run the A/B test would mean that the result is not significant.
• Deciding a minimum sample size: From your website traffic, you can get an idea about the minimum sample size using the A/A test. If you have a large sample size, you have a greater chance of taking into account all segments that impact the test.