A/B Testing in E-Commerce: How the Color of the "Buy" Button Affects CTR

When updating their websites, owners of e-commerce websites often overlook minor interface details, such as button color, call-to-action (CTA) wording, and block order. Many believe these changes are insignificant and focus only on major functional updates.

However, this article proves otherwise: even minor changes, such as the color of the "Buy" button, can significantly impact click-through rate (CTR) and the number of items added to the cart.

This direct evidence shows that small A/B tests can influence business metrics and produce tangible results.

What is A/B testing, and why is it important for eCommerce businesses?

It forms a reliable basis for website optimization because each hypothesis is tested on real visitors.

Here are a few examples of changes that affect user behavior:

• The color or size of the "Buy" button affects CTR and the number of items added to the cart.

• The wording of the CTA (e.g., "Buy," "Purchase Now," or "Add to Cart") can change the conversion rate.

• The order of blocks on the page or the display of badges ("Bestseller," "Sale," "Last Item") affects how thoroughly users view the catalog.

• Even small details, such as an accent background or icon, can increase trust and the number of completed payments.

On e-commerce websites, A/B testing allows you to test hypotheses on real users without risk. You can split the traffic between options and collect accurate data on which one works best.

Testing a color change for the Buy button

In this example, the website catalog used the basic button option, with only an outline and no fill. While this design was minimalistic, it was also less noticeable.

Our team decided to investigate ways to increase the CTR of the button.

We conducted an A/B test on the CTA button using VWO to test this theory.

Conducting an A/B test of the “Buy” button in the catalog: a brief guide

VWO (Visual Website Optimizer) offers a convenient, no-code solution for such experiments. You can select an element on the page, change its appearance in the editor, and run the test immediately.

In our case, however, we changed the color of the button on all pages of the catalog, so we used custom code (CSS/JS).

Selecting pages for testing 

The first thing you need to configure is the Pages block, which is where A/B testing will take place. Add the required URL from the site where testing is being conducted. 

In our case, these were all the pages in the catalog. 

Creating button versions

In the Variations block, we added two versions:

1. A standard "Buy" button with an outline

2. A "Buy" button with a fill

In the "Traffic Split" column, we set the traffic distribution. In our case, we divided it equally between the two options. This means that, of the two sessions, one user will see option A and the other will see option B.

Configuring metrics and events in VWO via Google Tag Manager

To track the CTR of the Buy button, you need to set up a custom event in VWO. 

In the Events tab in VWO, create a new event. Specify a name, such as "add_to_cart," and copy the API code.

Integrate with Google Tag Manager (tag + trigger).

Open Google Tag Manager (GTM) and create a new tag:

1. Select Tag Configuration > Custom HTML.

2. Paste the API code copied from VWO.

Next, configure the trigger. The type of trigger depends on the event. In our case:

• Trigger type: Custom Event

• Event name: add_to_cart

• This trigger fires on all catalog pages (filtered by URL)

Test the event in GTM Preview mode. If everything works correctly, publish the tag.

Then, return to Visual Website Optimizer. On the Metrics tab, select the event you created as a key metric.

The test is ready. Check the settings, launch the experiment, and begin collecting data.

A/B testing results

In this experiment, we analyzed two versions of the "Buy" button:

• Option A: a standard button without fill

• Option B: a black CTA button

According to the results, Option B was the winner, as it demonstrated a higher CTR and conversion rate.

A comparison of the CTRs for options A and B

The test results confirmed our hypotheses:

• The expected conversion rate was 1.4% for the black button, which is higher than the conversion rate for the button without fill (1.15%).

• The expected improvement is a 21.54% relative increase in conversion, which proves the effectiveness of the new CTA design.

• The probability that option B is better is 82.67%.

Business impact and benefits for e-commerce

The A/B test example discussed above proves that conversion optimization in e-commerce is an ongoing process. Using platforms such as VWO enables you to swiftly test hypotheses and identify the most effective solutions.

To achieve impactful results, it is worth following a few rules:

1. Conduct regular A/B testing of CTAs and other design elements, as they directly affect CTR.

2. Use no-code tools to quickly launch experiments without involving developers.

3. Focus on key metrics: CTR, conversion rate, and expected improvement. Try to achieve high winner threshold percentages so that decisions are statistically reliable.

Conclusions 

The results of the A/B test confirmed that changing the color of the button affects user behavior. The filled black button had a statistically significant increase in CTR, with a probability of better than 82.67%. This demonstrates that even small changes can have a significant effect.

For businesses, there are a few key takeaways:

1. Enjoy direct conversion growth. Using this strategy, you can get more purchases from the same number of visitors.

2. Perform website optimization without significant costs. Simple changes to the CTA design do not cost much but can deliver measurable results.

3. Regular A/B testing is important. Even minor details, like the color of a button, can significantly impact user behavior. This makes it important to regularly test different elements of a website.