Skip to main content

I have an A/B test tool in house, how do I use it with DataMilk?

  • Updated

Introduction

As businesses strive to improve their online performance, A/B testing plays an increasingly crucial role in optimizing website designs and user experiences allowing the statistical effectiveness of a change to be accurately measured. DataMilk automatically performs A/B testing to provide accurate measurement of effectiveness. In some cases, customers have an existing A/B testing tool and wish to use their own tool with DataMilk. This article will guide DataMilk customers through the process of integrating an existing A/B testing tool with DataMilk, to obtain the most accurate and actionable results.

 

The Challenge: Ensuring Accurate A/B Test Results

One of the challenges faced by DataMilk customers when using A/B testing tools is the potential for discrepancies between the results measured by the A/B testing tool and DataMilk. DataMilk effectiveness requires low latency loading in order to optimize the user experience as soon as a page is displayed and to send analytics events to measure the results. Third party A/B testing tools can delay DataMilk's execution and lead to incomplete data collection.

 

The following instructions are intended to address this issue and ensure that both DataMilk and customer measurements are more consistent. 

 

The Solution: Integrating an A/B Testing Tool with DataMilk

For DataMilk customers to integrate their existing A/B testing tool with DataMilk, the customer should add a publicly available script provided by DataMilk to the website's HEAD section. This script allows the customer to control if the user will be part of the CONTROL (no DataMilk) or TREATMENT (with DataMilk) group. 

 

The Script will do the following:

  1. Save the selection as a cookie to ensure consistency in follow-up runs of the script.
  2. Both Treatment and Control groups will send tags immediately to Google Analytics.
  3. In the TREATMENT group, DataMilk UX changes will be fetched and executed right after the tags are sent.
  4. DataMilk will send pixels as usual, ignoring sessions without those.

With this setup, DataMilk customers will get accurate session counters for both A and B groups in their Google Analytics account, allowing them to determine the number of conversions in each group. They can then plug these numbers into a free third-party A/B calculator to compute the results, including the p-value. 

 

This integration is valuable for various DataMilk customers:

  • API customers using A/B testing tools like Optimizely can test optimizations built with DataMilk APIs as they do with other features of their site.
  • Managed Services customers can use this integration to independently confirm performance figures in the DataMilk managed components dashboard.
  • Customers using GA and Google Optimize can continue A/B testing UX changes after Google Optimize is sunset by using the proposed setup.

Conclusion

Integrating a third party A/B testing tool with DataMilk can be used to measure effectiveness if using API or independently A/B test DataMilk managed service. By following the steps outlined in this article, DataMilk customers can achieve more accurate and actionable results from their A/B tests. DataMilk is available to support customers in their journey towards improved online performance and growth. For further assistance and more detailed information on this setup, contact your customer success manager or account manager.