Skip to main content

How can I independently run a custom AB test to validate DataMilk’s results?

  • Updated

Important Note: Customers can already validate DataMilk’s results via Google Analytics out of the box as mentioned in this article. However, if you want to run your own custom AB test, the information below will explain accordingly.

 

To maximally trust DataMilk’s results and data you can be in control of

  1. Choosing the control and treatment groups for sessions (turning DataMilk on or off depending on whether the session is in control or treatment);
  2. Seeing the results in an independent analytics tool of your selection.

The following article will help you run a low latency AB test to maximize performance while getting an independent measure.

 

How to be in charge of choosing the control and treatment groups for sessions

Use DataMilk’s simple open source Javascript code to choose control and treatment groups and load the DataMilk script only in the treatment groups. Find the code here.

 

How to use it:

  1. Install the javascript in the HEAD of your site (replace the DataMilk javascript if you already have DataMilk installed) YOU DO NOT NEED TO UPDATE THE DATAMILK PIXEL.
  2. Make sure to replace the domain id with the correct domain id
  3. Make sure to provide the correct analytics tracking ID to correctly send events to GA/Adobe

 

What will happen next:

  1. For 50% of the control sessions the script will simply send a tag  to your analytics software without loading DataMilk at all. That means these sessions will see your original site experience without DataMilk.
    • The tags will appear as the following: 
      • Event Category is: "DataMilk"
      • Event Action is: ‘A/B’
      • Event label is: ‘datamilk_ab_original’
  1. For the other 50% of the treatment sessions the script will send a tag ( to your analytics software and also load DataMilk. These sessions will be optimized by DataMilk.
      • The tags will appear as the following:
        • Event Category is: "DataMilk"
        • Event Action is: ‘A/B’
        • Event Label: ‘datamilk_ab_optimized’
  1. You can go into your analytics tool and build a custom report to compare the performance. Here is an article on how to do this.
    • In your GA, your custom report will look something like this (screenshot taken from article linked above)