Summary
DataMilk AI closely observes user behavior and intelligently determines the optimal timing for displaying each component to achieve maximum effectiveness. Due to this dynamic approach, there is no control group, and consequently, no uplift value can be assigned to individual components.
To measure performance, we continuously monitor granular data for each component and assess its statistical impact on the overall results. Based on this assessment, each component is assigned an effect status: good, learning, or auto-paused. This status is visible in the dashboard for every component.
If a component fails to deliver the desired results, the system will automatically pause it. The following is a summary of the Smart Component Performance Tags that you can find in your dashboard's Smart Component List:
- Good
AI believes this Smart Component is improving performance and is selectively showing on pageviews / sessions where it has a positive impact.
- Learning
AI is collecting data and observing the user’s behavior. AI does not have enough data to determine positive results yet.
- Maintenance
DataMilk has paused the component while maintenance is in progress.
- Auto paused
AI determined this component did not deliver sufficient positive results and has decided to pause the component.
- NA (Not applicable)
Experiment not yet live (example Coming Soon or Paused by client before AI can determine positive results).
Example:
Tooltips
- You can hover over any of the performance tags for a concise explanation.
- You can also sort the list by performance by clicking the column title.
FAQs
What are the status types that can be assigned to a component?
There are four status types: Good, Learning, Maintenance, and Auto-paused. They are defined above.
What is the time period the status accounts for? I.e. is it ‘Good’ for the last x days?
Performance can fluctuate on a daily basis. The AI classification occurs in real time utilizing the complete history of the component’s performance since it was launched. Additionally, recent data is given greater importance than older data.
Will ‘Auto-Paused’ components ever be turned on again automatically or can the customer or DataMilk remove them? Is there any impact to the site's vitals if those components are left in the library?
Smart Components that are auto-paused won’t be removed because there is a chance they may be utilized in the future and they will not negatively impact the site’s vitals.