Previewing predictions

A statistics tab for the Propensity and Lookalikes prediction types shows estimated prediction performance together with direct call-to-action capabilities. On the predefined preview dashboards, you can see predicted KPIs and additional metrics. In addition to that, it is also possible to:

• create segments of customers with the highest propensity or the closest lookalikes directly from the model’s dashboard.
• add your own custom dashboards to the prediction statistics

Propensity preview

1. To preview Propensity predictions, go to .
2. On the left pane, select Propensity.
3. From the Propensity prediction list, select the prediction you want to preview.
4. Under the prediction title, select the Statistics tab.

Statistics explanation

Model card

In this part of preview, you can get the basic information about the prediction and model results.

• Model type - The type of prediction.
• Model quality - The quality of the prediction. It can take five values: Very low, Low, Medium, High, and Very high. The quality of the prediction is estimated based on the data input. To increase the model quality, you may widen the segmentation of profiles for whom you prepare a prediction or extend the range of items for which the propensity prediction is calculated.
• Precision - It evaluates the share of positive conversions that were correctly predicted. The result can take values from 0 (lowest precision) to 1. In the case of propensity, it can be interpreted to what extent we can be sure that predicted conversions will be correct.
• AUC - Area under the ROC Curve is one of the most commonly used metrics for classification problems. It calculates the area underneath the entire ROC curve and ranges between 0 (lowest ROC) and 1. However, only results higher than 0.5 are considered better than random choice.
• Recalculation frequency - It describes how often the prediction is recalculated. This was defined in the prediction settings when the prediction was created.
• Last calculation - It is the time since the last recalculation of the prediction.
• Total number of generated predictions - This is the number of all snr.propensity.score events generated since the first calculation of the prediction.
  Note: The snr.propensity.score event is available on the activity list of each Profile and includes that Profile’s score. You can learn more about it here.

Audience summary

In this part of the preview, you can see the basic information about the segmentation for which the prediction was made.

• Audience - The name of the segmentation for which the prediction was made.
• Audience size - The number of profiles in the segmentation at the moment of the latest recalculation.
  
  Important: If you open the segmentation in Analytics, it is calculated every time you preview its results. Segmentations are likely to often change in size as profiles start or stop meeting their conditions.
• Profiles without generated predictions - The number of profiles for whom the snr.propensity.score event could not be generated (due to the lack of interactions) during the latest recalculation.

Distribution charts

In this part of the preview, you can see the distribution of the profiles according to percentile and score they received.

Score label

The score can be presented using one of the following scales (depending on the option you selected in the prediction settings):

1. Two-point scale (Low, High)
2. Five-point scale (Very low, Low, Medium, High, Very high)

On the right side of the chart, you can see the summary of each score label. Each column contains the following rows:

• Audience size - The number of profiles who received this score label.
• Audience share - How much (as a percentage) of the whole segmentation received this label.
• Estimated conversion rate - Estimated conversion, based on historical performance data for this segment.
• Estimated number of transactions - Estimated number of transactions for this segment, based on historical performance data.
• Gain - Percentage of the target covered in each score label.
• Lift - The gain percentage to the random percentage at a given score label level.

Percentile

On the right side of the chart, you can see the distribution of profiles in terms of percentiles. For each percentile range, you can check the following statistics:

• Estimated conversion rate - Estimated conversion, based on historical performance data for this segment.
• Estimated no. of transactions - Estimated number of transactions for this segment, based on historical performance data.
• Gain - Expected number of positive responses (conversions) for a segment to overall expected number of positive responses (conversions). Example: 50% of Gain for a specific score label or a decile. Let’s assume you want to contact TOP 20% customers with the highest score, and Gain for this segment equals 50%. It would mean that 50% of overall expected positive responses (conversions) are expected to be realized just from contacting TOP 20% of customers.
• Lift - Expected ratio of positive responses (conversion rate) for a segment to expected ratio of positive responses (conversion rate) for randomly picked customers. Example: Lift for a specific score or a decile equals 2,5x. It means that a conversion likelihood by contacting these customers is 2,5 times higher than by contacting a randomly picked group.

Lookalikes preview

1. To preview Lookalikes predictions, go to .
2. On the left pane, select Lookalikes.
3. From the Lookalikes prediction list, select the prediction you want to preview.
4. Under the prediction title, select the Preview tab.

Statistics explanation

Model card

In this part of preview, you can get the basic information about the prediction and model results.

• Model type - The type of prediction.
• Model quality - The quality of the prediction. It can take five values: Very low, Low, Medium, High, and Very high. The quality of the prediction is estimated based on the data input. To increase the model quality, you may increase the number of profiles in the source segmentation, or extend the range of events in Settings > AI Engine Configuration > Predictions.
• Precision - The share of positive conversions that were correctly predicted. The result can take values from 0 to 1. In the case of propensity, it can be interpreted to what extent we can be sure that predicted conversions will be correct.
• AUC - Area under the ROC Curve is one of the most commonly used metrics for classification problems. It calculates the area underneath the entire ROC curve and ranges between 0 and 1. However, only results higher than 0.5 are considered better than random choice.
• Recalculation frequency - It describes how often the prediction is recalculated. This was defined in the prediction settings when the prediction was created.
• Last calculation - Time since the last recalculation of the prediction.
• Total number of generated predictions - This is the number of all snr.lookalike.score events generated since the first calculation of the prediction.
  Note: The snr.lookalike.score event is available on the activity list of each Profile and includes that Profile’s score. You can learn more about it here.

Audience summary

In this part of the preview, you can see the basic information about the segmentation for which the prediction was made.

• Audience - The name of the segmentation for which the prediction was made.
• Audience size - The number of profiles in the segmentation at the moment of the latest recalculation.
  
  Important: If you open the segmentation in Analytics, it is calculated every time you preview its results. Segmentations are likely to often change in size as profiles start or stop meeting their conditions.
• Profiles without generated predictions - The number of profiles for whom the snr.lookalike.score event could not be generated (due to the lack of interactions) during the latest recalculation.

Distribution charts

In this part of the preview, you can see the distribution of the profiles according to percentile and score they received.

Score label

The score can be presented using one of the following scales (depending on the option you selected in the prediction settings):

1. Two-point scale (Low, High)
2. Five-point scale (Very low, Low, Medium, High, Very high)

On the right side of the chart, you can see the summary of each score label. Each column contains the following rows:

• Audience size - The number of profiles who received this score label.
• Audience share - How much (as a percentage) of the whole segmentation received this label.
• Number of transactions (last 30 days) - The number of transactions in the last 30 days for a group of profiles from the target segmentation who received this label.
• Historical conversion rate (last 30 days) - The unique conversion rate from the last 30 days prior most recent calculation for a given decile.
• Average daily page visits per user - Average number of page visits for profiles in the given decile per day.

Percentile

On the right side of the chart, you can see the distribution of profiles in terms of percentiles. For each percentile range, you can check the following statistics:

• Number of transactions (last 30 days) - The number of transactions in the last 30 days for a group of profiles from the target segmentation who received this label.
• Historical conversion rate (last 30 days) - The unique conversion rate from the last 30 days prior most recent calculation for a given decile.
• Average daily page visits per user - Average number of page visits for profiles in the given decile per day.

Adding custom dashboards

In the prediction statistics view, you can use the option that allows you to add your own dashboards, so you can maximize insights from prediction results.

6. Next to the Overview tab, click the icon.

   

7. From the dropdown list, select Manage dashboards.
8. On the pop-up, in the text field, enter the name of the dashboard you want to add.
9. Confirm your choice by clicking Add.
10. Optionally, you can define the order of displaying dashboards by dragging and dropping them in the desired order.
11. Confirm the dashboard settings by clicking Apply.