Evaluation and Testing of Predictions

It is a good practice to test and evaluate your results. In cases where you are using a prediction model together with some incentive for the customer, it is useful to evaluate not only the overall result, but also separate the model, and the incentive. Otherwise, you will not be able to evaluate the real impact of these two separately.

We suggest dividing customers into 4 groups regarding the two parameters:

  1. High/low probability of reaching the target
  2. Will/will not be included in the campaign based on predictions. Basically, it is the difference between being and not being in the control group.

The A and B groups will be included in the campaign while the C and D groups will not.

After the campaign you can calculate the performance of all the groups and calculate the following metrics:

Strength of model

Perf(D) / Perf(C)

Answers how well can the model segment people based on their probability to fulfill the target.

Strength of impression

Perf(A+B) / Perf(C+D)

Answers what can you achieve purely with the impressions without using any prediction model.

Strength of impression on high/low segment

high: Perf(B) / Perf(D)

low: Perf(A) / Perf(C)

Compares wich of the two segments (low/hight) would have higher uplift after a campaign. In other words, which segment is the one where the campaign leads to additional conversions which would not materialize if it was not for the campaign.

Updated 11 months ago

Evaluation and Testing of Predictions


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.


We rely on cookies

to optimize our communication and to enhance your customer experience. By clicking on the Accept and Close button, you agree to the collection of cookies. You can also adjust your preferences by clicking on Manage Preferences. For more information please see our Privacy policy.

Manage cookies
Accept & close

Cookies preferences

Accept & close
Back