Problem Statement and Metrics

Ad click prediction

1. Problem statement

Build a machine learning model to predict if an ad will be clicked.

For the sake of simplicity, we will not focus on the cascade of classifiers that is commonly used in AdTech.

• Let’s understand the ad serving background before moving forward. The ad request goes through a waterfall model where publishers try to sell its inventory through direct sales with high CPM (Cost Per Million). If it is unable to do so, the publishers pass the impression to other networks until it is sold.

2. Metrics design and requirements

Metrics

During the training phase, we can focus on machine learning metrics instead of revenue metrics or CTR metrics. Below are the two metrics:

Offline metrics

• Normalized Cross-Entropy (NCE): NCE is the predictive logloss divided by the cross-entropy of the background CTR. This way NCE is insensitive to background CTR. This is the NCE formula:

$NCE = \frac{-\frac{1}{N} \sum_{i=1}^n (\frac{1+y_i}{2} log(p_i)) + \frac{1-y_i}{2}log(1-p_i))} {-(p*log(p) +(1-p)*log(1-p))}$ ...