A11 · CVR, CTCVR, delay, and selection bias
Prerequisites: foundations 31–34; A10.
Evidence: explicitly constructed mechanism experiments, not real ad logs. This chapter does not claim industrial-system reproduction or online gains.
Learning order: predict → change one variable → run → explain.
English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.
# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
if (p / "reclab").is_dir()), None)
if _translation_root is None:
raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
_translation_sys.path.insert(0, str(_translation_root))
Problem and input/output contract
For post-click conversions, P(conv|exposure)=P(click|exposure)×P(conv|click,exposure). CVR training sees clicked samples whose feature distribution may differ from all impressions. Unclicked samples do not provide observed CVR truth. Before a delay window matures, an unobserved conversion is not necessarily negative. Construct ten impressions, four clicks, and two eventual conversions; never derive real conversion labels from Criteo clicks.
Before running, predict which result the changed variable will affect and which invariants must hold.
from pathlib import Path
import sys
root=Path.cwd()
if not (root/'reclab').exists(): root=root.parent
if str(root) not in sys.path: sys.path.insert(0,str(root))
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from reclab import ads_curriculum as acRun the one-variable comparison
Inputs are small constructed examples visible in source. Output units and constraints are defined above. Run assertions to verify mathematics and accounting.
early=ac.conversion_delay(2);mature=ac.conversion_delay(7)
display(pd.DataFrame([early,mature],index=['2 days','7 days']))
assert early['observed_cvr']<mature['observed_cvr']
assert early['clicked_high_share']!=early['exposure_high_share']
assert np.isclose(mature['ctr']*mature['eventual_cvr'],mature['eventual_ctcvr'])Saved notebook output · not executed on this page
| ctr | eventual_cvr | eventual_ctcvr | observed_cvr | unmatured_positive | clicked_high_share | exposure_high_share | |
|---|---|---|---|---|---|---|---|
| 2 days | 0.4 | 0.5 | 0.2 | 0.25 | 1 | 0.75 | 0.3 |
| 7 days | 0.4 | 0.5 | 0.2 | 0.50 | 0 | 0.75 | 0.3 |
Interpretation and limits
Changing the observation window from two to seven days leaves eventual CTCVR unchanged while observed CVR rises. High-response strata are overrepresented among clicks, illustrating selection. Joint CTR/CTCVR losses can explain entire-space constraints, but no real ESMM effectiveness follows without real conversion logs.
Inspect source, then try an exercise
Add an attribution window and mature non-converting samples. Report CVR by cohort maturity instead of immediately labeling every new impression zero.
import inspect
print(inspect.getsource(ac.conversion_delay))Saved notebook output · not executed on this page
def conversion_delay(cutoff=2):
# Each tuple is (clicked, eventual_conversion, delay_days, stratum).
logs=[(1,1,1,'high'),(1,1,5,'high'),(1,0,0,'high'),(1,0,0,'low')]+[(0,0,0,'low')]*6
clicks=sum(x[0] for x in logs); eventual=sum(x[1] for x in logs)
observed=sum(c and y and d<=cutoff for c,y,d,_ in logs)
return dict(ctr=clicks/len(logs),eventual_cvr=eventual/clicks,eventual_ctcvr=eventual/len(logs),
observed_cvr=observed/clicks,unmatured_positive=eventual-observed,
clicked_high_share=sum(c and s=='high' for c,_,_,s in logs)/clicks,
exposure_high_share=sum(s=='high' for *_,s in logs)/len(logs))
Sources and next step
A-RTB response/attribution topics. Delay and selection bias are Lab probability extensions, not a complete reproduction of the original slides. This is an original Lab exercise, not a copy of the complete lecture text.
Chinese GitHub chapter · Authors’ RTB slides. The Chinese deck is not a complete modern advertising-engineering course. Historical descriptions of platforms are not presented as current market facts.
After the exercise, return to the ads learning path and continue to the next semantic unit.