A17 · Ad experiments, bias, and fraud screening
Prerequisites: foundations 31–34; A16.
Evidence: explicitly constructed mechanism experiments, not real ad logs. This chapter does not claim industrial-system reproduction or online gains.
Learning order: predict → change one variable → run → explain.
English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.
# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
if (p / "reclab").is_dir()), None)
if _translation_root is None:
raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
_translation_sys.path.insert(0, str(_translation_root))
Problem and input/output contract
Exposure selection changes population mix, so aggregate CTR can differ even when within-stratum performance is identical. Request-level randomized buckets sharing campaign budgets interfere; campaign- or market-level designs must consider interference. Fraud thresholds are investigation signals, not ground truth. Prespecify anomalous-traffic exclusions and report false positives and business guardrails. Feature-hash splits prevent identical vectors crossing partitions, not user dependence or unfairness.
Before running, predict which result the changed variable will affect and which invariants must hold.
from pathlib import Path
import sys
root=Path.cwd()
if not (root/'reclab').exists(): root=root.parent
if str(root) not in sys.path: sys.path.insert(0,str(root))
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from reclab import ads_curriculum as acRun the one-variable comparison
Inputs are small constructed examples visible in source. Output units and constraints are defined above. Run assertions to verify mathematics and accounting.
a=ac.experiment_bias();print(a)
assert a['aggregate']['A']!=a['aggregate']['B']
assert np.isclose(a['standardized']['A'],a['standardized']['B'])
assert a['false_positive']==1
print('筛查precision',a['true_positive']/(a['true_positive']+a['false_positive']))
print('共享预算先到先得',ac.shared_budget_interference(),ac.shared_budget_interference(('B','B','A','A')))
assert ac.shared_budget_interference()['B']==0Saved notebook output · not executed on this page
{'aggregate': {'A': 0.19, 'B': 0.11}, 'standardized': {'A': 0.15000000000000002, 'B': 0.15000000000000002}, 'flagged': ['bot', 'campaign_burst'], 'true_positive': 1, 'false_positive': 1, 'interference': 'Shared campaign budgets couple request buckets; use campaign/market randomization or model spillovers.'}
筛查precision 0.5
共享预算先到先得 {'A': 2, 'B': 0} {'A': 0, 'B': 2}
Interpretation and limits
Using common stratum weights removes the difference in this constructed case. That does not prove every real effect comes from mixture; it is a counterexample to naive aggregate comparisons. A high-frequency rule falsely flags a legitimate campaign burst. Unknown labels cannot establish real fraud-detection accuracy.
Inspect source, then try an exercise
Design the primary metric, budget and user-experience guardrails, randomization unit, and stopping rule. Simulate how shared-budget exhaustion contaminates request buckets.
import inspect
print(inspect.getsource(ac.experiment_bias))Saved notebook output · not executed on this page
def experiment_bias():
# Group A/B have identical stratum rates; unequal mix creates aggregate difference.
counts={'A':[(90,18),(10,1)],'B':[(10,2),(90,9)]}
aggregate={k:sum(y for n,y in cells)/sum(n for n,y in cells) for k,cells in counts.items()}
standardized={k:sum(y/n for n,y in cells)/2 for k,cells in counts.items()}
# A rule is only a triage signal; legitimate burst is a false positive.
traffic=[('bot',50,True),('human',1,False),('campaign_burst',40,False)]
flags=[name for name,rate,_ in traffic if rate>20]
return dict(aggregate=aggregate,standardized=standardized,flagged=flags,true_positive=1,false_positive=1,
interference='Shared campaign budgets couple request buckets; use campaign/market randomization or model spillovers.')
Sources and next step
Other-technology topics in the A-LIU outline. The stratification counterexample, interference, and false-positive examples are Lab-authored. This is an original Lab exercise, not a copy of the complete lecture text.
Chinese GitHub chapter · Authors’ RTB slides. The Chinese deck is not a complete modern advertising-engineering course. Historical descriptions of platforms are not presented as current market facts.
After the exercise, return to the ads learning path and continue to the next semantic unit.