11 · A real competition: from the task to one prediction

Goal

Recommend products in sparse-interaction target markets t1 and t2 using shared-item information from s1, s2, and s3. Read the full official dataset and rank 100 supplied candidates. Each user has one validation positive; the success metric is nDCG@10.

This work follows the public OPDAI winning solution. The winner's 0.6773 on the official leaderboard is the historical hidden-test score. Local validation scores are never presented as that score. This reproduces core algorithms and the two-stage workflow with smaller graph models, feature sets, and ensembles. See reproduction notes for all differences.

Order: 12 Features → 13 Sampling and graph models → 14 Retrain the ranker → 15 Evaluation.

Setup

Use the project's .venv kernel. Run python -m reclab.competition prepare once to download inputs; the saved reference run used downloaded, checksum-verified data.

English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.

PYTHON · 01

# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
                          if (p / "reclab").is_dir()), None)
if _translation_root is None:
    raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
    _translation_sys.path.insert(0, str(_translation_root))

PYTHON · 02

from pathlib import Path
import sys, json, inspect
ROOT = Path.cwd()
if not (ROOT / "reclab").is_dir():
    ROOT = ROOT.parent
sys.path.insert(0, str(ROOT))
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display, Code
from reclab.competition import data
from reclab.competition.teaching import load_run, style, bar_scores
style()
pd.set_option("display.max_columns", 12)
pd.set_option("display.max_rows", 40)
run, rows, features = load_run()
print("实际训练快照:", run.name)

Saved notebook output · not executed on this page

Matplotlib is building the font cache; this may take a moment.

实际训练快照: 20261030T115556803593-cloud-restore

Step 1 · Decide which files may enter training

train.tsv contains rated interactions. train_5core.tsv contains filtered positive feedback, not unclicked impressions. The winning solution sets 5core ratings to 5, deduplicates user–item pairs, and gives original train records priority. Target-market qrels must not enter the feature-training corpus. Official hidden-test labels were not released.

PYTHON · 03

summary = []
for market in data.MARKETS:
    train = data.interactions(market)
    combined = data.corpus([market])
    summary.append({"market": market, "raw_train_rows": len(train),
                    "duplicate_user_item_pairs": int(train.duplicated(["userId", "itemId"]).sum()),
                    "merged_rows": len(combined), "users": combined.userId.nunique(),
                    "items": combined.itemId.nunique()})
summary = pd.DataFrame(summary)
display(summary)
display(data.interactions("t1").head(5))

Saved notebook output · not executed on this page

	market	raw_train_rows	duplicate_user_item_pairs	merged_rows	users	items
0	s1	840212	39564	809095	141508	35518
1	s2	115549	1147	130077	20327	5172
2	s3	54202	789	57113	8627	2922
3	t1	60168	1392	60400	9955	3543
4	t2	120320	1690	120178	18504	8919

	userId	itemId	rating
0	t1U1006129	P1016853	5.0
1	t1U1002135	P1019605	5.0
2	t1U1006710	P1032538	5.0
3	t1U1007418	P1012632	5.0
4	t1U1008062	P1022775	5.0

PYTHON · 04

fig, ax = plt.subplots(figsize=(8, 3))
ax.bar(summary.market, summary.merged_rows, color="#006e60")
ax.set(xlabel="Market", ylabel="Deduplicated interactions", title="Official training corpus by market")
ax.grid(axis="y", alpha=.15)
plt.show()

Saved notebook output · not executed on this page

<Figure size 880x330 with 1 Axes>

Step 2 · Follow a real user

A row is one competition-supplied candidate, not an impression. label=0 means it is not the held-out positive for this evaluation, not that the user genuinely dislikes it. User and item IDs are anonymized. There are no timestamps, product titles, prices, or interpretable country names. Features such as "popularity in the last seven days" cannot be invented here.

PYTHON · 05

user = rows.query("market == 't1'").userId.iloc[0]
example = rows[(rows.market == "t1") & (rows.userId == user)]
known = data.corpus(["t1"])
display(known[known.userId.eq(user)].head(10))
display(example.head(8))
print("候选数:", len(example), "正例数:", int(example.label.sum()))
print("正例 ID:", example.loc[example.label.eq(1), "itemId"].iloc[0])

Saved notebook output · not executed on this page

	userId	itemId	rating
6380	t1U1005570	P1030052	5.0
7115	t1U1005570	P1021106	5.0
9808	t1U1005570	P1014278	5.0
12074	t1U1005570	P1020517	5.0
13319	t1U1005570	P1031263	4.0

	userId	itemId	market	partition
0	t1U1005570	P1005811	t1	fit
1	t1U1005570	P1035751	t1	fit
2	t1U1005570	P1007542	t1	fit
3	t1U1005570	P1035694	t1	fit
4	t1U1005570	P1006599	t1	fit
5	t1U1005570	P1012377	t1	fit
6	t1U1005570	P1026761	t1	fit
7	t1U1005570	P1004549	t1	fit

候选数: 100 正例数: 1
正例 ID: P1026107

Step 3 · Trace the actual run's provenance

After generating full-data features, a stable hash divides official validation users into approximately 60% fit, 20% select, and 20% audit. Fit trains the ranker; select chooses the model; audit is inspected only after selection. Every candidate row stays with its user. The original solution used ten-fold row-level bagging; this local workflow instead isolates users.

PYTHON · 06

receipt = json.loads((run / "receipt.json").read_text())
display(pd.Series(receipt))
display(rows.groupby(["market", "partition"]).userId.nunique().unstack())
assert rows.groupby(["market", "userId"]).partition.nunique().eq(1).all()
source = json.loads(data.SPEC.read_text())
print("官方数据版本:", source["official_revision"])
print("冠军源码版本:", source["winner_revision"])
print("原始文件校验数:", len(source["files"]))

Saved notebook output · not executed on this page

run_id                  20261030T115556803593-cloud-restore
selected                                         lgb_memory
seconds                                          164.998926
peak_rss_mib                                    1782.796875
official_valid_users                                   8179
features                                                 21
status                                                 PASS
hidden_test_score                                      None
lightgbm_threads                                          1
dtype: object

partition	audit	fit	select
market
t1	543	1616	538
t2	1062	3267	1153

官方数据版本: f5b1f087a4054537cd8f46bf346f978094b8467e
冠军源码版本: 8af3260368b1980a85a555fc470caa84bf6697d6
原始文件校验数: 22

Check your understanding

Why would adding qrel positives to t1's interaction matrix improve scores when evaluating the same users?

Reasoning

The model has already seen its evaluation targets; this is not better generalization. Cross-market transfer shares item relationships from known training interactions, not target answers inserted into the graph.

## Next step Open 12. Predict first: must more source-market data improve both t1 and t2? Check against actual scores later.

11 · A real competition: from the task to one prediction