46 · End-to-end traces, failures, and course review
Units: S17, S18. Prerequisites: 35–45 and 25–30.
Sources: Wang Shusen’s pinned outline. This is a Lab-authored exercise. Wang's last four modules have a public outline only; this is not presented as the author's complete lecture material.
Evidence boundary: constructed trace API and formal SciFact API kept separate within the same service.
Learning method: predict each comparison, change one variable, run, and explain the difference.
English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.
# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
if (p / "reclab").is_dir()), None)
if _translation_root is None:
raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
_translation_sys.path.insert(0, str(_translation_root))
import numpy as np
import pandas as pd
from IPython.display import display
from reclab.learning.search_curriculum import *The normal flow
Input → normalization/rewriting → intent → lexical/mechanism-semantic candidates → candidate cache → field reranking. Trace each step. The mechanism-semantic channel is not chapter 27's neural encoder.
cache=CandidateCache()
t=pipeline('cellphone repair',cache); display(t)
assert t['query']=='phone repair' and t['ranking']Saved notebook output · not executed on this page
{'query': 'phone repair',
'intent': 'support',
'lexical': ['c', 'd', 'a', 'b'],
'semantic': ['a', 'c', 'd'],
'cache': 'miss',
'candidates': ['a', 'b', 'c', 'd'],
'ranking': ['c', 'd', 'a', 'b'],
'fallback': None,
'empty': False}Hits and degradation
Change only the failure flag. Callers must recognize lexical fallback. Cache keys require channel/model versions or failures may reuse candidates from a different policy. This demonstration uses separate caches for different policies.
print(pipeline('phone repair',cache,now=1))
fallback=pipeline('phone repair',fail_semantic=True)
print(fallback)
assert fallback['fallback']=='lexical'
empty=pipeline('unfindable'); assert empty['empty']; print(empty)Saved notebook output · not executed on this page
{'query': 'phone repair', 'intent': 'support', 'lexical': ['c', 'd', 'a', 'b'], 'semantic': ['a', 'c', 'd'], 'cache': 'hit', 'candidates': ['a', 'b', 'c', 'd'], 'ranking': ['c', 'd', 'a', 'b'], 'fallback': None, 'empty': False}
{'query': 'phone repair', 'intent': 'support', 'lexical': ['c', 'd', 'a', 'b'], 'semantic': [], 'cache': 'miss', 'candidates': ['a', 'b', 'c', 'd'], 'ranking': ['c', 'd', 'a', 'b'], 'fallback': 'lexical', 'empty': False}
{'query': 'unfindable', 'intent': 'abstain', 'lexical': [], 'semantic': [], 'cache': 'miss', 'candidates': [], 'ranking': [], 'fallback': None, 'empty': True}
A teaching endpoint in the same FastAPI service
/api/search/workshop/trace explicitly labels constructed data and does not replace the formal /api/search model. Each server request has an independent cache to avoid misleading cross-request state. Only the first two notebook cells test cross-request caching.
import urllib.request,json
payload=json.dumps({'query':'battrey repair','fail_semantic':True}).encode()
req=urllib.request.Request('http://127.0.0.1:18580/api/search/workshop/trace',data=payload,headers={'Content-Type':'application/json'})
with urllib.request.urlopen(req,timeout=20) as response: result=json.load(response)
assert result['evidence']=='constructed mechanism' and result['trace']['fallback']=='lexical'
print(result)Saved notebook output · not executed on this page
{'evidence': 'constructed mechanism', 'scope': 'Four artificial documents; rule semantic substitute; request-local cache; official SciFact service unchanged.', 'trace': {'query': 'battery repair', 'intent': 'support', 'lexical': ['c', 'd', 'a', 'b'], 'semantic': [], 'cache': 'miss', 'candidates': ['a', 'b', 'c', 'd'], 'ranking': ['c', 'd', 'a', 'b'], 'fallback': 'lexical', 'empty': False}}
Review matrix
Review needs/judgments → query understanding → retrieval → inference/training → ranking → suggestions → serving. Chapter 17 is an integrated reference, not a prerequisite. The old 300-query test is diagnostic only, not a tuning set for new models.
from pathlib import Path
curriculum=json.loads(Path('docs/curriculum.json').read_text())
for row in curriculum['search_units']: print(row['id'],row['topic'],row['chapters'])
assert len(curriculum['search_units'])==18Saved notebook output · not executed on this page
S01 任务、用户需求与搜索产品 [35] S02 相关性分档、标注与评估设计 [35, 25] S03 文本规范化、中文分词与新词 [36] S04 查询词权重与字段匹配 [37] S05 类目与意图识别 [38] S06 查询改写、纠错与语义漂移 [39] S07 倒排索引、布尔与位置约束 [40] S08 词法相关性与BM25 [26] S09 语义表示与向量召回 [27] S10 近似索引与候选预算 [28] S11 缓存召回与一致性 [41] S12 BERT相关性打分与推理 [42, 29] S13 相关性模型训练 [42] S14 排序目标与多路融合训练 [43, 29] S15 质量、多样性与业务约束 [44] S16 查询推荐与建议词 [45] S17 服务链路、误差定位与性能 [46, 30] S18 综合复盘与课程导航 [46]
Interpreting the saved run
cellphone is corrected to phone with traceable candidates, caching, and ranking. The second call hits cache; a semantic-channel failure explicitly reports lexical fallback; unknown words yield empty=true. The teaching endpoint in the same service reports constructed mechanism. Four-document examples do not replace chapter 30’s real SciFact service-consistency evidence.
After changing parameters, interpret the new outputs. This fixed commentary is not a new measurement.
Review and exercise
Write down inputs, outputs, what stayed fixed, and conclusions this chapter cannot support. Predict and rerun one single-variable comparison above; retain failures. Return to the search learning path for the next semantic unit.