41 · Candidate caches, TTL, and consistency

Units: S11. Prerequisites: 40 and 26–28.

Sources: Wang Shusen’s pinned outline. This is a Lab-authored exercise. Wang's last four modules have a public outline only; this is not presented as the author's complete lecture material.

Evidence boundary: a logical clock and constructed inventory, without real cache traffic.

Learning method: predict each comparison, change one variable, run, and explain the difference.

English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.

PYTHON · 01

# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
                          if (p / "reclab").is_dir()), None)
if _translation_root is None:
    raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
    _translation_sys.path.insert(0, str(_translation_root))

PYTHON · 02

import numpy as np
import pandas as pd
from IPython.display import display
from reclab.learning.search_curriculum import *

What is cached?

Feature caches store vectors; candidate caches store document IDs; result caches include final ranking. Only candidate caching is implemented here. A hit saves loader calls; reported cost is simulated, not measured millisecond gains.

PYTHON · 03

cache=CandidateCache(ttl=5)
for t in [0,1,2,6]: print(t,cache.get('apple',t,0,lambda:['a','b']))
assert cache.cost==2
print('loader calls',cache.cost)

Saved notebook output · not executed on this page

0 (['a', 'b'], 'miss')
1 (['a', 'b'], 'hit')
2 (['a', 'b'], 'hit')
6 (['a', 'b'], 'miss')
loader calls 2

Stale caches

Delete a and advance generation. Ignoring version still yields a hit before TTL expires. Change only check_generation.

PYTHON · 04

stale=CandidateCache(); stale.get('apple',0,0,lambda:['a','b'])
bad,state=stale.get('apple',1,1,lambda:['b'],check_generation=False)
good,state2=stale.get('apple',1,1,lambda:['b'])
print('stale',bad,state,'version checked',good,state2)
assert 'a' in bad and 'a' not in good

Saved notebook output · not executed on this page

stale ['a', 'b'] hit version checked ['b'] miss

Expiration and failure fallback

Empty candidates may be cached, but short TTL limits persistent misses. A backend failure should explicitly fall back to lexical retrieval, not become an empty result.

PYTHON · 05

empty=CandidateCache(ttl=1)
print(empty.get('rare',0,0,lambda:[]))
print(empty.get('rare',2,0,lambda:['d']))
trace=pipeline('phone repair',fail_semantic=True)
assert trace['fallback']=='lexical' and trace['ranking']
print(trace)

Saved notebook output · not executed on this page

([], 'miss')
(['d'], 'miss')
{'query': 'phone repair', 'intent': 'support', 'lexical': ['c', 'd', 'a', 'b'], 'semantic': [], 'cache': 'miss', 'candidates': ['a', 'b', 'c', 'd'], 'ranking': ['c', 'd', 'a', 'b'], 'fallback': 'lexical', 'empty': False}

Cost and differences

Fix the request sequence and vary TTL only. Longer TTL increases hits but lags changes without generation checks. There is no real latency measurement or online-gain claim.

PYTHON · 06

for ttl in [1,10]:
    c=CandidateCache(ttl); results=[]
    for t in range(4): results.append(c.get('q',t,0,lambda:[str(t)])[0])
    print(ttl,'calls',c.cost,'results',results)

Saved notebook output · not executed on this page

1 calls 4 results [['0'], ['1'], ['2'], ['3']]
10 calls 1 results [['0'], ['0'], ['0'], ['0']]

Interpreting the saved run

Four accesses with TTL=5 invoke the loader twice. Ignoring generation returns deleted a; version checks refresh immediately. TTL=1 reloads on every request, while TTL=10 reloads once and keeps returning stale values. This is a constructed hit-rate/freshness tradeoff, not measured acceleration of the local real service.

After changing parameters, interpret the new outputs. This fixed commentary is not a new measurement.

Review and exercise

Write down inputs, outputs, what stayed fixed, and conclusions this chapter cannot support. Predict and rerun one single-variable comparison above; retain failures. Return to the search learning path for the next semantic unit.

41 · Candidate caches, TTL, and consistency