FM、DCN、LHUC 与 FiBiNET
Goal
把概念变成可检查的输入、运算、梯度与对照。
数据类型：张量与梯度机制。本地教学参数不是生产默认值；机制实验不是原论文完整复现。
Setup
每本首次从 Setup 起顺序运行，后面的格会复用前面的变量。之后只改一个参数，重跑对应格及下游格。不要把构造数据当作真实业务收益。
PYTHON · 01
from pathlib import Path
import sys
import inspect
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display, Code
PROJECT = next(p for p in [Path.cwd(), *Path.cwd().parents]
if (p / "reclab/models.py").exists())
if str(PROJECT) not in sys.path:
sys.path.insert(0, str(PROJECT))
from reclab.data import Dataset
from reclab.paths import ARTIFACTS, PROCESSED, read_json
plt.rcParams.update({"figure.figsize": (9, 3.8), "font.size": 11,
"axes.spines.top": False, "axes.spines.right": False,
"axes.labelcolor": "#182b2b", "text.color": "#182b2b",
"axes.facecolor": "white", "figure.facecolor": "white",
"grid.color": "#d4ddda", "axes.axisbelow": True})
dataset = Dataset()
print("Project:", PROJECT)
print("Dataset version:", dataset.manifest["dataset_version"])
print("Python:", sys.executable)
assert Path(sys.executable).parent == PROJECT / ".venv/bin", "请选择 Recommendation Lab 内核"Notebook 保存的输出 · 本页未执行
Project: /Users/lizuolin/local/recommendation-lab Dataset version: 2772cab0aca8cb81 Python: /Users/lizuolin/local/recommendation-lab/.venv/bin/python
Steps
讲义对应：04_Cross_01.pdf, 04_Cross_02.pdf, 04_Cross_03.pdf, 04_Cross_04.pdf
FM 的等价公式与梯度
直接核对枚举交互和向量化表达式。
先预测： 字段数翻倍后，枚举交互数量怎样变化？
PYTHON · 02
import torch
from reclab.learning.networks import fm_interaction,fm_explicit,CrossV2,LHUC,SEBilinear
torch.manual_seed(42);torch.set_num_threads(4)
fields=torch.randn(4,5,8,requires_grad=True)
a=fm_interaction(fields);b=fm_explicit(fields)
assert torch.allclose(a,b,atol=1e-5)
ga=torch.autograd.grad(a.sum(),fields,retain_graph=True)[0]
gb=torch.autograd.grad(b.sum(),fields)[0]
assert torch.allclose(ga,gb,atol=1e-5)
display(Code(inspect.getsource(fm_interaction),language="python"))
display(pd.DataFrame({"fast":a.detach().numpy(),"explicit":b.detach().numpy()}))Notebook 保存的输出 · 本页未执行
def fm_interaction(fields):
return 0.5 * ((fields.sum(1) ** 2) - (fields**2).sum(1)).sum(-1)| fast | explicit | |
|---|---|---|
| 0 | -11.419153 | -11.419154 |
| 1 | 16.409000 | 16.408998 |
| 2 | 0.380213 | 0.380212 |
| 3 | 4.063474 | 4.063476 |
DCN V2 单层
full-rank cross 的算子实验，不含原论文完整低秩 MoE 配方。
先预测： 将 affine 的权重和偏置设成零，输出应是什么？
PYTHON · 03
x0=torch.randn(4,6);layer=CrossV2(6)
out=layer(x0,x0)
assert torch.allclose(out,x0*(x0@layer.affine.weight.T+layer.affine.bias)+x0,atol=1e-6)
display(Code(inspect.getsource(CrossV2),language="python"))
print("Shape:",out.shape,"parameters:",sum(p.numel() for p in layer.parameters()))Notebook 保存的输出 · 本页未执行
class CrossV2(nn.Module):
def __init__(self, dim):
super().__init__()
self.affine = nn.Linear(dim, dim)
def forward(self, x0, x):
return x0 * self.affine(x) + xShape: torch.Size([4, 6]) parameters: 42
LHUC 与 SE / Bilinear
LHUC 缩放隐藏单元；SE-style sigmoid gate 与共享 W 的 bilinear 是教学变体，不是完整 FiBiNET/PPNet。
先预测： 门控是在选择字段、隐藏单元，还是候选物品？三者如何区分？
PYTHON · 04
lhuc=LHUC(3,12); hidden=torch.randn(4,12);context=torch.randn(4,3)
adjusted,scale=lhuc(hidden,context)
assert bool(((scale>0)&(scale<2)).all())
display(pd.DataFrame(scale.detach().numpy()).round(3))
se=SEBilinear(5,8);interaction,weights=se(fields.detach())
assert interaction.shape==(4,80)
display(Code(inspect.getsource(SEBilinear),language="python"))
display(pd.DataFrame(weights.detach().numpy(),columns=[f"field{i}" for i in range(5)]))Notebook 保存的输出 · 本页未执行
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.829 | 0.521 | 0.913 | 0.428 | 0.850 | 1.074 | 1.149 | 1.355 | 0.600 | 0.717 | 1.803 | 0.734 |
| 1 | 1.117 | 0.623 | 1.122 | 0.954 | 0.745 | 0.915 | 0.585 | 0.833 | 0.784 | 1.520 | 1.131 | 0.801 |
| 2 | 1.063 | 0.738 | 0.852 | 0.643 | 1.072 | 0.971 | 1.104 | 1.172 | 0.662 | 0.912 | 1.537 | 0.965 |
| 3 | 1.308 | 0.725 | 0.870 | 0.947 | 1.008 | 0.607 | 0.899 | 0.576 | 0.783 | 1.333 | 0.870 | 1.054 |
class SEBilinear(nn.Module):
"""SE-style sigmoid field gates + shared-W bilinear interactions, not full FiBiNET."""
def __init__(self, n_fields, dim):
super().__init__()
self.gate = nn.Sequential(
nn.Linear(n_fields, max(2, n_fields // 2)),
nn.ReLU(),
nn.Linear(max(2, n_fields // 2), n_fields),
nn.Sigmoid(),
)
self.bilinear = nn.Linear(dim, dim, bias=False)
def forward(self, fields):
weights = self.gate(fields.mean(-1))
rescaled = fields * weights.unsqueeze(-1)
pairs = [
self.bilinear(rescaled[:, i]) * rescaled[:, j]
for i in range(fields.shape[1])
for j in range(i + 1, fields.shape[1])
]
return torch.cat(pairs, -1), weights| field0 | field1 | field2 | field3 | field4 | |
|---|---|---|---|---|---|
| 0 | 0.479550 | 0.678508 | 0.314935 | 0.400290 | 0.639549 |
| 1 | 0.471220 | 0.686676 | 0.301384 | 0.388626 | 0.652943 |
| 2 | 0.494937 | 0.663158 | 0.340756 | 0.422088 | 0.614286 |
| 3 | 0.502482 | 0.655506 | 0.353774 | 0.432886 | 0.601669 |
Checks
所有断言和输出都可回溯到代码。它们验证算子、数据快照或机制，不是线上收益。
Next Steps
先记录预测，只改一个变量，再运行对应实验格。需要查看辅助函数时，打开 reclab/learning/。