06 · FM, DCN, LHUC, and FiBiNET

FM, DCN, LHUC, and FiBiNET

Goal

Turn concepts into inspectable inputs, operations, gradients, and comparisons.

Evidence: tensor and gradient mechanisms. Local teaching parameters are not production defaults; mechanism experiments are not full reproductions of the original papers.

English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.

PYTHON · 01

# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
                          if (p / "reclab").is_dir()), None)
if _translation_root is None:
    raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
    _translation_sys.path.insert(0, str(_translation_root))

Setup

On the first pass, run from Setup in order: later cells reuse earlier variables. Then change one parameter and rerun that cell and its downstream cells. Constructed data does not establish real business impact.

PYTHON · 02

from pathlib import Path
import sys
import inspect
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display, Code

PROJECT = next(p for p in [Path.cwd(), *Path.cwd().parents]
               if (p / "reclab/models.py").exists())
if str(PROJECT) not in sys.path:
    sys.path.insert(0, str(PROJECT))
from reclab.data import Dataset
from reclab.paths import ARTIFACTS, PROCESSED, read_json

plt.rcParams.update({"figure.figsize": (9, 3.8), "font.size": 11,
    "axes.spines.top": False, "axes.spines.right": False,
    "axes.labelcolor": "#182b2b", "text.color": "#182b2b",
    "axes.facecolor": "white", "figure.facecolor": "white",
    "grid.color": "#d4ddda", "axes.axisbelow": True})
dataset = Dataset()
print("Project:", PROJECT)
print("Dataset version:", dataset.manifest["dataset_version"])
print("Python:", sys.executable)
assert Path(sys.executable).parent == PROJECT / ".venv/bin", "请选择 Recommendation Lab 内核"

Saved notebook output · not executed on this page

Project: /Users/lizuolin/local/recommendation-lab
Dataset version: 2772cab0aca8cb81
Python: /Users/lizuolin/local/recommendation-lab/.venv/bin/python

Steps

Lecture references: 04_Cross_01.pdf, 04_Cross_02.pdf, 04_Cross_03.pdf, 04_Cross_04.pdf

Equivalent FM formulas and gradients

Compare explicit pair enumeration with the vectorized expression directly.

Predict first: How does the number of enumerated interactions change when the number of fields doubles?

PYTHON · 03

import torch
from reclab.learning.networks import fm_interaction,fm_explicit,CrossV2,LHUC,SEBilinear
torch.manual_seed(42);torch.set_num_threads(4)
fields=torch.randn(4,5,8,requires_grad=True)
a=fm_interaction(fields);b=fm_explicit(fields)
assert torch.allclose(a,b,atol=1e-5)
ga=torch.autograd.grad(a.sum(),fields,retain_graph=True)[0]
gb=torch.autograd.grad(b.sum(),fields)[0]
assert torch.allclose(ga,gb,atol=1e-5)
display(Code(inspect.getsource(fm_interaction),language="python"))
display(pd.DataFrame({"fast":a.detach().numpy(),"explicit":b.detach().numpy()}))

Saved notebook output · not executed on this page

def fm_interaction(fields):
    return 0.5 * ((fields.sum(1) ** 2) - (fields**2).sum(1)).sum(-1)

	fast	explicit
0	-11.419153	-11.419154
1	16.409000	16.408998
2	0.380213	0.380212
3	4.063474	4.063476

One DCN V2 layer

An operator experiment with a full-rank cross layer, without the paper's complete low-rank mixture-of-experts recipe.

Predict first: What should the output be when the affine weights and bias are zero?

PYTHON · 04

x0=torch.randn(4,6);layer=CrossV2(6)
out=layer(x0,x0)
assert torch.allclose(out,x0*(x0@layer.affine.weight.T+layer.affine.bias)+x0,atol=1e-6)
display(Code(inspect.getsource(CrossV2),language="python"))
print("Shape:",out.shape,"parameters:",sum(p.numel() for p in layer.parameters()))

Saved notebook output · not executed on this page

class CrossV2(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.affine = nn.Linear(dim, dim)

    def forward(self, x0, x):
        return x0 * self.affine(x) + x

Shape: torch.Size([4, 6]) parameters: 42

LHUC and SE / bilinear interactions

LHUC scales hidden units. The SE-style sigmoid gate and shared-W bilinear interaction are teaching variants, not complete FiBiNET or PPNet.

Predict first: Is the gate selecting fields, hidden units, or candidate items? How do these three differ?

PYTHON · 05

lhuc=LHUC(3,12); hidden=torch.randn(4,12);context=torch.randn(4,3)
adjusted,scale=lhuc(hidden,context)
assert bool(((scale>0)&(scale<2)).all())
display(pd.DataFrame(scale.detach().numpy()).round(3))
se=SEBilinear(5,8);interaction,weights=se(fields.detach())
assert interaction.shape==(4,80)
display(Code(inspect.getsource(SEBilinear),language="python"))
display(pd.DataFrame(weights.detach().numpy(),columns=[f"field{i}" for i in range(5)]))

Saved notebook output · not executed on this page

	0	1	2	3	4	5	6	7	8	9	10	11
0	0.829	0.521	0.913	0.428	0.850	1.074	1.149	1.355	0.600	0.717	1.803	0.734
1	1.117	0.623	1.122	0.954	0.745	0.915	0.585	0.833	0.784	1.520	1.131	0.801
2	1.063	0.738	0.852	0.643	1.072	0.971	1.104	1.172	0.662	0.912	1.537	0.965
3	1.308	0.725	0.870	0.947	1.008	0.607	0.899	0.576	0.783	1.333	0.870	1.054

class SEBilinear(nn.Module):
    """SE-style sigmoid field gates + shared-W bilinear interactions, not full FiBiNET."""

    def __init__(self, n_fields, dim):
        super().__init__()
        self.gate = nn.Sequential(
            nn.Linear(n_fields, max(2, n_fields // 2)),
            nn.ReLU(),
            nn.Linear(max(2, n_fields // 2), n_fields),
            nn.Sigmoid(),
        )
        self.bilinear = nn.Linear(dim, dim, bias=False)

    def forward(self, fields):
        weights = self.gate(fields.mean(-1))
        rescaled = fields * weights.unsqueeze(-1)
        pairs = [
            self.bilinear(rescaled[:, i]) * rescaled[:, j]
            for i in range(fields.shape[1])
            for j in range(i + 1, fields.shape[1])
        ]
        return torch.cat(pairs, -1), weights

	field0	field1	field2	field3	field4
0	0.479550	0.678508	0.314935	0.400290	0.639549
1	0.471220	0.686676	0.301384	0.388626	0.652943
2	0.494937	0.663158	0.340756	0.422088	0.614286
3	0.502482	0.655506	0.353774	0.432886	0.601669

Checks

Every assertion and output is traceable to code. They check operators, data snapshots, or mechanisms, not online gains.

Next steps

Record your prediction, change only one variable, and run the relevant experiment cell. Open reclab/learning/ to inspect helper functions.

06 · FM, DCN, LHUC, and FiBiNET

FM, DCN, LHUC, and FiBiNET

Goal

Setup

Steps

Equivalent FM formulas and gradients

One DCN V2 layer

LHUC and SE / bilinear interactions

Checks

Next steps

Sources for this lesson

Shusen Wang · Recommender Systems

Factorization machines

DCN V2

LHUC / PPNet

SENet and FiBiNET

FM, DCN, LHUC, and FiBiNET

Goal

Setup

Steps

Equivalent FM formulas and gradients

One DCN V2 layer

LHUC and SE / bilinear interactions

Checks

Next steps

Sources for this lesson

Shusen Wang · Recommender Systems ↗

Factorization machines ↗

DCN V2 ↗

LHUC / PPNet ↗

SENet and FiBiNET ↗

Shusen Wang · Recommender Systems

Factorization machines

DCN V2

LHUC / PPNet

SENet and FiBiNET