31 · Linear algebra and tensor shapes
Foundations path: 31 → 32 → 33 → 34, then choose search, ads, or recommendation.
Scope: constructed small-array mechanism experiments on CPU; no real logs, formal model training, or business-impact claims.
For each section: predict → change one variable → run → explain. Execute from the beginning first; after edits, rerun affected downstream cells.
English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.
# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
if (p / "reclab").is_dir()), None)
if _translation_root is None:
raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
_translation_sys.path.insert(0, str(_translation_root))
Inputs and linear maps
X is [samples,features]; W is [features,outputs]. Matrix multiplication sums over the shared feature axis. Predict output shape before running.
import torch
import matplotlib.pyplot as plt
torch.set_num_threads(1)
X = torch.tensor([[1., 2.], [3., 4.], [0., 1.]])
W = torch.tensor([[2.], [-1.]])
scores = X @ W
assert scores.shape == (3, 1)
assert torch.equal(scores, (X * W.T).sum(1, keepdim=True))
print('X, W, scores:', X.shape, W.shape, scores.flatten().tolist())Saved notebook output · not executed on this page
X, W, scores: torch.Size([3, 2]) torch.Size([2, 1]) [0.0, 2.0, -1.0]
Change one variable: feature units
Scale the first feature by ten, leaving weights unchanged. Predict whether scores remain the same.
scaled = X.clone()
scaled[:, 0] *= 10
changed = scaled @ W
assert not torch.equal(scores, changed)
print(torch.cat([scores, changed], dim=1))Saved notebook output · not executed on this page
tensor([[ 0., 18.],
[ 2., 56.],
[-1., -1.]])
A broadcasting trap
Subtracting [3] from [3,1] runs successfully but produces [3,3]. Align label axes explicitly.
y = torch.tensor([0., 1., 0.])
wrong = scores - y
right = scores.squeeze(-1) - y
assert wrong.shape == (3, 3) and right.shape == (3,)
print('错误广播 / 正确逐样本残差:', wrong.shape, right.tolist())Saved notebook output · not executed on this page
错误广播 / 正确逐样本残差: torch.Size([3, 3]) [0.0, 1.0, -1.0]
Compensate for units and inspect the plot
Now divide only the corresponding weight in the new coordinates by ten; verify the linear relationship.
compensated = W.clone()
compensated[0] /= 10
assert torch.allclose(scaled @ compensated, scores)
plt.plot(scores.flatten(), 'o-', label='original')
plt.plot(changed.flatten(), 'x--', label='feature x10')
plt.xlabel('constructed sample'); plt.ylabel('score'); plt.legend(); plt.show()Saved notebook output · not executed on this page
<Figure size 640x480 with 1 Axes>
Explain
Why can changing units change an inner product? Why can valid broadcasting change the meaning of a loss? Before expanding two feature columns to four, list every shape that must change.
In the next cell, record your prediction, actual result, reason for any difference, and applicability limits. Keep failed results.
reflection = {'预测': '', '结果': '', '解释与边界': ''}
reflectionSaved notebook output · not executed on this page
{'预测': '', '结果': '', '解释与边界': ''}