43 · Pointwise, pairwise, and listwise objectives

Units: S14. Prerequisites: 29, 32–34, 42.

Sources: Wang Shusen’s pinned outline. This is a Lab-authored exercise. Wang's last four modules have a public outline only; this is not presented as the author's complete lecture material.

Evidence boundary: constructed graded labels and actual gradient optimization; chapter 29 retains real multi-feature fusion training.

Learning method: predict each comparison, change one variable, run, and explain the difference.

English companion. All instructional Markdown is translated. Experiment code, Chinese language examples, execution counts, and saved outputs remain as in the source edition; historical output text may therefore be Chinese. The initial setup cell anchors the working directory and Python import path for this companion's deeper folder; it does not change the experiment. Saved outputs are historical evidence, not a new run. Run from the top after preparing the required data and models.

PYTHON · 01

# English companions live one directory deeper; keep the original experiment code unchanged.
from pathlib import Path as _TranslationPath
import os as _translation_os
import sys as _translation_sys
_translation_root = next((p for p in [_TranslationPath.cwd(), *_TranslationPath.cwd().parents]
                          if (p / "reclab").is_dir()), None)
if _translation_root is None:
    raise RuntimeError("Open this notebook from inside the Search Recommendation Ads Lab checkout.")
_translation_os.chdir(_translation_root)
if str(_translation_root) not in _translation_sys.path:
    _translation_sys.path.insert(0, str(_translation_root))

PYTHON · 02

import numpy as np
import pandas as pd
from IPython.display import display
from reclab.learning.search_curriculum import *

Three objective families

Pointwise fits each label; pairwise fits ordered pairs; listwise models a distribution over the candidate list. Fix the same three candidates and vary loss only, with actual gradient training. This is not LambdaMART.

PYTHON · 03

import torch
torch.manual_seed(42)
x=torch.tensor([[1.,0.],[0.,1.],[.8,.4]])
y=torch.tensor([3.,0.,2.])
print('features',x.shape,'labels',y.tolist())

Saved notebook output · not executed on this page

features torch.Size([3, 2]) labels [3.0, 0.0, 2.0]

Objective implementations

Pointwise MSE fits grades; pairwise softplus(−(s_i−s_j)) applies when y_i>y_j; listwise cross-entropy targets softmax(y). Scaling labels changes pointwise and listwise objectives; pairwise uses order only.

PYTHON · 04

def objective(s,y,kind):
    if kind=='pointwise': return ((s-y)**2).mean()
    if kind=='pairwise':
        mask=y[:,None]>y[None,:]
        return torch.nn.functional.softplus(-(s[:,None]-s[None,:])[mask]).mean()
    return -(torch.softmax(y,0)*torch.log_softmax(s,0)).sum()
for kind in ['pointwise','pairwise','listwise']:
    print(kind,float(objective(torch.zeros(3),y,kind)))

Saved notebook output · not executed on this page

pointwise 4.333333492279053
pairwise 0.6931471824645996
listwise 1.0986123085021973

Real optimization with a fixed budget

Use the same initialization, steps, and learning rate. Different loss scales mean a shared learning rate is not necessarily an optimally matched configuration. Rankings may coincide; retain ties and negative results.

PYTHON · 05

rows=[]
for kind in ['pointwise','pairwise','listwise']:
    w=torch.zeros(2,requires_grad=True); opt=torch.optim.SGD([w],lr=.1)
    for step in range(60):
        opt.zero_grad(); loss=objective(x@w,y,kind); loss.backward(); opt.step()
    scores=(x@w).detach().numpy(); order=np.argsort(-scores)
    rows.append({'objective':kind,'loss':float(loss.detach()),'scores':scores.tolist(),'order':order.tolist(),'ndcg':graded_ndcg(order.tolist(),{0:3,1:0,2:2})})
display(pd.DataFrame(rows))

Saved notebook output · not executed on this page

	objective	loss	scores	order	ndcg
0	pointwise	0.029805	[2.8116273880004883, -0.07029175013303757, 2.2...	[0, 2, 1]	1.0
1	pairwise	0.237659	[1.0923994779586792, -1.136237382888794, 0.419...	[0, 2, 1]	1.0
2	listwise	0.756533	[0.8868535757064819, -0.986713707447052, 0.314...	[0, 2, 1]	1.0

Ranking metrics and loss can disagree

Adding a constant to all scores preserves ranking. Pairwise/listwise losses remain unchanged, but MSE changes. Verify the invariant, then check in chapter 29 that fusion scalers fit training queries only.

PYTHON · 06

s=torch.tensor([2.,0.,1.])
for kind in ['pointwise','pairwise','listwise']:
    a=objective(s,y,kind); b=objective(s+10,y,kind)
    print(kind,float(a),float(b))
    if kind!='pointwise': assert torch.allclose(a,b,atol=1e-5)

Saved notebook output · not executed on this page

pointwise 0.6666666865348816 87.33333587646484
pairwise 0.2511504590511322 0.2511504590511322
listwise 0.7373405694961548 0.7373405694961548

Interpreting the saved run

Loss scales differ, but after 60 steps all three objectives yield [0,2,1] and NDCG=1. Do not select by absolute training loss. Adding 10 to every score preserves order and pairwise/listwise losses, while MSE changes from about 0.667 to 87.333. This exposes the difference between fitting scores and optimizing order.

After changing parameters, interpret the new outputs. This fixed commentary is not a new measurement.

Review and exercise

Write down inputs, outputs, what stayed fixed, and conclusions this chapter cannot support. Predict and rerun one single-variable comparison above; retain failures. Return to the search learning path for the next semantic unit.

43 · Pointwise, pairwise, and listwise objectives