Chapter 21Evaluating Generated Text

The measurement problem

Chapter 14 evaluated a classifier, and the job was straightforward. There is one right label, the model gives one label, and you compare.

Generation breaks that. There are many correct outputs, they do not resemble each other as strings, and nobody can enumerate them.

Take a translation of “Il pleut”. It is raining, It’s raining, Rain is falling are all correct. A metric that credits only the first is measuring the wrong thing.

Everything in this chapter is an attempt to work around that, and none of the attempts fully succeeds.

When the answer is a label

Where the output space is closed, use Chapter 14’s machinery unchanged. Per-class precision, recall and $F_1$, macro-averaged, with a confusion matrix to say what is failing.

Two cautions carry over from there. Accuracy hides which class is wrong, and micro-$F_1$ equals accuracy on single-label tasks, so reporting both tells you nothing twice.

The rest of the chapter is about the open case.

Overlap metrics

The oldest workable idea is to compare $n$-grams against one or more references.

Precision has to be clipped

Start with unigram precision: what fraction of the candidate’s words appear in a reference?

It is trivially gameable. Take the references “the cat is on the mat” and “there is a cat on the mat”, and the candidate:

Every one of those seven words appears in a reference, so unclipped precision is $7/7 = 1.0000$. A perfect score for a sentence that says nothing.

Modified precision caps each $n$-gram at the most any single reference contains. Reference one has the twice, so the ceiling is two.

Clipping is not a refinement. Without it the metric can be won by repeating one word, so it would measure nothing at all.

The brevity penalty

Precision still rewards saying less. Against the reference “the cat is on the mat”:

The first row is one word, perfectly precise, and useless. Precision alone would rank it top.

The brevity penalty fixes it:

$$\begin{equation} \text{BP} \;=\; \begin{cases} 1, & c > r, \\ \exp\!\big(1 - r/c\big), & c \le r, \end{cases} \label{eq:bp} \quad\text{(21.1)} \end{equation}$$

where $c$ is the candidate length and $r$ the closest reference length.

It is deliberately asymmetric. Too short is punished. Too long is not, because a long candidate is already punished by precision, as the last row shows.

BLEU, end to end

BLEU combines clipped precisions for $n = 1$ to $4$, using their geometric mean, times the brevity penalty.

Three things that table shows.

The geometric mean means one zero sends the whole score to zero. The third candidate differs from a reference by a single word and scores $0$.

That severity is why BLEU is reported over a corpus rather than a sentence. On one sentence it is close to useless.

The last candidate is the reference words in scrambled order. Its unigram precision is perfect and its higher $n$-grams collapse. That is the only way BLEU sees word order at all.

ROUGE, and the other direction

BLEU asks how much of what you said was in the reference. That is precision, and it suits translation.

Summarisation cares about the opposite. Did you leave out the important part? That is recall, and it is what ROUGE measures.

Row three is padded, and ROUGE gives it a perfect score because every reference $n$-gram is present. Recall alone can be won by saying everything, which is why ROUGE is normally reported with an $F$-measure variant.

The row that matters

Row four is a correct paraphrase of the reference. It shares no content word with it.

BLEU gives it $0.0000$. ROUGE-2 gives it $0.0000$. ROUGE-1 gives it $0.3333$, and every point of that comes from the word the.

Not one content word was credited, and the candidate is right.

Neither metric has any notion of meaning. They count string overlap, and a paraphrase overlaps in the wrong places. That single row is the argument for everything that came after.

Try it yourself. code/worked_examples/bleu.py produces every table here. --clip builds the degenerate candidate, --brevity sweeps the length term, --bleu scores four candidates in full, and --rouge runs the paraphrase.

Run it in ColabNotebookSource

Beyond surface overlap

Two families of response, and both trade a known weakness for a new one.

Embedding-based metrics compare meaning rather than strings. BERTScore matches candidate and reference tokens by the cosine similarity of their contextual embeddings from Chapter 15.

The paraphrase row above now scores well, which is the point. The cost is that the metric is now a model, with its own training data and its own blind spots, and its number cannot be recomputed by hand.

Model-as-judge asks a strong language model to rate an output against a rubric .

It handles open-ended tasks nothing else can score, and it correlates reasonably with human preference. It also inherits every bias of the judge model, including a documented preference for longer answers and for its own outputs.

A judge is a measuring instrument that has opinions. It is useful, and it must be calibrated against human judgement rather than trusted.

Human evaluation

Human judgement remains the standard everything else is validated against, and it has its own methodology.

Pairwise comparison is more reliable than absolute rating, because people are better at ranking two things than at scoring one. Multiple annotators with a reported agreement statistic are necessary, because a single annotator’s number is unmeasurable.

It is slow and expensive, which is why the field keeps trying to automate it and keeps having to come back.

The metric that becomes the target

One closing caution, and it applies to every number in this chapter.

When a measure becomes a target, it ceases to be a good measure.

That is Goodhart’s law, in Strathern’s phrasing , and benchmark history is a long demonstration of it.

Benchmarks saturate. Test sets leak into pretraining corpora. Models are tuned against the leaderboard until the leaderboard stops measuring the thing it was built to measure.

The defence is not a better metric. It is the discipline of Chapter 14: state your metric, hold out your test set, report the runs that failed, and look at the actual outputs.

A number is a summary of an experiment. It is not a substitute for having looked.

Further reading.

is BLEU and is ROUGE. introduce BERTScore. study language models as judges and document their biases carefully. cover evaluation across tasks with more examples.