Accuracy is not enough

Accuracy is the share of rows the model got right. It is the first number everyone reaches for, it is easy to explain, and on its own it is one of the most misleading numbers in the whole field. Here is the trap, and it needs no complicated model to spring. It is a medical test for a rare disease in a city of a thousand people.

disease screening, 1000 people, 10 are ill

model: always answer no

correct: 990 healthy people
wrong:   10 ill people missed

accuracy = 990 / 1000 = 99 percentA model that does nothing at all, and scores 99 percent.

That model answers no for every single person. It has learned nothing, it will never find a single patient, and it still scores 99 percent, because the disease is rare and the healthy people are the majority. Accuracy quietly rewards the majority class. Whenever one answer is much more common than the other, a lazy model that always answers the common one looks excellent on paper while being worthless in practice.

Now the second trap, which is about honesty rather than rarity. A model can be checked on the very rows it learned from, the way a pupil could be handed back the worksheet with the answers still on it. Of course the score is high. The number you actually want comes from rows the model has never met. So before training you split the table: one part trains the model, the rest is sealed and opened once, at the end. That sealed part is the test set.

Testing on training data is not a small mistake, it is the whole measurement being thrown away. If you tune the model, look at the test score, and tune again, the test set gradually stops being unseen and starts being memorised. Professionals split the data into three parts for this reason: training, validation for the tuning, and a test set touched exactly once. Touch it more often and its number stops meaning anything.

spam filter, 100 emails, 20 are spam

+----------------+----------------+----------------+
|                | says spam      | says inbox     |
+----------------+----------------+----------------+
| is spam        | 16             | 4              |
| is not spam    | 3              | 77             |
+----------------+----------------+----------------+

accuracy  = (16 + 77) / 100 = 93 percent
precision = 16 / 19         = 84 percent
recall    = 16 / 20         = 80 percentEvery prediction of a spam filter falls into exactly one of these four boxes.

The four boxes are the confusion matrix, and the two useful numbers are read straight off it. Precision asks: of everything the model flagged as spam, what share really was spam? Low precision means innocent mail is being destroyed, which is exactly the complaint users make. Recall asks: of all the spam that existed, what share did the model catch? Low recall means spam keeps arriving, which is the complaint nobody bothers to send. Both describe the same model, and they are not the same question at all.

The two numbers pull against each other, and a single dial moves both. Make the spam filter stricter and precision rises while recall falls: fewer false alarms, more spam slipping through. Make it looser and the reverse happens. There is no setting that maximises both, so the real question is never what is the best score, it is what does each kind of mistake cost here. Blocking a job offer is a disaster; letting one advertisement through is a shrug.

same test, two thresholds

+----------------+-----------+-----------+
| threshold      | precision | recall    |
+----------------+-----------+-----------+
| strict         | high      | low       |
| loose          | low       | high      |
+----------------+-----------+-----------+

ask instead: what does a miss cost here?One threshold, two numbers moving in opposite directions.