Machine learning: rules from examples

Until now you did the work yourself. In data:1 you chose the rows, grouped them, and decided what the numbers meant. Machine learning flips that order. You stop writing the rule and start showing examples, and the program produces the rule. The rule is not written down anywhere as a sentence. It lives inside the model as a pile of numbers that only makes sense together.

+--------------------------+--------------------------+
| rules written by hand    | rules learned from rows  |
+--------------------------+--------------------------+
| who wrote it: a person   | who wrote it: the data   |
| new case: read the rules | new case: nearest rows   |
| breaks on: exceptions    | breaks on: new territory |
+--------------------------+--------------------------+Two ways to get a working rule, and how each one fails.

Take a table your school already has. Each row is one pupil, and every column is something you knew before the exam, plus one column that you only learned after it. That last column is the whole reason the table is useful.

practice_table
+-------------+-------------+----------------+
| study_hours | sleep_hours | passed         |
+-------------+-------------+----------------+
| 1           | 5           | no             |
| 2           | 6           | no             |
| 4           | 7           | yes            |
| 6           | 6           | yes            |
| 9           | 8           | yes            |
| 3           | 4           | no             |
+-------------+-------------+----------------+Six rows is a toy table, but the shape is the same one a bank uses with six million rows.

The columns the model is allowed to read are called features. The column it must produce is called the label. In this table study_hours and sleep_hours are features and passed is the label. For a new pupil the model receives the features and produces its own guess at the label. That guess is a prediction, and it is the only thing the model ever does: features in, label out. The label is what makes learning possible at all, because it is the answer the model compares itself against while it trains.

Sometimes there is no label column at all. Nobody wrote down the answer, so there is nothing to copy. Then the model can only look at how the rows relate to each other: which rows resemble each other so strongly that they probably belong to the same group. A shop can do this to its receipts without ever deciding in advance what the groups are.

receipts with no label at all

+--------+--------+--------+
| item   | price  | hour   |
+--------+--------+--------+
| coffee | 2.40   | 08:10  |
| bun    | 1.80   | 08:12  |
| soup   | 4.20   | 13:05  |
| salad  | 4.50   | 13:20  |
| cola   | 2.10   | 17:40  |
+--------+--------+--------+

rows that look alike: 1 with 2, 3 with 4
nobody wrote those two groups downNo answer column anywhere, only rows that happen to sit close together.

Now the part that is usually left out. The model does not understand the subject. It has a shape with a handful of adjustable numbers, and training means nudging those numbers until the shape passes near the training labels. Only two things decide what it can learn: which features you gave it and which shape you chose. If the real pattern is not something that shape can express, no amount of data will save you, and the model will still answer confidently.