Caching: the same answer, asked for again

You have a client, a server, and a contract between them. Now watch what the server is actually asked. A popular shop page is requested thousands of times an hour by thousands of different people, and the answer is identical every single time. The server re-reads the same rows from the database, re-builds the same JSON, and sends the same bytes — thousands of times, for one fact that changed this morning. Caching is the decision to stop doing that.

The reason caching works is not clever storage. It is the shape of real traffic: requests repeat. A few items are asked for enormously often, and most items are asked for almost never. When the same question arrives a thousand times, doing the work once and keeping the answer turns a thousand pieces of work into one plus nine hundred ninety-nine copies. This is also why caching sometimes fails badly: if every request is genuinely different — a page personalised for each user — there is nothing to reuse, and a cache just adds a lookup in front of the real work.

Caches live at several levels, and each level trades distance for control. The table below is worth reading by column: a hit closer to the user is faster and cheaper, but a stale copy closer to the user is also the hardest one to correct, because you no longer control the machine holding it.

WHERE IT LIVES        WHO SHARES IT    HIT COST     SIZE      LIFETIME
--------------------  ---------------  -----------  --------  ------------------
browser cache         one user         milliseconds smallest  dies with the tab
CDN edge              one city         tens of ms   large     shared by strangers
application cache     one service      under a ms   medium    lives in RAM
database cache        one query        under a ms   large     lives next to the dataThe same trade at every level: the closer the copy sits to the user, the faster the hit and the harder it is to take back.

The hard part has a name: invalidation. When the price changes in the database, every stored copy of the old price is now a lie, and the server must either delete those copies or make them expire. Getting this right costs real work. You can expire by time — simple, and wrong for a window of minutes. You can expire on write — correct, and it means every piece of code that changes the data must now remember to tell the cache, which is the kind of rule that gets forgotten in a bug fix at 2 a.m. There is no free option; there is only a choice about which mistake you can live with.

So every cache holds a deliberate bet: how stale may this answer be? A logo may be a year old and nobody notices. A product photo, an hour. A price, a few seconds. A bank balance, zero — do not cache it at all. Notice that freshness is a product decision, not a technical one. The engineer choosing max-age=60 is really answering the question "would it be acceptable for a customer to see a price that is one minute out of date?". If the answer is no, the cache must be corrected the moment the price changes, and that is a different and more expensive design.