A database is careful: it writes to disk, it keeps data safe, it answers complicated questions. That care makes it slow compared to memory. A cache is a small, fast store in memory (Redis or Memcached, usually) that keeps copies of data you read often.
Cache-aside is the most common way to use one. The name says it all: the cache sits to the side. Your application talks to both, and the application decides what goes in the cache.
Reading: check the cache first
When a request needs the price of product 42:
- Ask the cache for
product:42. - Hit? The cache has it. Return it. Done in about a millisecond, and the database never heard about it.
- Miss? The cache doesn't have it. Read from the database, then put a copy in the cache (with an expiry time, say 10 minutes), then return it.
The request that misses is a little slower than with no cache at all (it does extra work), but every request after it is much faster. With 50,000 reads a minute and a price that changes twice a day, almost every read is a hit.
Lazy loading: the cache only ever holds things somebody actually asked for. Nothing is loaded "just in case". That keeps the cache small, and an empty cache fills itself. One warning: while it fills, every read misses and goes to the database. A cold cache in front of heavy traffic (after a restart, or when you add cache servers) can overload the database. Unit 4's lesson on stampedes and the consistent hashing lesson come back to this.
Writing: update the database, delete the cache entry
When the price changes:
- Write the new price to the database. The database stays the source of truth.
- Delete
product:42from the cache.
The next read misses and loads the fresh price. Why delete instead of writing the new value into the cache? Because two writes can race. If request A and request B update the price at almost the same time, they can write to the cache in the opposite order from the database, and the cache would keep the older value. Two deletes can't be "out of order" like that, so deleting is safer.
Safer, not perfectly safe. One race remains:
- A reader misses and reads the old price from the database.
- Before it fills the cache, a writer saves the new price and deletes the cache entry.
- The reader now fills the cache with the old price it read in step 1.
The cache is wrong until that entry expires. The window is small (the reader has to be slower than a whole write), so most systems accept it and rely on the TTL. Systems that can't sometimes delete the entry a second time a moment after the write, or use leases so a stale reader can't fill.
The catch: stale data
Between "the database changed" and "the cache entry is gone", readers can see the old value. The race above can also put an old value back. And if the delete fails (a network blip, a crash between steps 1 and 2), the old value stays until it expires.
So cache-aside always comes with:
- An expiry (TTL) on every entry. It caps how long anything can be stale: with a 10-minute TTL, a wrong price can survive 10 minutes at most.
- Choosing what to cache. A price that can be 10 minutes stale on a product page? Usually fine. A bank balance? Don't cache it this way.
Key idea: read from the cache, fall back to the database and fill the cache on a miss; on writes, update the database and delete the cached copy. Always set a TTL, because sometimes the cache will be wrong for a while.
Go deeperfor interviews and production
The numbers behind "almost every read is a hit"
Say a cache read takes about 1 ms and a database read about 10 ms. A miss costs both, plus the write into the cache, so call it 12 ms. With a hit ratio of 99%:
- Average latency: 0.99 × 1 ms + 0.01 × 12 ms ≈ 1.1 ms, about nine times faster than always reading the database.
- Database load: only misses reach it. Scale the product page up to a busy shop at 50,000 reads a second and the database sees about 500 a second. That 100× drop is usually the real reason to cache: it's what lets one database serve the whole site.
Hit ratio is the number to watch. Going from 99% to 98% doubles the database's read load, even though it sounds like a 1% change.
What it looks like in code
def get_product(id):
key = f"product:{id}:v1"
cached = redis.get(key)
if cached is not None:
return decode(cached) # hit
row = db.query("SELECT ... WHERE id = %s", id)
redis.set(key, encode(row), ex=600) # fill with a 10-minute TTL
return row
def update_price(id, price):
db.execute("UPDATE products SET price = %s WHERE id = %s", price, id)
redis.delete(f"product:{id}:v1") # after the database commit, never before
Two details interviewers like to probe:
- Version the key (
:v1). When the cached shape changes, bump the version and old entries simply stop being read, then expire. No mass delete needed. - Delete after the commit. Deleting first opens a window where a reader refills the cache with the old row before the write lands.
Closing the stale-set race with leases
The race from the write section has a name: a stale set. Facebook's memcache fleet fixed it with leases. On a miss, the cache hands the reader a token. A delete for that key invalidates the token, so when the slow reader tries to fill the cache with its old value, the cache rejects it. The same token limits how often one hot key can be refilled, which also tames stampedes (more on those in the invalidation and stampedes lesson).
Asking for things that don't exist
If someone requests product 99999999 over and over, every request misses and hits the database, because there's nothing to cache. That's cache penetration, and it's a cheap way to attack a site. The fix is negative caching: store "not found" too, with a short TTL (say 30 seconds), so the database answers each missing ID once per TTL. That only helps when the same IDs repeat. A bot that invents a fresh ID every time needs the heavier-duty fix: a Bloom filter of valid IDs in front of the cache, which turns away IDs that can't exist before anything is looked up.
When the cache is down
Cache-aside degrades gracefully in theory: every read becomes a miss and goes to the database. In practice, a database sized for 500 reads a second can't take 50,000, so "the cache is down" turns into "the database is down". Production setups:
- Fail open, but limit it: treat cache errors as misses, with short cache timeouts (a few milliseconds) so a sick cache doesn't slow every request.
- Shed load: rate-limit or serve a degraded page rather than pass the full read load through.
- Run the cache with replicas (Redis with a replica and failover), and warm a new node before sending it traffic.
Interview follow-ups
- Why not just use a read replica? A replica still does disk and query work, at maybe 5–10 ms; the cache answers from memory and scales reads much more cheaply. Many systems use both.
- How big should the cache be? Big enough for the working set, the data read often. If 10% of products get 90% of views, caching that 10% gets most of the benefit.
- When is cache-aside the wrong choice? When you can't accept stale reads at all (balances, stock you sell out of), or when every read is unique, so the hit ratio would be near zero.