Skip to content
../writing

Size a System on the Back of an Envelope

Naseebullah AhmadiSenior Software Engineer, London

The interviewer asks how much traffic you're expecting. Round to powers of ten, turn per-day into per-second, and end every number on the design decision it forces: one node, a cache, or a shard.

#interviews
#scalability
#distributed-systems
#architecture
#postgres
#redis
4 min read
#interview
In one line

Per day ÷ 10⁵ = per second, × 2 to 3 for peak, round everything to a power of ten, and end every number on what it decides: one node, a cache, or a shard.

You're here when

Signs this is your situation

  • The interviewer asks "how much traffic are we talking about?"

  • Someone proposes sharding before anyone has done the arithmetic

  • You need to know if a new service fits on one box

The numbers to carry

ConvertRuleAnchor
Per day → per second÷ 10⁵1M/day ≈ 10/s
Per month → per second÷ 2.5 × 10⁶100M/month ≈ 40/s
Average → peak× 2 to 340/s ≈ 100/s at peak
Requests in flightrate × latency (Little's law)1K/s × 50 ms = 50
Multiplydigits ×, exponents +10⁸ × 5 × 10² = 5 × 10¹⁰
@itsnas byte-ladder

1 TB = 10 to the power 12 bytes

Terabyte · Server SSD. ≈ 2B rows of 500 B, about a laptop SSD.

~100 µs access latency5 years: 3 TB
A billion 1 KB rows is a terabyte: one server's disk. Tags show where the URL shortener lands.
@itsnas a-day-of-traffic.svg
A day of traffic against its daily averageRequests per second over 24 hours as a multiple of the daily average. Traffic bottoms out near 0.2x at 8am and climbs to 2.5x at 8pm. The flat line at 1x is what dividing a daily count by 10⁵ gives you; traffic sits above it from about 3pm to 1am.÷ 10⁵ averageTrafficpeak: 2.5x0x1x2x3x00:0006:0012:0018:0024:00Hour of dayLoad vs. daily average
÷ 10⁵ gives the flat line, and runs about 14% low (a day is 86,400 seconds). The shaded 40% of the day runs above it: size for the dot.
@itsnas latency-ladder
Log scale: each step right is 10x slowerUnder 1 ms1 ms or more
  1. Memory reference 100 nsat human scale: 1 second
  2. SSD random read 100 µsat human scale: 17 minutes
  3. Round trip, same datacenter 0.5 msat human scale: 1.4 hours
  4. Disk seek 5 msat human scale: 14 hours
  5. Round trip, across an ocean 150 msat human scale: 17 days
The gaps matter, not the exact values. If memory is a second, an ocean round trip is over two weeks: that's the case for caches and CDNs.

What one node takes. Say these out loud as assumptions:

One…Takes about
App instance1K req/s
#postgres primary5K simple writes/s
#redis node100K ops/s
Server100 GB RAM, 10 TB SSD

The play

  1. 1

    Assumptions in a corner of the board: users, actions per user, read:write, record size, retention.

  2. 2

    Traffic at peak, reads and writes apart: writes pick the database, reads pick the cache.

  3. 3Storage: writes × record size × retention.
  4. 4

    Memory: the hot ~20% of recent data, say a month of new rows.

  5. 5Servers: peak QPS ÷ 1K per instance.
  6. 6Close the loop: say what each number decides.

Say this

  • "One Postgres primary: it's about 40 writes a second."
  • "3 TB over five years, so it fits on one node. No sharding."
  • "Peak reads are around 10K a second, so a cache in front."

Not this

  • "1,157.4 requests per second."
  • "We'll need to shard," with no number behind it.
  • "Storage is 3 TB," and then moving on.

Worked: a URL shortener

100M new links a month, 100 reads per write, 500 bytes a record, kept 5 years. On the board:

@itsnas whiteboard
writes     10⁸/month ÷ 2.5 × 10⁶   ≈ 40/s      # one Postgres primary
reads      40 × 100 × 2.5 peak     ≈ 10K/s     # one Redis node in front
storage    10⁸ × 60 × 500 B        ≈ 3 TB      # 5 years fits one node
memory     20% × 10⁸ × 500 B       ≈ 10 GB     # a fifth of a month, in RAM
bandwidth  10K × 500 B             ≈ 5 MB/s    # no CDN needed
codes      6 × 10⁹ ÷ 62⁷           ≈ 0.17%     # 7 characters is plenty
servers    10K ÷ 1K                ≈ 10        # app instances at 1K/s
main
Nas (@itsnas)
What goes on the board, and what each line decides

Now scale it and watch which numbers cross one node's limit:

@itsnas sizing
New links a month 100MPast one node 0 of 6One node's limit

At 1x, 0 of 6 estimates outgrow one node.

  • Writes 40/sone Postgres primary(limit 5K/s)
  • Peak reads 10K/sone Redis node in front(limit 100K/s)
  • Storage, 5 years 3 TBone database node(limit 10 TB)
  • Hot set 10 GBfits in one node's RAM(limit 100 GB)
  • Peak bandwidth 5 MB/sno CDN needed(limit 100 MB/s)
  • 7-char codes used 0.17%random codes rarely collide(limit 10%)
At 1x nothing needs sharding, and saying so is as strong a signal as knowing how to shard. At 100x every number but the writes outgrows one node.

Pro tips

  • Round load up and capacity down, so errors land as spare room.
  • Ask one clarifying question, then assume out loud and keep moving.
  • When the interviewer changes a number, change one line and rerun.
  • Five minutes, not fifteen: the estimate leads to the design, it isn't the interview.
Go deeper
What Breaks From 1k to 1M Requests Per SecondHashing and the Birthday Paradox