Part I · Chapter 1 · The Diagram · lesson 5
Why the diagram has two of everything
14 min
Part I · Chapter 1 · The Diagram · lesson 5
14 min
Go back to the whiteboard from lesson one. Two app servers, a load balancer in front of them, and a database with a second copy beside it.
Somebody decided each of those duplicates was worth paying for. There are exactly two reasons to add a second machine, they are completely different from each other, and almost every confused conversation about infrastructure is two people each holding one of them.
Tell them apart and half the diagram explains itself.
Predict
Capacity. One machine cannot do all the work, so the work is divided between several. Adding machines adds throughput. This only functions when requests can go to any of them, which is a real constraint on how you write the program.
Survival. One machine will eventually stop — a reboot, a hardware fault, a deploy gone wrong. A second one means the site continues. This buys you nothing in throughput and everything in uptime, and it is the reason that holds even when every machine is nearly idle.
The words you will hear are scaling out for the first and redundancy or high availability for the second. They look identical on a whiteboard — two rectangles — and they justify different things. A team scaling for capacity will put the machines wherever is cheapest. A team buying survival will deliberately put them in different buildings, which costs more and is the entire point.
Once there are two app servers, something has to decide which one gets each request. That is the load balancer, and it does two jobs that are worth separating in your head.
It spreads the work. Requests go round the servers so no one of them gets everything. This is the capacity job and it is the one the name describes.
It notices when one is broken and stops sending to it. This is the survival job, it is called health checking, and it is the part that actually delivers the uptime. A load balancer that keeps sending a share of requests to a dead machine has made things worse than one server, because now some fraction of users get an error instead of none.
That second job is why the box is there even when capacity is not a problem. It is also where a surprising number of outages come from: a health check that tests the wrong thing will cheerfully route traffic to a server that answers the check and fails everything else.
With one app server you can keep things in the program's memory and they are there next time. With two, the next request may land on the other machine, which knows nothing about it.
Anything remembered in memory becomes unreliable. Who is logged in, what is in the basket, a half-finished upload. The fix is to keep that somewhere both machines can see — the database, or the cache box from lesson one — rather than inside either program.
Anything scheduled runs twice. A nightly job written as "when the clock says three, do this" now has two programs that both see three o'clock.
Anything written to local disk is invisible to the other. An uploaded file saved next to the program exists on one machine and not the other, so half of all requests to view it fail.
Each of these is the same discovery in a different costume: state kept inside one process stopped being shared the moment there were two processes. That is the real cost of the second machine, and it is why "just add another one" is cheap to say and not always cheap to do.
Check
Ask which reason, every time, and expect a specific answer. "For reliability" and "for scale" are the two answers, and a team that cannot say which usually has the machines for neither reason — they are there because the template had two.
Do survival before capacity, if you are choosing. Running out of capacity is gradual, visible and arrives with warning. A single machine failing is sudden and arrives at an inconvenient hour.
Count the places state is hiding before adding the second machine. In- memory sessions, local uploads, scheduled jobs, anything written to a temporary directory. That list is the actual work of going from one to two, and it is almost always longer than expected.
The condition under which all of this is premature: when one machine can serve the load, and an occasional outage is genuinely acceptable.
That describes a great many internal tools, early products and side projects honestly. The second machine is not free: it is more money, a load balancer to operate, a health check to get right, and the state problem above to solve. Paying all of that to avoid an outage nobody would have minded is a bad trade, and it is a trade made constantly because the diagram with two boxes looks more professional than the diagram with one.
The honest version of the question is: what actually happens if this is down for an hour on a Tuesday? Sometimes the answer is "we lose the business" and sometimes it is "four people wait". Chapter two is where that question gets a number attached to it.