Part V · Chapter 8 · Running It · lesson 1
Prove the toolchain before you trust anything it tells you
14 min
Part V · Chapter 8 · Running It · lesson 1
14 min
Three files and one command. At the end of it your terminal prints four lines: the first two identical on every machine in the world, the last two your machine's baseline for the rest of this track.
The handler you are about to paste costs two milliseconds of processor time and eighteen milliseconds of waiting, and one worker serves one request at a time. Commit to what that implies before you install anything.
Predict
Nothing is taught below this line. Everything in it relocates something you already know onto a machine you control.
Install the container runtime for your operating system, from its own installer: the desktop application on Windows and macOS, the engine package on Linux. That is the only install this track asks for, and it is the same install on all three platforms.
Then make a folder — call it lab — and put three files in it.
compose.yamlservices:
service:
image: python:3.13-slim
working_dir: /lab
volumes:
- .:/lab
command: python service.py
environment:
WORKERS: "1"
SPIN_MS: "2"
WAIT_MS: "18"
LIMIT: "0"
ports:
- "18080:18080"
- "18081:18081"
load:
image: python:3.13-slim
working_dir: /lab
volumes:
- .:/lab
depends_on:
- service
entrypoint: ["python", "load.py", "--host", "service"]
command: ["--check"]
service.py"""
A service with one knob and nothing hidden.
Every request to /work costs the same: SPIN_MS of processor time, then WAIT_MS
of waiting on something that is not the processor. WORKERS of them are worked
on at a time; the rest sit in a queue. If LIMIT is above zero, a connection
that arrives while the queue already holds LIMIT of them is refused at the
door, before anything reads it.
The counters live on a second port, so you can read them while the first port
is saturated.
"""
import json
import os
import queue
import socket
import threading
import time
HOST = "0.0.0.0"
PORT = int(os.environ.get("PORT", "18080"))
ADMIN_PORT = int(os.environ.get("ADMIN_PORT", "18081"))
WORKERS = int(os.environ.get("WORKERS", "1"))
SPIN_MS = float(os.environ.get("SPIN_MS", "2"))
WAIT_MS = float(os.environ.get("WAIT_MS", "18"))
LIMIT = int(os.environ.get("LIMIT", "0"))
BACKLOG = 4096
lock = threading.Lock()
counts = {
"accepted": 0,
"refused": 0,
"completed": 0,
"abandoned": 0,
"in_service": 0,
"queued_max": 0,
}
work = queue.Queue()
def bump(key, by=1):
with lock:
counts[key] += by
def spin(ms):
"""Occupy the processor for ms milliseconds. No sleeping, no waiting."""
end = time.perf_counter() + ms / 1000.0
while time.perf_counter() < end:
pass
def snapshot():
with lock:
out = dict(counts)
out["queued"] = work.qsize()
out["config"] = {
"workers": WORKERS,
"spin_ms": SPIN_MS,
"wait_ms": WAIT_MS,
"limit": LIMIT,
}
return out
def path_of(conn, budget=5.0):
conn.settimeout(budget)
data = b""
while b"\r\n\r\n" not in data and len(data) < 8192:
chunk = conn.recv(4096)
if not chunk:
break
data += chunk
line = data.split(b"\r\n", 1)[0].decode("latin1", "replace")
bits = line.split(" ")
return bits[1] if len(bits) > 2 else "/"
def respond(conn, status, body, extra=""):
payload = body.encode()
head = (
"HTTP/1.1 %s\r\n"
"Content-Type: text/plain\r\n"
"Content-Length: %d\r\n"
"Connection: close\r\n"
"%s"
"\r\n" % (status, len(payload), extra)
).encode()
conn.sendall(head + payload)
def shut(conn):
try:
conn.shutdown(socket.SHUT_WR)
except OSError:
pass
try:
conn.close()
except OSError:
pass
def serve_work(conn):
try:
path_of(conn)
spin(SPIN_MS)
time.sleep(WAIT_MS / 1000.0)
respond(conn, "200 OK", "done\n")
bump("completed")
except OSError:
# The caller gave up while this request was queued or in service. The
# work was still done. Nobody is left to receive it.
bump("abandoned")
finally:
shut(conn)
def refuse(conn):
"""Turn a connection away without reading the request it carries.
The one recv is not politeness. A socket closed with unread bytes still in
its receive buffer is reset rather than closed, and a reset can take the
refusal away with it, so the caller would see a dropped connection instead
of an answer.
"""
try:
conn.settimeout(0.05)
try:
conn.recv(8192)
except OSError:
pass
respond(conn, "503 Service Unavailable", "busy\n", extra="Retry-After: 1\r\n")
bump("refused")
except OSError:
pass
finally:
shut(conn)
def worker():
while True:
conn = work.get()
bump("in_service")
try:
serve_work(conn)
finally:
bump("in_service", -1)
work.task_done()
def listener(port, handler):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((HOST, port))
sock.listen(BACKLOG)
while True:
conn, _ = sock.accept()
conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
handler(conn)
def door(conn):
depth = work.qsize()
with lock:
if depth > counts["queued_max"]:
counts["queued_max"] = depth
if LIMIT > 0 and depth >= LIMIT:
refuse(conn)
return
bump("accepted")
work.put(conn)
def admin(conn):
try:
path = path_of(conn, budget=2.0)
if path.startswith("/reset"):
with lock:
for key in ("accepted", "refused", "completed", "abandoned", "queued_max"):
counts[key] = 0
respond(conn, "200 OK", "reset\n")
elif path.startswith("/stats"):
respond(conn, "200 OK", json.dumps(snapshot()) + "\n")
else:
respond(conn, "200 OK", "ok\n")
except OSError:
pass
finally:
shut(conn)
def main():
for _ in range(WORKERS):
threading.Thread(target=worker, daemon=True).start()
threading.Thread(target=listener, args=(ADMIN_PORT, admin), daemon=True).start()
print(
"service up: %d worker(s), %g milliseconds of spin, %g milliseconds of wait, "
"limit %s" % (WORKERS, SPIN_MS, WAIT_MS, LIMIT if LIMIT > 0 else "off"),
flush=True,
)
listener(PORT, door)
if __name__ == "__main__":
main()
load.py"""
A load generator that does not wait for permission.
Arrivals are scheduled from a fixed random seed and fired whether or not the
previous one has come back. That is the whole point: a generator that waits for
a reply before sending the next request can never offer more load than the
service can take, so it can never show you a plateau.
python load.py --host service --check
python load.py --host service --rate 10,20,30,40,45,48 --seconds 20
"""
import argparse
import asyncio
import json
import random
import statistics
import time
import urllib.request
ARGS = None
def pct(values, q):
if not values:
return float("nan")
ordered = sorted(values)
k = max(0, min(len(ordered) - 1, int(round((q / 100.0) * (len(ordered) - 1)))))
return ordered[k]
def admin(path):
url = "http://%s:%d%s" % (ARGS.host, ARGS.admin_port, path)
with urllib.request.urlopen(url, timeout=5) as r:
return r.read().decode()
def wait_for_service(seconds=20):
deadline = time.time() + seconds
while time.time() < deadline:
try:
admin("/health")
return True
except Exception:
time.sleep(0.25)
return False
async def request(host, port, result):
start = time.perf_counter()
writer = None
try:
reader, writer = await asyncio.open_connection(host, port)
writer.write(
("GET /work HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n" % host).encode()
)
await writer.drain()
head = await reader.readline()
await reader.read(-1)
elapsed = (time.perf_counter() - start) * 1000.0
bits = head.split(b" ")
code = bits[1].decode() if len(bits) > 1 else "?"
if code == "200":
result["ok"].append(elapsed)
elif code == "503":
result["refused"].append(elapsed)
else:
result["error"] += 1
except asyncio.CancelledError:
raise
except Exception:
result["error"] += 1
finally:
if writer is not None:
try:
writer.close()
except Exception:
pass
async def attempt(timeout, result):
"""One arrival, given up on if it has not answered inside the client budget."""
try:
await asyncio.wait_for(request(ARGS.host, ARGS.port, result), timeout)
except (asyncio.TimeoutError, asyncio.CancelledError):
result["gaveup"] += 1
async def offer(rate, seconds, timeout, seed):
"""Fire Poisson arrivals at `rate` for `seconds`, then wait for the stragglers."""
rng = random.Random(seed)
result = {"ok": [], "refused": [], "gaveup": 0, "error": 0, "sent": 0}
tasks = []
begin = time.perf_counter()
at = 0.0
while True:
at += rng.expovariate(rate)
if at > seconds:
break
gap = begin + at - time.perf_counter()
if gap > 0:
await asyncio.sleep(gap)
result["sent"] += 1
tasks.append(asyncio.ensure_future(attempt(timeout, result)))
await asyncio.gather(*tasks, return_exceptions=True)
result["window"] = seconds
return result
COLUMNS = [
("offered", 8),
("sent", 7),
("ok", 6),
("refused", 8),
("gaveup", 7),
("error", 6),
("ok_per_sec", 11),
("p50_ms", 8),
("p99_ms", 8),
("done", 7),
("lost", 7),
("queued", 7),
]
def line(values):
return " ".join(
str(value).rjust(width) for value, (_, width) in zip(values, COLUMNS)
)
def header():
print(line([name for name, _ in COLUMNS]), flush=True)
def row(rate, result, stats):
ok = result["ok"]
print(
line(
[
rate,
result["sent"],
len(ok),
len(result["refused"]),
result["gaveup"],
result["error"],
round(len(ok) / result["window"], 1),
round(pct(ok, 50), 1),
round(pct(ok, 99), 1),
stats.get("completed", -1),
stats.get("abandoned", -1),
stats.get("queued_max", -1),
]
),
flush=True,
)
def drain(seconds):
"""Wait for the queue to empty, so the next measurement starts from rest."""
deadline = time.time() + seconds
while time.time() < deadline:
try:
if json.loads(admin("/stats"))["queued"] == 0:
time.sleep(1.0)
return
except Exception:
pass
time.sleep(0.5)
def check():
"""One command, one exact line, and this machine's baseline."""
if not wait_for_service():
print("lab check: no service answering on %s" % ARGS.host)
return 1
config = json.loads(admin("/stats"))["config"]
samples = []
for _ in range(60):
start = time.perf_counter()
with urllib.request.urlopen(
"http://%s:%d/work" % (ARGS.host, ARGS.port), timeout=10
) as r:
r.read()
samples.append((time.perf_counter() - start) * 1000.0)
warm = samples[10:]
print("lab check: ok")
print(
"service: {} worker(s), {:g} milliseconds of spin, "
"{:g} milliseconds of wait, limit {}".format(
config["workers"],
config["spin_ms"],
config["wait_ms"],
config["limit"] if config["limit"] else "off",
)
)
print(
"one at a time: median {:.1f}, slowest {:.1f} (milliseconds)".format(
statistics.median(warm), max(warm)
)
)
print(
"that median implies one worker finishes {:.1f} requests "
"per second".format(1000.0 / statistics.median(warm))
)
return 0
def main():
global ARGS
parser = argparse.ArgumentParser()
parser.add_argument("--host", default="service")
parser.add_argument("--port", type=int, default=18080)
parser.add_argument("--admin-port", type=int, default=18081)
parser.add_argument("--rate", default="")
parser.add_argument("--seconds", type=float, default=20.0)
parser.add_argument("--timeout", type=float, default=2.0)
parser.add_argument("--seed", type=int, default=1)
parser.add_argument("--check", action="store_true")
ARGS = parser.parse_args()
if ARGS.check or not ARGS.rate:
raise SystemExit(check())
if not wait_for_service():
raise SystemExit("no service answering on %s" % ARGS.host)
rates = [int(x) for x in ARGS.rate.split(",") if x.strip()]
header()
loop = asyncio.new_event_loop()
for rate in rates:
try:
admin("/reset")
except Exception:
pass
result = loop.run_until_complete(offer(rate, ARGS.seconds, ARGS.timeout, ARGS.seed))
try:
stats = json.loads(admin("/stats"))
except Exception:
stats = {}
row(rate, result, stats)
drain(40)
loop.close()
if __name__ == "__main__":
main()
From inside the lab folder:
docker compose run --rm load --check
The first run downloads a small runtime image — a few tens of megabytes — and takes a minute. Later runs reuse it and start immediately.
lab check: ok
service: 1 worker(s), 2 milliseconds of spin, 18 milliseconds of wait, limit off
one at a time: median 20.9, slowest 44.6 (milliseconds)
that median implies one worker finishes 47.8 requests per second
The first two lines are exact. Compare them character for character. They
are produced from the configuration in compose.yaml, which is the same file
you pasted, so they do not depend on your hardware at all. If line one says
anything other than lab check: ok, stop and read the next section.
The last two lines are yours. The median will land somewhere between about twenty and about thirty on most machines, and the implied rate between about thirty-three and fifty a second. Write both down. The rest of the track compares everything to them.
The run above is the one every later lesson in this track quotes: a Windows laptop, one worker, nothing else competing. Yours will differ in the third and fourth lines and nowhere else.
Not that the arithmetic is right — that comes later. They prove four separate things, which is why the acceptance test is four lines and not one.
| line | what it rules out |
|---|---|
lab check: ok | the runtime is absent, the files are missing, a port is taken |
the service: line | you edited the configuration without meaning to |
| the median | the container is throttled hard enough to invalidate the track |
| the implied rate | your arithmetic and the machine's disagree about the ceiling |
Three of those four are plumbing. Finding them now costs a minute; finding them in the middle of lesson five costs you a wrong conclusion about a bottleneck.
Three things go wrong here, and all three are plumbing. Read the first line of the error rather than the last.
The runtime is not running. The message names a socket or a pipe it could not connect to, and asks whether the daemon is running — on Windows it mentions a pipe name and says the system cannot find the file specified. The lab's code has not been read at this point. Start the desktop application, wait until it reports that the engine is running, and run the command again. This is the most common failure by a wide margin.
The files are not where the command is looking. The container starts and
then says it cannot open a file, naming /lab/service.py or /lab/load.py
with an error about no such file or directory. The folder you ran the command
from is not the folder holding the three files, or a file was saved with a
different name — a text editor appending .txt is the usual culprit. Run the
command from inside lab, and check that the three names match exactly.
A port is already taken. The message names 18080 and says the address is
already in use or the port is already allocated. Something else on your machine
is listening there. Change both numbers on the left of the colon in
compose.yaml — "18090:18080" and "18091:18081" — and run the command
again. Nothing inside the lab cares which host port it is published on.
When you are done for the day:
docker compose down
The image tag in compose.yaml is python:3.13-slim rather than
python:slim or python:latest, and that choice is the whole difference
between a lab that still works next year and one that silently changes
underneath you. A minor-version tag is the right amount of pinning for
teaching: it keeps the interpreter's behaviour stable while still collecting
security patches.
What I would not do here is pin the digest. A @sha256:… reference is the
correct answer for a production deployment and the wrong one for a lab,
because the day that digest is garbage-collected from the registry the reader
gets a pull failure with no obvious cause, and they will conclude the lesson
is broken rather than the pin. The tradeoff is reproducibility against
diagnosability, and in teaching material diagnosability wins.
Check