Blog ·
Egress is the bottleneck
Queue, Heimdal and Pages are three products, but they are one bill. Same Supabase organisation, same Vercel account, same free tier under all of them. No paying customers yet, so the plan is to stay on the free tier as long as I can.
I kept assuming CPU or storage would be the thing that pushes me onto a paid plan. Turns out neither was even close. The thing I was running out of was bandwidth.
Supabase’s free plan gives 5 GB of egress a month. Vercel’s Hobby plan gives 100 GB of transfer and a million edge requests. None of that is generous, and one of them runs out about ten times sooner than the rest.
So I started asking a different question. Not “is this fast?” but “how many times does this cross the network, and who is paying for it?”
Everything below came out of asking that about three apps that were working perfectly fine. For me. Alone.
Heimdal: the 19 MB hello
Heimdal reads faces at a door. That needs models in the browser: a MediaPipe landmarker, a segmentation model for the bare-face check at enrolment, two detectors. They were living in Supabase storage, which is a fine place to put files and a terrible place to serve them from.
A device that has never seen the app pulls about 19 MB of model blobs before it can look at anybody. Against a 5 GB monthly cap:
5 GB ÷ 19 MB ≈ 260 devices
Two hundred and sixty. Not two hundred and sixty a day. Two hundred and sixty, then the month is over for every other thing that wanted to send a byte. One school running an enrolment drive on the teachers’ own phones is a month’s allowance and a paused project.
The fix is not clever. It is a different host:
// lib/env.public.ts
export const MODELS_BASE =
publicEnv.NEXT_PUBLIC_MODELS_URL?.replace(/\/$/, "") ??
`${publicEnv.NEXT_PUBLIC_SUPABASE_URL}/storage/v1/object/public/models`;
R2 charges nothing for egress, so the blobs go there and the number stops existing. One optional environment variable, unset in development, and the fallback is exactly the arrangement it replaced, which makes the rollback an env var rather than a deploy.
Two things did not move, and both are the interesting part. The wasm runtimes
still load same-origin from public/, because the kiosk runs under COEP
isolation and a cross-origin worker needs a CORP header that object storage
will not send. And the server-side embedder, a 70 MB ONNX file, is fetched at
build time into the deployment, not at runtime:
scripts/fetch-server-models.mjs puts them there at build time, so a cold
instance reads locally instead of pulling 73MB across the network before it
can answer anyone
Egress you pay per cold start is the worst kind. Serverless is entitled to cold start whenever it likes.
Queue: the read that multiplies
Queue’s Tracker is the screen a waiting customer keeps open on their phone. It is also, by a distance, the hottest read in the system, and for a reason worth saying out loud: its cost is customers × queue length. Everything else in the app is one poll per staff member. This one is one poll per person in the room, and each of those polls was reading the whole session.
Two things were wrong, and they multiply together.
The first is what was being selected. The projections were reading the same fifteen-column row the staff dashboard reads, for every entry created since the session opened. By the afternoon that is hundreds of rows, most of them long finished, and most of the columns not rendered anywhere. So there is a second, narrower read:
const PROJECTION_ENTRY_COLUMNS =
"token, state, created_at, called_at, completed_at, counter_id";
Six short values instead of fifteen, and the row count bounded rather than open: the live line is however many people are actually queued, and the finished entries are capped at the sample size the wait estimate looks at anyway. It was throwing the rest away after paying to move it.
Dropping client_token from that list is not an egress win at all. It is a
customer’s bearer secret, and it had no business on a public TV’s response
however narrow the path from the data layer to the screen looks today. That one
was free.
The second thing was the polling. Every live surface refetched every 10 seconds. But the poll is not how the screens stay current. A Realtime broadcast does that, and the poll exists only to catch a missed broadcast. So it only needs to be fast when the broadcast isn’t there:
const LIVE_POLL_MS = 45_000;
const FALLBACK_POLL_MS = 10_000;
function backstopInterval(queueId: string | null | undefined): number {
return isQueueLive(queueId) ? LIVE_POLL_MS : FALLBACK_POLL_MS;
}
isQueueLive reads the channel’s subscription status. While the socket is up
the poll is a slow backstop; the moment it drops, the next tick is back to ten
seconds on its own, without anything having to notice or coordinate. It is
deliberately a plain read rather than React state, because it is consulted
inside TanStack Query’s refetchInterval, evaluated fresh at every tick, and
making it stateful would re-render every subscribed screen in the building to
change a number only a timer ever reads.
The two together, for one customer waiting twenty minutes in a line that has already served a hundred people:
before 120 polls × ~40 KB ≈ 4.8 MB
after 27 polls × ~5 KB ≈ 0.15 MB
Same screen, same freshness, about a thirtyfold difference in what it costs to show it.
Pages: the CDN was already paid for
Pages had the opposite problem: the expensive path was the one that looked
free. A Page is rendered once at publish and served from Vercel’s CDN forever
after, which is the whole design. But the images on it were <img src>
pointing straight at the Supabase bucket, so the HTML was a CDN hit and every
byte of the picture on it was billed to the tier with the tightest cap.
So the images get served from our own origin instead, with the same static arrangement the Page itself uses:
// app/media/[...path]/route.ts
export const dynamic = "force-static";
export const revalidate = false;
The first visitor fetches from the bucket; everyone after that is a CDN hit.
Nothing in that bucket is ever overwritten. Every path is a fresh uuid and a
changed image is a new upload plus a sweep of the old one, so
max-age=31536000, immutable is a fact rather than a bet.
The first attempt was a next.config rewrite, which failed in a way worth
remembering: a rewrite to an external destination streams the upstream response
through untouched, so your own headers() never apply to it, and Cloudflare’s
__cf_bm set-cookie rides along. A response carrying set-cookie is a
response no CDN will cache. The rewrite was one line and did the opposite of
its purpose, silently.
Because that path segment gets pasted onto a bucket URL, it is also a trust boundary, so it is a route with a guard rather than a passthrough:
const MEDIA_PATH = /^(av|bg|im)\/[0-9a-f-]{36}\.(avif|webp)$/;
Anything that is not a path we minted is a 404. Without it, a crafted path is a request for whatever else the service role can reach.
So does “thousands” hold up?
I have been telling myself that this work means BearLog can carry thousands of users before anything needs paying for. It is worth checking that rather than believing it, so: on Queue’s numbers above, with the published limits.
Supabase egress, 5 GB a month, at roughly 150 KB per customer visit, is about 34,000 visits a month. Vercel’s million edge requests, at 27 polls a visit, is about 37,000 visits. Before the change, the same arithmetic gave a thousand visits and change. So the monthly claim holds, with room: the bottleneck genuinely moved.
The concurrent claim does not, and this is the part I had not looked at. Supabase’s free plan allows 200 concurrent Realtime connections per project. Every waiting customer’s Tracker opens one. That is not a budget that refills at midnight, it is a ceiling that applies right now, and no amount of trimming columns moves it. Two hundred people waiting at once is the wall, whether that is one busy hospital or forty small clinics at 9 a.m.
Behind it is a second one with the same shape. The free plan includes 2M Realtime messages a month, and every subscribed client counts separately for every message. Fifty people in a line and one “next please” is fifty messages, not one. A queue serving 200 people a day with an average of 50 watching is about 300,000 messages a month, all by itself, so roughly six busy queues before that runs out, which is a much smaller number than 34,000 visits and arrives much sooner.
Those two are scoped differently, which took me a second read of the pricing page to notice. Egress and the message allowance are billed per organisation, so Queue’s busy morning really does spend Heimdal’s bytes: three products on one tier is one budget wearing three hats. The connection limit is per project, so Queue owns its own two hundred. That sounds like the better deal and is actually the worse one, because it means the two hundred cannot be borrowed from the two apps that are sitting there not using theirs.
So: thousands of visits a month, yes, comfortably, and that is a real change from where it started. Thousands at once, no: a couple of hundred, and the limit is a connection count rather than a byte count. Those are different claims and I had been making the wrong one.
The next wall, which is a different kind of wall
Egress, CPU and storage are budgets. They are consumption, they refill on the first of the month, and the way through them is to consume less. Everything above is that: fewer columns, fewer polls, fewer bytes, a cheaper host. The work has a dial on it, and turning the dial always helps a bit.
Two hundred concurrent connections has no dial. It is not spend, it is a ceiling on a mechanism, and nothing I do to the size or frequency of a message changes how many sockets are open. I could get the Tracker down to one byte a minute and the two hundred and first customer still would not get in. There is nothing to optimise, because optimisation is the wrong verb.
So this work did not really finish. It moved the wall, looked past it, and found a different sort of wall standing behind it, and the second one does not answer to any of the techniques that beat the first. Connection pooling, who holds a socket and who is allowed to do without one, is the next thing I have to solve, and I would rather write it up once I have actually solved it than guess at it here.
That is the next post.
What I took from it
The biggest thing I got out of this was not the bandwidth savings.
It was realising how many assumptions I was making because I was the only user.
When you are testing your own app, everything looks cheap. One websocket connection is cheap. One 19 MB download is cheap. One query returning a hundred rows is cheap. At one user, every multiplier is 1.
You do not notice that a query is expensive, because it runs once. You do not notice that a realtime connection costs anything, because there is only one connection. You do not notice the 19 MB, because you only downloaded it once.
But users do not come one at a time. They come in hundreds, and every small inefficiency gets multiplied by all of them. Imagine the same screen open on a hundred phones, then a thousand, and the bottleneck becomes obvious.
A load test would not have caught most of this either. What caught it was reading each request and asking who else is making that same request at the same time.
The free tier helped, honestly. It puts a hard number on carelessness. Every one of these fixes is something I would have wanted eventually anyway, and none of them would have felt urgent this year if somebody else was absorbing the bytes.
The other thing is knowing which of your numbers is a budget and which is a ceiling, because they do not take the same kind of work. I spent days optimising egress only to find out that the thing that will probably stop me first is a realtime connection limit I did not even know existed. That is probably the most useful thing I got from this whole exercise.