63ms to the first page, not 8,608 jobs later

Job Wolverine is a job board fed by crawlers. Its home page is a grid of listings, and the grid gets everything from one request, GET /api/jobs. Search, sort and both filters run in the browser over every job on the board, so that one request has to carry all of them. The commit that changed it states the constraint up front: "the payload cannot shrink. What can change is when the browser is allowed to start drawing."

Until 2026-08-22 the browser was allowed to start drawing at the very end. The endpoint ran "one query for 8,608 whole rows, normalise 49.5 MB of description text, serialise 5.3 MB of JSON, then send it." Nothing compressed it. The grid showed skeleton cards for the whole wait.

It had only been like that for twelve days. Until 2026-08-10 the query ended in .limit(150), which kept it quick and meant search covered "150 of 9,534 jobs". Dropping the limit made search honest, and it made the first byte wait for the entire corpus. From then on, every job a crawler found made the page a little slower to start. The better the system did its job, the worse the page got.

What changed

a82151d streams the listing. The order of work is the important part:

1. Sort the ids, and only the ids. listing_ids() orders every job on the brand by date, carrying nothing but the id. The commit timed that at 45ms, against 660ms for sorting whole rows.
2. Read rows by id, in batches. 64 first, which is four pages of the grid at 16 a page, then 500 at a time. Each batch is a short query that finishes before the handler flushes, so no read stays open on the one shared SQLite connection while the network catches up.
3. Write and flush after every batch, gzipped, which it had never been.

The response is still one valid JSON document, so response.json() works. It is also one job per line, so a reader in the browser can parse each job as its line lands. The results, measured in the browser:

ttfb   1.3s+  ->  63ms      (first 64 jobs, four pages' worth)
total  ~1.9s  ->  530ms
wire   5.3 MB ->  811 KB

The commit also measured the obvious alternative, paging with LIMIT/OFFSET, at 24,000ms for the whole corpus, because "each page re-sorts the join".

The wait is shorter, not constant

It is easy to read "63ms" as "streaming made the wait constant." It didn't. Step 1 still runs before the first byte, and step 1 touches every job.

To see the shape, both versions of the handler can be run against SQLite databases built at different sizes. The data is synthetic, generated to match the commit's figures: about 5,800 characters of description per job, 284 timestamps shared out of 8,608. It runs on a desktop over localhost, using each commit's own handler classes:

At August's size, streaming got the first byte out about sixteen times sooner, and the line is still straight. It changed the slope, not the shape. What it waits for is the sort, and the sort reads every row: on the reproduction's schema, SQLite plans it as a full scan of the table plus a temporary B-tree to order the result. Reading the first 64 rows costs about 2ms at any size.

So the page still gets slower as the board grows. It just takes much more growth to notice, and the board is growing. A search index built on 2026-09-03 covered 13,015 jobs, and the commit that built it notes that the set of ids changes "whenever the crawler adds a job, which is most runs".

Anything that has to see every row before the first byte keeps the first byte proportional to the corpus. A global sort is the obvious case. Streaming can't help with it. Only an index the sort can walk, or fewer rows, can.

What streaming cost

Three things came free with a response built in one piece, and all three now take work.

The status code. Once the first byte is out, the response is a 200, whatever happens next. The handler knows this. It reads the first batch before writing anything, so a missing column, which SQLite catches when it prepares the statement, still comes back as a clean 500. The commit names what a failure in any later batch becomes: "a silent half-listing". It is exactly that. Injecting a failure into the third batch of the rebuilt handler returns 200 OK, a correctly terminated chunked body, and 564 of 8,608 jobs with no closing bracket. And the reader in the browser ends its loop when the stream ends, whether or not the closing line arrived, then marks the listing complete. By the code, the page would show 564 jobs as if that were the whole board. The closing ]} is already on its own line. The reader just has to insist on seeing it.

The total. Before, the count on the page was the length of the array. Now it climbs while jobs arrive. The commit handles that with care. It adds a line under the pager "so a page count that grows while you look at it reads as loading rather than as a bug", and anything that concludes something from a job's absence waits for the stream to end, since "'nothing matched your search' is a lie while jobs are still arriving". But the server knows the total before it writes a byte. It is the length of the id list it just sorted. Streaming did not take the total away. This handler just doesn't send it, and putting it on the first line would cost nothing.

One moment in time. The old listing was one query. The new one is a query for ids, then a query per batch. A job archived between the two is skipped without a word, and a job crawled mid-stream doesn't appear. For a job board that is fine. It is still a choice, and worth making on purpose.

The other half

Four days later, 0c22cb3 went after the size of the corpus itself, starting from "Every job we have ever crawled is still on the board", 10,161 rows and none archived. It stamps each job with the last crawl that still listed it. On feeds that list their whole board, it archives jobs that haven't been seen for about seven crawls, and it refuses to archive more than 20% of a feed at once, so an empty response can't wipe out a source. The listing has excluded archived rows since 2024, so the stream gets shorter by itself as soon as anything is archived.

As of this writing, nothing has been. The switch, archive_stale, ships off, and nothing in the repository turns it on. Worse, the commit's migration wasn't run on deploy, and production ran the new code against the old schema. "Nothing was crawled between Aug 26 and now," says the fix five days later, which made migrations run on every deploy. The commit meant to stop serving dead jobs first stopped the crawlers finding live ones.

It is still the right half to work on. The fastest way to stream 8,608 jobs is to stop listing the ones that are gone. That part hasn't happened yet.

What to take from it

• Streaming changes when the user sees something, not how much work there is. Whatever runs before the first flush still scales with the data. Find it, and index it or shrink it.
• Decide what the first line says. If the server knows the total, send it.
• Make the end of the stream something the client checks for. A truncated 200 that looks complete is worse than a 500.
• Pair it with doing less. Sending 8,608 jobs faster is good. Not sending the dead ones is better.