# transformers.js's progress_callback Fetched Every Model File Twice

I run OpenAI's Whisper in the browser through `@huggingface/transformers` 4.3.1, as an `automatic-speech-recognition` pipeline on WebAssembly with 8-bit weights. The model is served from the same site, not from Hugging Face. To show download progress, the worker used to pass `progress_callback` into `pipeline()`.

On the live site, the first transcription failed a few megabytes in with a generic "network error". The underlying Chrome failure, recorded in the commit this post is based on, was `net::ERR_CACHE_WRITE_FAILURE`. This post is about why that happened, and how progress is counted now without that callback.

## What `progress_callback` does in 4.3.1

I read the library's 4.3.1 source in `node_modules` while writing this. When `pipeline()` is given a `progress_callback`, it first calls `get_file_metadata` for every expected model file so it can emit a total. That is meant to be a cheap size check.

For a remote `http:` or `https:` URL, `_get_file_metadata` is careful: it sends `Range: bytes=0-0` (`fetch_file_head`) and reads the size from `Content-Range`. A site path is not treated as a URL. This project sets `env.localModelPath` to `/models/` on purpose — a 4.3.1 comment in the worker says the library only checks that a local file exists when the path is not a URL, and otherwise loads no tokenizer.

For a non-URL local path, the metadata helper falls through to `getFile(localPath)`. In the browser that is a normal `env.fetch` of the whole file:

```js
export async function getFile(urlOrPath) {
  // …filesystem branch omitted…
  return env.fetch(urlOrPath, {
    headers: getFetchHeaders(urlOrPath),
  });
}
```

It does that only to read `Content-Length`. The pipeline then fetches the same files again to load them. The two weight files in this model are `encoder_model_quantized.onnx` and `decoder_model_merged_quantized.onnx`. Each was requested twice at once.

The commit records a DevTools check: four `.onnx` requests before the change, two after. I did not re-count those requests myself.

## Why one of those requests failed

The model folder is named after a pinned revision, so the files never change under that name. `next.config.mjs` therefore serves `/models/:path*` with:

```http
Cache-Control: public, max-age=31536000, immutable
```

Two overlapping GETs of the same immutably cached URL is the situation the commit ties to `ERR_CACHE_WRITE_FAILURE`. Even if the cache write had succeeded, the model would have been downloaded twice. I have not reproduced the Chrome error myself; I am describing the source and the commit record.

## Counting bytes without the callback

The worker no longer passes `progress_callback`. Progress is counted from the body of a wrapped `env.fetch`, and the known total comes from the pinned file list (`SPEECH_MODEL_BYTES`, about 81 MB — the ONNX runtime is not included in that figure):

```ts
const received = new Map<string, number>();
const baseFetch = env.fetch;
env.fetch = async (input: string | URL, init?: RequestInit) => {
  const response = await baseFetch(input, init);
  const url = typeof input === "string" ? input : input.href;
  if (!url.includes(SPEECH_MODEL_PATH) || !response.body || !response.ok) {
    return response;
  }
  let seen = 0;
  const counted = response.body.pipeThrough(
    new TransformStream<Uint8Array, Uint8Array>({
      transform(chunk, controller) {
        seen += chunk.byteLength;
        received.set(url, seen);
        let loaded = 0;
        for (const n of received.values()) loaded += n;
        post({ kind: "download", loaded, total: SPEECH_MODEL_BYTES });
        controller.enqueue(chunk);
      },
    }),
  );
  return new Response(counted, {
    status: response.status,
    statusText: response.statusText,
    headers: response.headers,
  });
};
```

`pipeline()` is then called with only `device: "wasm"` and `dtype: "q8"`. The wrapper still lets the library fetch each file once. It only taps the stream, sums bytes per URL, and posts `{ kind: "download", loaded, total }`.

## What the test pins

`tests/transcribe.test.ts` now asserts that the worker source does not contain `progress_callback:` and does contain `env.fetch = async`. I read that assertion. I did not run the suite, and the test does not drive a browser or assert the request count.

## Takeaways

- In transformers.js 4.3.1, `progress_callback` is not a passive listener. On a local site path it triggers a full GET of each file just to learn its size, then a second GET to use the file.
- The library's cheap Range probe applies to `http:` / `https:` URLs. A path such as `/models/...` does not take that path.
- `Cache-Control: immutable` and two concurrent writes of the same URL are a bad pair. A "network error" a few megabytes in may be a cache write failure, not a dropped connection.
- If you already know the file sizes — this project pins them next to hashes — count bytes on the one fetch you cannot avoid, and do not ask the library to measure them first.

The page that uses this worker is [AUP4 to Text](https://aup4.aivismonitor.com/aup4-to-text). **Disclosure: I built it.**

---

*Source note (2026-11-12): I read Waveward commit fe8fe14 (AuthorDate 2026-11-12 11:56:00 +0800), the current `src/lib/transcribe/transcribe.worker.ts` and `model.ts`, `next.config.mjs` cache headers, the new assertions in `tests/transcribe.test.ts`, and `@huggingface/transformers` 4.3.1 (`get_file_metadata.js`, `getFile` in `hub.js`, and the `progress_callback` branch in `pipelines.js`) on the Mac. I did not run the test suite or re-trace the network panel.*