Reading a checkpoint in the browser

A model checkpoint is one of the few files people routinely load without ever looking inside. You download several gigabytes, hand them to a framework, and trust that what came out the other end is what you expected. The weights visualiser opens the file in the browser instead — every tensor's name, dtype, shape, distribution and a real heatmap of its values — and never sends a byte anywhere. This post is the whole reader: the two container formats, the pickle virtual machine you need for one of them, the half-precision decoders JavaScript doesn't provide, and the striding trick that lets a 7 GB file be explored without ever materialising a single tensor.

What a checkpoint actually is

Strip away the framework and every checkpoint format on earth is the same three things: a manifest (names, dtypes, shapes), an offset table (where each tensor's bytes start), and a blob. The formats differ only in how much ceremony surrounds those three facts — and, crucially, in whether recovering them requires executing code.

safetensors is barely a format

The whole spec fits in a sentence: eight bytes of little-endian uint64 giving the header length, that many bytes of JSON, then the tensor data end to end. Offsets in the header are relative to the end of the header, so a tensor's absolute start is 8 + header_len + data_offsets[0]. That is the entire reader:

MK.parseSafetensors = function (buf) {
  var dv = new DataView(buf);
  var n  = Number(dv.getBigUint64(0, true));                 // header length
  if (n <= 0 || n > buf.byteLength - 8) throw new Error("bad safetensors header");
  var H  = JSON.parse(new TextDecoder().decode(new Uint8Array(buf, 8, n)));
  var base = 8 + n, raw = [];
  for (var name in H) {
    if (name === "__metadata__") continue;
    var info = H[name], shape = info.shape || [];
    raw.push({ name: name, dtype: ST_DT[info.dtype], shape: shape,
               numel: prod(shape), byteStart: base + info.data_offsets[0] });
  }
  return { raw: raw, dv: dv, format: "safetensors" };
};

Fourteen lines, no allocation proportional to file size, and it works on a 40 GB shard as readily as on a 2 MB adapter because nothing but the header is touched. This is why safetensors exists and why it won: there is no code-execution surface, because there is no code — only an offset table. The security argument and the performance argument turn out to be the same argument.

torch .pt is a ZIP with a pickle in it

The other format people actually have is a torch checkpoint, and that is a different proposition. A .pt is a ZIP archive; inside it are a data.pkl describing the object graph and a data/ directory of raw storage blobs. To read it you need two things browsers do not have: a ZIP reader and a pickle interpreter.

The ZIP half is easy, for one specific reason

Torch writes archives stored, not deflated — the tensor bytes are incompressible anyway and torch would rather have mmap-ability than a 2% saving. So no inflate is needed. Scan backwards from the end for the end-of-central-directory signature 0x06054b50, walk the central directory collecting names and local-header offsets, then skip each local header's variable-length fields to find where the payload begins:

// central directory entry → { method, csize, lho }
var name = new TextDecoder().decode(u.subarray(p + 46, p + 46 + fnl));
E[name] = { method: dv.getUint16(p + 10, true),
            csize:  dv.getUint32(p + 20, true),
            lho:    dv.getUint32(p + 42, true) };

// the local header is 30 bytes + filename + extra field, then the bytes
function at(e) {
  var lfnl = dv.getUint16(e.lho + 26, true),
      lefl = dv.getUint16(e.lho + 28, true);
  return e.lho + 30 + lfnl + lefl;
}

Those two lengths must be read from the local header, not reused from the central directory — ZIP permits them to differ, and torch's 64-byte alignment padding lives in exactly that extra field. Reuse the central values and every tensor is offset by a few dozen bytes, which decodes as plausible-looking noise rather than as an error. That is the nastiest bug in this file, and the symptom is a histogram that looks almost right.

The pickle VM

Pickle is a stack machine, and that is the part that sounds alarming and isn't. Opcodes push values, build tuples and dicts, memoise into a side table, and call reduction functions. You do not need all of it — you need the subset torch emits, which is about thirty opcodes, plus two special cases:

The reduce handler is a whitelist and nothing else. Everything unrecognised collapses to an inert marker that is treated as data and never invoked:

function reduce(fn, args) {
  var g = fn && fn.__g;
  if (g === "torch._utils._rebuild_tensor_v2" || g === "torch._utils._rebuild_tensor")
    return { __t: true, storage: args[0], offset: args[1] || 0, shape: args[2] || [] };
  if (g === "torch._utils._rebuild_parameter") return args[0];
  if (g === "collections.OrderedDict")         return new Map();
  return { __g: g, args: args };          // recognised as data, never called
}

Persistent ids carry the dtype in the storage class name — FloatStorage, BFloat16Storage, HalfStorage — which is why the mapping is a table keyed on strings rather than on any structured type field:

function persid(pid) {                            // ("storage", cls, key, device, numel)
  var tg  = pid[1] && pid[1].__g ? pid[1].__g : String(pid[1]);
  var cls = tg.split(".").pop();                  // "FloatStorage"
  return { __s: true, dtype: TORCH_DT[cls] || null, key: String(pid[2]), numel: pid[4] };
}

Resolve those two and you have, for every tensor, a name, a dtype, a shape and a byte offset into the archive — exactly the same four facts safetensors hands you for free. From there both formats flow through identical code, and MK.buildModel never learns which one it got.

Half precision, by hand

JavaScript has Float32Array and Float64Array and nothing between. Modern checkpoints are almost entirely f16 or bf16, so both decoders are written out. bfloat16 is the easy one — it is literally the top 16 bits of a float32, so shift it back up and reinterpret through a scratch buffer:

var _bf = new DataView(new ArrayBuffer(4));
function bf16ToFloat(h) { _bf.setUint32(0, (h << 16) >>> 0, true); return _bf.getFloat32(0, true); }

function halfToFloat(h) {                                  // IEEE 754 binary16
  var s = (h & 0x8000) >> 15, e = (h & 0x7C00) >> 10, f = h & 0x03FF;
  if (e === 0)  return (s ? -1 : 1) * Math.pow(2, -14) * (f / 1024);   // subnormal
  if (e === 31) return f ? NaN : (s ? -Infinity : Infinity);
  return (s ? -1 : 1) * Math.pow(2, e - 15) * (1 + f / 1024);
}

The >>> 0 in the bf16 path is load-bearing: h << 16 produces a signed 32-bit value in JS, and handing a negative number to setUint32 is not what you meant. Half of every bfloat16 tensor is negative, so the bug would fire on half the weights and leave the histogram looking merely lopsided.

The subnormal branch matters more than it looks, too. After sustained weight decay a meaningful fraction of a trained f16 tensor is subnormal; returning zero there would quietly flatten the interesting left tail of every distribution in the app.

Never materialising a tensor

An embedding matrix might be 128 000×4096 — half a billion values, two gigabytes at f32. Rendering it as a heatmap needs perhaps 80×80 of them. So the toolkit never reads a whole tensor at all. Two primitives do the work, both reading straight out of the DataView at computed offsets, with no intermediate copy of anything.

sample() strides through a tensor and accumulates summary statistics in the same pass — min, max, mean, standard deviation via the sum-of-squares identity, and a near-zero count that is the cheapest available sparsity probe:

MK.sample = function (dv, byteStart, numel, dtype, maxN) {
  var R = READERS[dtype]; maxN = maxN || 4096;
  var stride = Math.max(1, Math.floor(numel / maxN));
  var out = new Float32Array(Math.ceil(numel / stride));
  var idx = 0, min = Infinity, max = -Infinity, sum = 0, sq = 0, z = 0;
  for (var i = 0; i < numel; i += stride) {
    var v = R.read(dv, byteStart + i * R.size);            // ← the only I/O
    out[idx++] = v;
    if (v < min) min = v; if (v > max) max = v;
    sum += v; sq += v * v; if (Math.abs(v) < 1e-6) z++;
  }
  var mean = sum / idx;
  return { values: out.subarray(0, idx),
           stats: { min: min, max: max, mean: mean,
                    std: Math.sqrt(Math.max(0, sq / idx - mean * mean)),
                    zeros: z / idx, n: idx } };
};

grid() is the 2D version: rows are shape[0], columns the product of the rest, with independent row and column strides landing the reads on a lattice. The dtype layer is a table of ten readers, so adding a format is one line rather than a branch in every loop.

Finding the experts

Mixture-of-experts structure is discovered by name, not declared — no checkpoint format has a field for "this is an MoE". Names in practice look like model.layers.3.block_sparse_moe.experts.5.w1.weight, so one regex groups tensors by layer prefix and expert index, and a second pass finds the routing matrix under the same prefix:

var m = t.name.match(/^(.*?)experts?\.(\d+)\.(.*)$/);      // prefix, index, leaf
if (m) { var Lr = layers[m[1]]; (Lr.experts[+m[2]] = Lr.experts[+m[2]] || []).push(t);
         maxE = Math.max(maxE, +m[2]); }
…
if (/gate|router|\.wg\b/.test(t.name)) { /* attach to the layer sharing its prefix */ }

return { detected: order.length > 0 && maxE >= 1, expertCount: maxE + 1, topk: 2, layers: … };

It is a heuristic over a convention — Mixtral, DeepSeek-MoE, Qwen-MoE and Grok agree closely enough — and a checkpoint that names things differently reports no MoE rather than mis-detecting one. Failing closed is the right default for a detector whose output feeds a pruning recommendation.

Using it yourself

In the page: drag any .safetensors, .pt, .pth, .bin or .ckpt onto it. Nothing uploads; the file is read through a FileReader into an ArrayBuffer and stays there, subject only to your tab's address space.

In your own code: shared/modelkit/modelkit.js is dependency-free and about 120 lines. The whole API is five calls:

<script src="modelkit.js"></script>
<script>
const buf   = await file.arrayBuffer();
const model = MODELKIT.parseFile(buf, file.name);   // sniffs the container

model.totalParams                                   // 46702792704
model.tensors.length                                // 995
model.moe.detected && model.moe.expertCount         // 8

const t = model.byName["model.layers.0.self_attn.q_proj.weight"];
const s = model.sample(t, 8192);                    // { values, stats }
console.log(s.stats.std, s.stats.zeros);            // 0.0198  0.0001

const g = model.gridOf(t, 80, 80);                  // { data, rows, cols, amax }
</script>

Three things this is immediately good for, none of which need a GPU or a framework:

• Verifying a download. A truncated file, or an LFS pointer where you expected weights, fails at the header rather than after a forty-second framework load. Shapes and dtypes are visible before anything is allocated.
• Spotting a broken train. A layer whose std is orders of magnitude off its neighbours, or whose zeros fraction is 1.0, is a dead layer — visible in the per-tensor stat line long before an eval would surface it.
• Auditing a merge or a quantisation. Open both files, compare the same tensor's heatmap. Structural damage from a bad merge shows up as banding you can see.

The synthetic model

With no file loaded the page builds a seeded Mixtral-shaped MoE — 4 layers, 8 experts, d=96, d_ff=192 — from a small deterministic RNG, laid out as real bytes behind a real DataView so every path (heatmap, histogram, routing, signatures) is exercised exactly as it would be for a loaded file. It exists so the page is never empty, but it earns its keep as a fixture: experts 0–3 are distinct archetypes and 4–7 are those archetypes plus 16% noise, so the redundancy analysis in the pruner has something real to find.

var arche = em ? (parseInt(em[1], 10) % 4) : null;    // expert index mod 4 → archetype
var er    = MK.rng(1000 + arche * 17 + wi);           // same archetype → same seed
f32[cur + j] = (er() * 2 - 1 + (em ? (noiseR() * 2 - 1) * 0.16 : 0)) * scale;

Break the similarity maths and the demo stops showing four obvious pairs — a regression test you can see from across the room.

What it deliberately doesn't do

No GGUF. No sharded-index resolution — model.safetensors.index.json points at files this reader would need handed to it one at a time. No dequantisation of int4/int8 blocks back to float, and no tensor arithmetic: you can look, not compute. Those are gaps rather than positions. The only principled restriction is the pickle whitelist, and that one isn't moving.

Related: what to do once you can see the experts — deciding which experts to drop.