BLAH Lens Bundle v1

optional

A stack-neutral format for exposing what happens inside a model, so evals.blah.dev can render a site-by-token view of its internal representations.

The boundary

evals.blah.dev owns the interoperability contract and the visualisation. Each model framework owns faithful execution of its own model. Hugging Face owns immutable distribution of model and lens artifacts.

Nothing in the format or protocol is named after any framework. A bundle produced by PyTorch, JAX, MLX, Candle, Rust, Zig or a bespoke TypeScript engine is indistinguishable to the platform.

This is entirely optional. Models without a lens are registered, evaluated and ranked exactly as before. A bundle only unlocks the model-internals view.

Names

Bundle format:    blah-jacobian-lens
Format version:   1
Runtime protocol: blah-lens-http/1
HF library tag:   blah-jlens

How a model gets a lens

Three related resources: the model checkpoint on Hugging Face, a lens artifact (usually a second repository, because lens artifacts are checkpoint-specific, can be large, and may have several fitting variants), and an execution adapter — either loaded by a worker or exposed by you over HTTP.

your-org/model-name
your-org/model-name-jlens     # recommended

# or inside the model repo
your-org/model-name/evals/jacobian-lens/v1/

Hand this to your coding agent

Run this from the root of your model repository. It inspects your architecture, builds a thin adapter around your native implementation, fits the transports, validates, and prepares the Hugging Face upload — without moving your model into someone else's framework.

or fetch it directly:

curl -L https://evals.blah.dev/lens/agent-prompt.txt -o BLAH_LENS_TASK.md

Verify before you publish

The platform exposes the same validator it runs on import, so nothing needs installing and a bundle cannot pass your check and fail ours. A bundle that fails fatally is rejected rather than stored — a broken lens produces confident, wrong readouts, which is worse than no lens. The response is the full report: every check, its status, and why it failed.

npx @blahai/lens validate ./dist/blah-lens
npx @blahai/lens validate-runtime --url http://localhost:8000 \
  --manifest ./dist/blah-lens/lens-manifest.json

# exit codes: 0 pass, 1 fail, 2 partial

# or with nothing installed:
curl -sX POST https://evals.blah.dev/api/v1/lens/validate \
  -H "Content-Type: application/json" \
  -d '{"lens_repo":"your-org/model-name-jlens","lens_revision":"<sha>"}'

Machine-readable schemas: manifest · protocol

Attach it

Once the bundle is on Hugging Face, attach it to your registered model. The revision must be an immutable commit SHA — a branch name would let the artifact change underneath a published result.

curl -X POST https://evals.blah.dev/api/v1/models/MODEL_ID/lens \
  -H "Authorization: Bearer blah_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"lens_repo":"your-org/model-name-jlens","lens_revision":"<40-char sha>"}'

The response carries the full conformance report. Your model then has a /models/<id>/lens page.

Execution modes

remote_http — your runtime stays under your control and implements the protocol. Supported today.

huggingface — a platform worker loads your model from Hugging Face. The bundle format is defined; the isolated worker is not deployed yet.

oci — your runtime as a reproducible container, pinned by image digest. Defined; not deployed yet.

precomputed_only — no live prompts; uploaded example runs display and the UI marks live analysis unavailable.

Terminology

The format says site, not "layer" or "residual stream". A transformer's sites happen to be post-block residuals, but a recurrent, state-space or hybrid model's are not, and the grid must not imply they share semantics. Sites are ordered, token-aligned representations — whatever those are for your architecture.