Engine release notes

NeuroBrix 0.6.4

NeuroBrix 0.6.4 is a corrective release of 0.6.3, with nbxtensor 0.0.9.

• macOS 14 and CPython 3.10–3.13 on Apple GPUs. The macOS 14 Metal wheels (Python 3.10–3.13) are published (triton-ext release); python -m nbxtensor install-metal now selects them: on macOS 14 every covered Python gets the macOS 14 build, and Python 3.12 on macOS 15 or later keeps the macOS 15 build. The Triton engine runs on macOS 14: its CPU pool no longer needs Accelerate's thread API of macOS 15, and python -m nbxtensor doctor says when Accelerate's threads are not controlled (VECLIB_MAXIMUM_THREADS=1, set before Python starts, is the user's choice).
• A first launch with the Triton engine no longer stops when the reload it times holds a zero-size constant (Sana 1024's text encoder on a fresh machine with 0.6.3).

Known limit: no macOS 14 machine has run this release: the evidence is Accelerate's macOS 14 BLAS simulated on a newer macOS (TinyLlama, whisper and Sana outputs equal by sha) and all 238 Metal kernels of those three models compiled for macOS 14's Metal language (Metal 3.1, -mmacosx-version-min=14.0) without an error.

Install: pip install neurobrix (the Triton engine, no PyTorch) or pip install "neurobrix[pytorch]"; on a Mac, python -m nbxtensor install-metal. The library is nbxtensor 0.0.9.

The full list is in the CHANGELOG.

NeuroBrix 0.6.3

NeuroBrix 0.6.3 serves models: neurobrix serve keeps a model warm in a daemon, neurobrix chat talks to it, and the same daemon now answers an OpenAI-compatible HTTP API (serve --http, default 127.0.0.1:8765: /v1/models, /v1/chat/completions with streaming and tool calls, /v1/completions, /v1/images/generations, /v1/audio/transcriptions, /v1/audio/speech). A conversation keeps its KV cache between turns, so each turn prefills only its new tokens (run --turns does the same from the command line). Every model has one name, its hub name (TinyLlama/TinyLlama-1.1B-Chat-v1.0), recorded at import.

The terminal shows what the user needs (-v / -vv for the rest), and the user-path failures of 0.6.2 are fixed or said in one line. Laptops with an integrated GPU are detected from what the operating system ships: an integrated GPU is planned as unified memory, and a GPU no backend drives is refused by name while the machine runs on its CPU. On Apple GPUs, a first run that meets an out-of-memory now re-plans and continues. The macOS 14 Metal wheels (Python 3.10–3.13) are published and install by hand (triton-ext release); install-metal installs macOS 15 / Python 3.12 only, and chooses the macOS 14 wheels from nbxtensor 0.0.9 / engine 0.6.4.

Install: pip install neurobrix (the Triton engine, no PyTorch) or pip install "neurobrix[pytorch]"; on a Mac, python -m nbxtensor install-metal. The library is nbxtensor 0.0.8.

The full list, the fixes and the known limits of this release are in the CHANGELOG.

The Apple certified directory is complete under the published Triton build for Apple GPUs: 18 552 kernel settings are certified, among them the settings 0.6.1 left to a first-run tuning, and the arena probe never tunes a kernel. The kernels that compute a model on NVIDIA are 0.6.1's.

Eight Apple kernel settings stay refused by size on an 18 GB Mac (the certifier's synthetic draw and its oracle do not fit beside each other there): Real-ESRGAN x8's fp32 convolution at 8192 x 8192, Sana 4K's VAE convolutions, an RMS norm of 16M x 128 rows, softmaxes of 25M x 1024 and 25M x 256 rows, and one bf16 convolution (48, 192, 480, 832). A model that launches one of them tunes it once at its first run and keeps the setting; of the models the release trial runs, Sana 4K is the one that does.

The Triton arena factor stays declared per hardware family (vendors/family_facts.yml): a measure on a user's machine holds a fixed part and a part proportional to the activations, which one factor does not describe, so no measured factor is written into a profile.

Fixed

• The kernel certifier prices the size it draws: on a unified-memory device (Apple) a key is drawn at the size its phase price fits, but the door before any draw priced it at its recorded top — the open element-wise class (its label, 68 719 476 736 elements) asked 2.2 TB of an 18 GB Mac and was refused at every re-proof, so a new Mac tuned it once at its first Sana run.
• The kernel certifier holds only what its oracle reads: a row-windowed oracle (softmax, layer and RMS norms, group norm) reads three windows of rows, but the certifier kept the whole host draw for it and uploaded it again at every call; on a unified-memory Mac the draw and its device copy share one pool, and the largest keys of HAT-L, HAT-S and mochi were refused (20-33 GB asked of 18). The draw is now uploaded once and only its windows' rows stay on the host.
• The Apple certified directory holds every setting the reference models launch on an 18 GB Mac but eight: the 22 settings the certifier's price refused at its re-proof under the published Triton build for Apple GPUs (0.6.1 tuned them once at a first run) and the second pass's are certified, 18 552 in all, with two of chatterbox's attention settings the census listed in fp32 where its run launches them in bf16.
• The arena probe never tunes a kernel: run as a tool, it reads only certified kernel settings; a request that would need a setting the directory does not serve is the measure's bound, never a tuning at run time.

A first run never ends in a Traceback, the package names none of the project's machines, and the documentation says where NeuroBrix runs and how to start. The kernels that compute a model are 0.6.0's: its final gate is carried, and five reference models through the entry point give the same outputs; what changed is how a new machine starts (its first-run measures, its arena factor, its home), what the packages carry, and Windows.

The Apple certified directory is proven again under the published Triton build for Apple GPUs (18 352 kernel settings);

Changed

• The README and the documentation say where it runs and how to start: Linux, Windows and macOS; on a Mac, NeuroBrix's Triton build for Apple GPUs (NeuroBrix/triton-ext); every neurobrix import example carries --accept-license, which the import asks of a model with a license (the README's first run stopped there).

Fixed

• A first run on a new machine never ends in a Traceback because its arena probe ran out of memory: the first launch measures the Triton arena's hold on the cache's smallest model by doubling its request while a prediction says the next fits; on an Apple GPU the prediction let a request through that the device could not hold, the probe's process ran out of memory, and its error came back as the interpreter's shutdown noise ("UNEXPECTED ERROR" at the first neurobrix run). The device's refusal is now the measure's bound (the largest request that fitted is kept), and the probe's own error is said by name.
• A new machine's first run does not measure the Triton arena: the first launch probed what the Triton arena holds over a component's activations on the cache's smallest model, and that probe scaled its inputs unlike the request Prism prices: on a fresh Mac it measured 28 where the machine's value is about 1.5, so Prism reserved some 19 times the activation memory, and once corrected it still depended on the model imported (2.4 on a VAE) and swept uncertified kernels for 40 minutes. The factor is now declared — by the device's vendor file, else by its family (Apple 1.477, measured on an M4 Pro; NVIDIA 1.35, measured on Volta; AMD and Intel the NVIDIA value until measured) — and a first run never probes it; a value an earlier first launch wrote into a profile is no longer read. The probe scales only what the priced request scales (a tool, until it is bounded to certified shapes).
• A run starts on Windows: the plan's resident term is read from the process's working set (GetProcessMemoryInfo) where it needed /proc or libproc, and a first-launch read-rate measure Windows cannot make (no way to read past its page cache) is declared unmeasured and the run reads serially, said on its launch line. Written and tested with Windows simulated: the project has no Windows machine yet.
• Text in UTF-8 on every platform: a generated text written to a file, the vendor tokenizer files read at load, the agent's file edits and transcript, and the console output now use UTF-8 where the platform's default code page is another (Windows' cp1252: a wrong vocabulary read silently, or a run ended by UnicodeEncodeError); a debug dump's file name drops every character a file name refuses on some system; a stale install lock is asked of the kernel on Windows instead of signalling the process (signal 0 is Ctrl+C there).
• The engine depends on nbxtensor 0.0.5: a certified setting the library does not serve says why (another code generator, or another memory class).
• A configured home is followed everywhere: with NEUROBRIX_HOME set, the replay cache, the license acceptances, the hardware profiles, the calibration records, the render checkpoints and the serving daemon's files still went to ~/.neurobrix; they now follow the configured home.
• neurobrix doctor on a Mac without PyTorch told the user to pick a CUDA build of PyTorch; it now says pip install torch, PyTorch's own macOS wheel.
• The package names none of the project's machines: the snapshots root it shipped was a path of the project's own machine (wrong on every other machine) and is now ~/hf_snapshots (NEUROBRIX_HF_SNAPSHOTS overrides it); the census input records name the files they read relative to the engine's home (home:) instead of a user's home directory; the certified directory's proofs no longer record the host name of the machine that certified them (85 723 removed, and the certifier no longer writes it).
• A first run on a busy storage is never a Traceback: at its first launch the engine measures how many parallel reads the machine's model cache serves best; where no stable read rate was found (a storage another reader was using — a clean container whose cache was a network mount), the run ended in "UNEXPECTED ERROR: ZERO FALLBACK: no stable read-rate measure" and a Traceback. The run now reads serially, its launch line says so ("io workers: 1 for this run — ...; measured again at the next run"), and the profile keeps no count, so the next run measures again.

Install: pip install neurobrix — the Triton engine, no PyTorch. With the PyTorch engine as well: the PyTorch build for your hardware, then pip install "neurobrix[pytorch]". On an Apple-silicon Mac (macOS 15 or later, Python 3.12): pip install neurobrix, then python -m nbxtensor install-metal. neurobrix doctor says what each engine needs on your machine.

The release that moves NeuroBrix's Triton engine onto nbxtensor (0.0.4). Its final gate ran the 49 catalogue models through the entry point in both engines on both V100 card classes, every output judged by eye and every time in the gate register; that gate is this release's benchmark, per model. In short:

• The Triton engine imports nbxtensor, not PyTorch: --triton runs on nbxtensor, Triton and NumPy, and PyTorch is no longer a required dependency; the engine, the library and five reference models run in an environment where torch is not installed, with no attempt to import it.
• Every kernel setting is certified per launch key and per card class: the keys a model forms are derived from its graph without running it, certified once against a float64 oracle, and served certified-only — a missing key is an error naming its census row, never a sweep in the middle of a run.
• The V100's tensor cores (m8n8k4) carry the matrix products where certified faster: GEMMs, the mixture-of-experts prefill projections, fp32 1 x 1 convolutions and transposed 1-D convolutions.
• The replay fixes: a replayed component returns its views in order, a long video's encoding no longer runs the card out of memory at its replay, a replay plan recorded against an old KV cache is retired, a layer-streamed piece that frees its inputs is not replayed, and the replay's check says what it observed — a match with the normal path on one step, not a proof of the output.
• Apple Metal on the Triton engine: nbxtensor's Metal driver runs the engine on Apple GPUs, with its own certified directory and a vendor file of measured facts (the conv3d workspace, the arena factor, the reload rate); Triton for Metal is installed by python -m nbxtensor install-metal (nbxtensor 0.0.4).
• The TilingEngine cuts only where the graph proves the cut exact, refuses what it cannot place instead of falling back, and no longer reads host switches.
• The light install runs as installed: pip install neurobrix is the Triton engine alone (1.48 GiB installed), pip install "neurobrix[pytorch]" adds the PyTorch engine; a run with no mode flag takes the engine that is installed and says which on its launch line.
• Prism plans from the machine's facts: the hardware profile and vendor files carry what it prices (measured reserves, reload rates, workspaces), a busy machine is re-planned on the memory really free with a clear warning, and a model is refused from one place only when no plan fits.
• Prism prices a PyTorch-engine load as the loader performs it: one tensor at a time per worker, the pinned host cache emptied after each component, a host-held component loaded unpinned then copied into one registered block — where it priced every component read and pinned whole (Qwen3-Coder-30B: 115 GB of loading priced, 62.5 GB measured).
• Certify your GPU and share it: the README's four steps and the certificate pull request template.
• AMD and Intel GPUs in the Triton engine (nbxtensor's HIP and Level Zero drivers, written, not yet run on a card): vendor files for AMD CDNA4, RDNA3 and RDNA4 and Intel Data Center GPU Max, Arc A and B and Lunar Lake, and neurobrix doctor names, for every GPU on the PCI bus, the runtime library and Triton backend it lacks and the vendor's commands that install them.

Install: pip install neurobrix — the Triton engine, no PyTorch (1.48 GiB installed on a clean python:3.10-slim). With the PyTorch engine as well: the PyTorch build for your hardware, then pip install "neurobrix[pytorch]" (6.82 GiB with torch 2.14.1+cu126). On an Apple-silicon Mac (macOS 15 or later, Python 3.12): pip install neurobrix, then python -m nbxtensor install-metal, which installs Triton for Metal from NeuroBrix/triton-ext (the AppleGPU backend by imperatormk, triton-lang/triton-ext#126). neurobrix doctor says what each engine needs on your machine.

The full list of changes (75 added, 97 changed, 2 removed, 330 fixed) is in the changelog.

NeuroBrix 0.5.4 — NVIDIA (CUDA) and CPU

The published version has been 0.5.3 since 2026-09-03, and main is 850 commits past it. Testers have been running an engine we know is wrong on points that are fixed here.

Apple Silicon is not in this release. Its work lives on metal-first-light, unmerged, and its CUDA-side proofs are not all measured yet. Apple ships in 0.5.5, with its numbers already published upstream.

Read this first: installing on an NVIDIA machine

pip install neurobrix alone will give you a CPU-only engine if your driver is CUDA 12.x, because pip resolves the newest PyTorch from PyPI and that is built for CUDA 13. Install PyTorch from the index that matches your driver first:

# Volta (V100) and other CUDA 12.x drivers — see the README for the full table
pip install torch --index-url https://download.pytorch.org/whl/cu126
pip install neurobrix
neurobrix doctor

neurobrix doctor is the check, and it names this exact mistake rather than letting you discover it at the first run:

  PyTorch:        2.14.0+cu130
  built for:      CUDA 13.0
  NVIDIA driver:  535.309.01
  GPU visible:    NO
    But the driver DOES see: Tesla V100-SXM2-16GB (7.0), ...
    So PyTorch was installed for a CUDA version this driver or GPU cannot serve.
  1 problem(s) that will block a real run

Run neurobrix doctor before anything else. If it is green, the engine is ready.

What does not work, said plainly

A tester who knows what to avoid leaves satisfied; one who finds out alone does not.

The video family is under investigation

Wan2.1-T2V-1.3B-Diffusers produces a degenerate output — a textured field with no scene. The cause is not found, and these axes are already excluded by measurement, so nobody needs to repeat them:

• three engine commits, including the one whose artefact was coherent two days before
• two stacks: torch 2.14.0+cu126 and torch 2.5.1+cu121
• autotune on, and NBX_DISABLE_AUTOTUNE=1
• the local replay cache emptied
• the certified autotune directory switched off
• both execution modes, including the compiled oracle
• the container's bytes — CRC-checked against its archive, 45 members, the only difference a build timestamp

Another video model, CogVideoX-2b, renders coherently at 9, 13 and 17 frames, and no other family is affected.

A warning that generalises beyond this bug: degeneracy statistics did not catch it. Per-frame standard deviation 26-59, 163-248 distinct values, frame-to-frame change 1.7-15.9, zero flat rows — every mechanical number reads healthy and the picture is noise, because a fine dither supplies variance, distinct values and inter-frame motion all at once. Judge a video by looking at it.

Automatic op-level tiling is not wired

A request too large for your card is refused with its figures — how many tiles the work would fit in, what each would cost, and what that is as an input scale — and is not reshaped for you:

model: 16,794 MB of activations against 32 MB of weights — it fits in 2 tiles of
about 8,629 MB each, inside a 9,952 MB budget (a floor: a real tiling's halo costs
more); the same reduction from the input side is each spatial dimension at 0.71x

Prism's cascade now contains the tiling rung and consults it before falling back to the host. The detector it asks decides overflow from an op's own footprint against the card, rather than against what is left of the card at that point in the schedule, so it declines on the case that motivated it. Use a smaller input, or the tile count the refusal names.

The memory estimate is not exact, and its error has two signs

Measured on a four-V100 rack, planned against the drained watermark:

An over-estimate is safe against a crash and can place a model on a heavier rung than it needs. No single multiplier corrects both signs.

What does work, and how it was checked

Each family below was judged by an instrument outside the engine, on the release commit:

For NeuroBrix Studio

Checked by reaching the code, not by reading the request list.

Delivered

• --json on every read command — info, list, hub, inspect, coverage, doctor, autotune status, autotune check, run --explain-plan. Exactly one JSON record on stdout (schema neurobrix.<command>/<version> plus engine), every human line on stderr, so stdout can be piped without filtering. Contract: docs/reference/json-output.md.
• Discovery without loading a model — neurobrix info --json and the daemon's status share one identity record: engine, protocol, endpoint, operations.
• A compatibility handshake — every response envelope carries protocol (currently 1) and engine, so a client refuses an engine it does not know.
• The install lifecycle stays in the engine — NDJSON phases with real byte counts, and an install that cannot leave a half-written model behind: a per-model lock, a staging tree carrying host and pid, and a swap done in one syscall where the kernel allows it.

Still open, and not promised here

• Decoded text deltas: the stream emits {"step","n","token","done"} with a token ID, not decoded text.
• Acknowledged cancellation: there is no cancel operation.
• A daemon responsive during inference: the server accepts one connection at a time.
• Windows instance isolation: NBX_SOCKET_PATH isolates the Unix socket; the default Windows loopback port is still fixed.

Upgrading

Nothing in the .nbx container format changed, and no model was re-traced for this release, so your existing containers work unchanged and nothing needs re-downloading from the hub.

This release exists for one reason: a clean pip install neurobrix could not run a single model on a large class of machines, and had not been able to since v0.3.0. The new-user path was walked by hand from an empty virtualenv and was broken three times over — the first-run crash below, a doctor command that reported no problem while PyTorch could not see the GPU at all, and GPU pinning being ignored. All three are fixed and the whole path is verified end to end: install, diagnose, fix, import, first generation.

Fixed

• --triton on an Apple Silicon Mac gave an out-of-date refusal, and exited as if it had succeeded. The message said Triton support on Metal "will be supported in a future version"; an out-of-tree Triton backend for Apple GPUs now exists. NeuroBrix now checks whether one is installed and, if not, names it and the install command — and exits with an error code instead of reporting success. Apple GPUs are also no longer missing their hardware profile, which made any code path that asked for one fail with a missing-config error. As with AMD, these paths are prepared but not validated: no Apple GPU was available, and no support is claimed until they run on one.

• An unsupported GPU now says so, instead of failing with a file path. NeuroBrix reads every hardware-dependent value from a per-architecture profile and refuses to guess when none matches — a guessed block size produces wrong output rather than an error. But the refusal was a bare missing-file error naming an internal path. It now names the GPU that was detected, lists the hardware that is covered, and says the gap is ours to fill rather than a limit of your card. Architectures currently without a profile include NVIDIA Blackwell, AMD RDNA and Vega, and pre-Volta NVIDIA.

• --triton on a CPU-only machine no longer fails deep inside the compiler. NeuroBrix enabled its CPU Triton path without checking that the required triton-cpu package was installed, so the run died later with an unrelated-looking driver error. It now says what is missing, how to install it, and that --compiled already works on CPU today.

• A clean install could not run a single model on many machines — present in every release since v0.3.0 (2026-07-07). After pip install neurobrix, the first run failed with autotune() got an unexpected keyword argument 'cache_results'. The engine used a Triton option that only exists in recent Triton versions, while the install path documented in the README (pip install torch --index-url .../cu121, needed by anyone on a pre-Ampere GPU) brings an older one. NeuroBrix now adapts to the Triton it finds and says so once, instead of crashing: results are identical, autotuning is simply re-measured per process rather than cached to disk.

• neurobrix doctor now checks whether the engine can actually use your GPU. It used to check only that the command was on PATH and reported "No action needed" even when PyTorch could not see the GPU at all. It now reports the PyTorch and CUDA build, the driver, and each GPU — and when the GPU cannot be used it says why and prints the exact command that fixes it. This matters most on a plain pip install neurobrix: pip takes whatever CUDA build is current, which may be newer than your driver, and CUDA 13 dropped support for V100-class cards entirely. doctor now exits non-zero when it finds something that will block a real run.

• CUDA_VISIBLE_DEVICES is now respected when choosing GPUs. Hardware detection listed every card in the machine and ignored the variable, so a process restricted to one GPU could be planned onto a card it was not allowed to use — failing with invalid device ordinal after the model had already loaded, or running on a GPU the user had deliberately excluded. Devices are now filtered and renumbered exactly as CUDA does, and a saved automatic profile is regenerated when the visible GPUs change instead of being reused.

• AMD Instinct MI200 and MI300 cards no longer fail before running anything. NeuroBrix recognised these cards but had no matching hardware profile for them, so startup stopped with a missing-config error on every MI210, MI250, MI250X, MI300A and MI300X. Each CDNA generation now has its own profile, carrying the values that actually differ between them (MI300 has 2.5x the on-chip memory of MI100/MI200, and is the only one with FP8). No AMD hardware was available to test on: these paths are prepared, not validated, and no AMD support is claimed until they run on a real card.

• On AMD cards, a GPU capability was being read from the wrong place. The kernel autotuner decided how deeply to pipeline by asking the NVIDIA driver for a compute-capability number. On AMD that call returns an architecture code instead, which happened to compare as "modern" and silently enabled a pipelining depth that had never been validated on those cards. The setting now comes from each card's own hardware profile. NVIDIA behaviour is unchanged.

Deprecated

• Four hub models are marked deprecated while their published defects are fixed. Each stays listed with its stated reason; nothing is deleted, and each returns to public with its validated fix. canopylabs/Orpheus-3B: speech generated in --triton mode can mispronounce words (the default compiled engine is unaffected). resemble-ai/Chatterbox: generation currently fails with an internal dtype error on both engines. nvidia/Parakeet-TDT-1.1B: long recordings are only partially transcribed (a 10-minute input returns only its opening seconds; short clips are correct). TinyLlama/TinyLlama-1.1B-Chat: prompts longer than ~580 tokens fail with a KV-cache capacity error although the model's window is 2048 (short prompts are correct).

• Back to public. nvidia/Parakeet-TDT-1.1B and hpcai-tech/Open-Sora-v2 return with the long-form transcription and attention-chunk fixes below; Efficient-Large-Model/SANA-Video-2B-720p returns with a container rebuilt by the current toolchain and validated in all four execution modes; canopylabs/Orpheus-3B returns after the reported --triton mispronunciation was traced to a greedy tie between two equally likely tokens (the Triton engine follows the reference op-by-op engine token for token; the compiled engine took the other branch of an exact tie). openai/Whisper-V3-Turbo was replaced with a container that carries the model's timestamp and token-suppression data, which long-form transcription needs.

Added

• The neurobrix hub listing now shows a model's distribution state. A deprecated or withdrawn model displays its state in the STATUS column, so the badge reaches you before the download, not after; the stated reason is returned by the download API.

Fixed

• Serve time-to-first-token no longer pays a per-request graph reload. Every generation request re-read and re-parsed the model's graph description from the runtime cache (244 MB for a 30B mixture-of-experts model, 8 s per request over the network cache) although the loaded executor already holds it. Both engines now read it from the executor; the file is opened only once, when the executor is first created.

• --triton serving no longer reserves the whole card for the key-value cache on the first request. The serving plan sizes the cache for the model's full context window as a ceiling and carries a small initial size; the Triton cache allocated the ceiling up front — 10.8 GB pinned by a short first request on a 32 GB card for a 30B mixture-of-experts model — so every later long prompt ran its attention at the memory edge (chunks halved and retried hundreds of times). The cache now starts at the plan's initial size and re-allocates a layer when a longer request arrives, as the PyTorch engine's cache always did.

• --triton gathers and scatters now fail loudly on an out-of-range index, exactly where PyTorch raises. The index-select, index-put and embedding kernels used to be silent: a bad index skipped the store, wrote outside the tensor, or read the row at the wrong address, and the run went on with corrupted values. Each kernel now traps on the bound (index-put also accepts negative indices, as PyTorch does) and the next device sync raises with the kernel line and the message index out of range.

• --triton gathers now honour negative indices like PyTorch. The index-select kernel treated a negative index as invalid and left the corresponding output element unwritten, so it held whatever the allocation contained — zeros on fresh device memory, stale values once freed blocks are reused. The HAT upscalers index their relative-position bias with negative entries, so their --triton outputs carried a silently wrong (zero) bias, a ±1 difference per pixel that no shape or tone gate could see; the pool's byte gate caught it. Negative indices wrap from the end in both the single-index and the joint-index paths (pinned byte-exact against a NumPy reference).

• Runtime shape rewrites no longer touch a model's architectural constants. The engine promotes concrete sizes that match a traced sequence length to runtime symbols; two of those rules could also rewrite a literal that merely equalled trace length + 1 (the head count of a video transformer, for example), which surfaced as a shape error in the text cross-attention of Allegro-TI2V at any prompt length other than the traced one. Size entries now match exactly (slice ends keep their +1 rule), and the constants declared in a component's profile are never rewritten — in the default and --triton engines alike.

• The --triton engines no longer import PyTorch through package initialisers. Two package inits pulled the scheduler and executor stacks (and PyTorch with them) into the Triton speech path on a plain submodule import; both now export their names lazily. A fresh interpreter importing the shared audio DSP or the decode-bound helper keeps PyTorch out of sys.modules (pinned by a test).

• Long recordings are transcribed in full by both speech-to-text families. Whisper-class models (encoder_decoder flow) now run the vendor's own long-form algorithm: each 30-second window is decoded with timestamps and the next window starts where the last complete segment ended, so no word is cut at a window boundary; the vendor's token-suppression rules apply as well. The timestamp and suppression data come from the model's own generation config, carried by the container: a container built before this release refuses a long input with a clear message naming the rebuild, and short clips are unaffected. RNN-T models (rnnt flow, Parakeet) run buffered inference — overlapping windows merged by the frame each token was emitted at — with the overlap declared in the family configuration. A 10-minute recording that used to return only its opening seconds now returns the whole transcript, identically on both engines. NBX_DISABLE_STT_CHUNKING=1 restores the previous single-window behavior for diagnosis.

• RNN-T / TDT decoding matches the vendor's greedy algorithm. The prediction network's state was advanced on every blank step and a non-blank symbol's predicted duration never advanced time, which dropped or duplicated symbols inside words ("vis" for "visit", "effecteded"). The decode loop now follows the reference exactly: the state is committed only when a symbol is emitted and the duration head advances time after every symbol. On a 10-minute reference the word-level agreement with the vendor's own output rises from 95.6% to 99.0%, and both engines produce byte-identical transcripts.

• A large attention chunk refused by the GPU no longer aborts a video generation. The chunked deterministic attention path retries a refused chunk at half its rows (down to the architecture's row block), keyed on a typed allocator error, and says so loudly. Bytes are identical whether or not the retry fires.

• A refused command could exit with success. When neurobrix run refused a request loudly (for example two execution-mode flags at once), the process still exited with code 0 — scripts and harnesses read the refusal as a pass. Refusals now exit non-zero.

• A runtime shape-recovery pass could corrupt correctly-prepared models. The engine's load-time pass that recovers dynamic shapes on older model containers could re-introduce a shape confusion on models whose containers are already correct (a model constant sharing a value with an unrelated dimension), crashing or corrupting generation at non-default sizes. The pass now carries the same safeguards as the preparation toolchain and can no longer alter a correct container's behavior; older containers keep their recovery.

• resemble-ai/Chatterbox failed to generate any audio. Both engines crashed with an internal dtype error at the first generated token: the speech-head projection assumed the language model's output precision instead of following it. The projection now matches the model's actual output dtype (upcasting when the engine's mixed precision returns full precision). Verified end-to-end on both engines with an independent speech-to-text probe hearing the prompted phrase verbatim. The model returns to public with this fix.

• Prompts longer than a few hundred tokens crashed models whose default generation budget is small. The KV cache was sized from the model's default max_tokens plus a fixed margin and was blind to the actual prompt: on TinyLlama-1.1B-Chat any prompt beyond ~580 tokens failed with a cache-overflow error on both engines, although the model's context window is 2048. The cache now sizes itself from the real prompt at prefill (bounded by the model's own context window, which now produces a clear "model's own limit" message instead of a budget error), and a --max-tokens larger than the model default correctly enlarges the cache too. Short prompts allocate exactly as before. TinyLlama-1.1B-Chat returns to public with this fix.

• Long-context prompts crashed the Triton engine on machines that mix GPU memory sizes. On a host with both 16 GB and 32 GB GPUs, running a large language model with a long prompt (~8k tokens) in --triton mode failed with an out-of-memory error: the attention memory budget was set per GPU architecture, so a working buffer sized for the 32 GB cards was also requested on the 16 GB cards. The budget now adapts to the memory of the specific GPU executing each operation, and the same prompt completes with identical output across repeated runs. As a side effect of the same routing, Sana_1600M_4Kpx_BF16 image generation in --triton mode got about 10% faster (409 s vs 454 s on one V100) while producing an equivalent image. worked at one input size.** Three published super-resolution models — Swin2SR-Classical-x4, SwinIR-Classical-x4 and HAT-L-x4 — shipped with defects that made their output unusable.

  Swin2SR-Classical-x4 shifted every pixel of every image it produced: white backgrounds rendered grey and dark areas were crushed, on both engines. Structure, sharpness and the 4x factor were correct, which is why the defect was easy to miss at a glance and impossible to miss side by side with the reference. The cause was a normalisation step the model applies around its own network that was not captured when the model was prepared, so the engine ran the network without ever undoing it.

  All three models additionally accepted only the exact image size they had been prepared at, failing on any other size with an internal shape error. Real-ESRGAN was unaffected by both defects.

  Swin2SR-Classical-x4 has been corrected and republished. If you downloaded it before 2026-08-28, re-run neurobrix import to pick up the corrected container and regenerate any images made with the earlier one.

  SwinIR-Classical-x4 and HAT-L-x4 have been corrected and republished (2026-08-29). Both now work at any input size — square or not, aligned to the model's window or not — with correct output verified against their reference implementations at multiple sizes and shapes on every supported engine. The interim deprecation badge is lifted; download counts, tags and history are untouched. If you downloaded either model before 2026-08-29, re-run neurobrix import to pick up the corrected container.

  Additionally, for every upscaler: inputs whose size is not a multiple of the model's processing window are now padded to the model's own declared alignment (HAT uses 16 where others use 8 — it previously failed on some sizes), and the output is cropped to exactly input size x scale factor, where the padding could previously leak mirrored edge rows into the result. Real-ESRGAN-x4 was republished with this metadata as well. The same exact-size guarantee now also applies when upscaling through neurobrix serve — the served image previously kept the padding on inputs whose size was not a multiple of the model's window.

Added

• Upscaler outputs are now checked for tonal fidelity, not just size. The upscaler regression suite previously asserted only that the output had the expected dimensions, which is what let the tonal defect above ship unnoticed. It now also asserts, on both engines, that the output preserves the input's overall brightness and that a uniform background survives upscaling, with the tolerances measured against four independent reference implementations.

• Serving releases GPU memory promptly on unload. Unloading a model from a serving engine left freed weights waiting for a garbage-collection pass (executor graphs hold reference cycles), so loading several models in sequence inside one process accumulated GPU memory. Unload now runs the collection step itself before clearing the cache. Note that fully returning every byte between sequential loads in one process is still being worked on — one model per serving process remains the recommended shape.

• Upscalers now work through the serving daemon. Requesting an upscale from a warm neurobrix serve session failed on both engines: the serving path prepared the input image with the video/VLM contract instead of the super-resolution one, so the model's actual input never received the image and saving crashed. The serving path now uses the exact same input preparation as the neurobrix upscale command (one shared, data-driven brick), and a permanent regression test exercises the warm path on both engines with the output resolution verified.

• Flex.1-alpha generates images again. Since the FLUX-video flow landed in late June, image models of the FLUX family (Flex.1-alpha) crashed at the first denoising step with KeyError: 'frames' in both engines: the video-conditioning synthesis keyed on "the model declares FLUX positional ids", which is true for the whole FLUX family, and then read video-only packing metadata that image models do not have. The synthesis now fires only for genuinely video-packed (5-dimensional) latents; FLUX image models keep their own conditioning path, and video models are unaffected. The internal classifier was also renamed to say what it detects (family, not video) so the distinction cannot silently regress.

• DeepSeek-MoE generation no longer degenerates in Triton modes. Models that rebuild their rotary-position tables on every forward pass (DeepSeek-MoE among them) produced garbage from the moment a generation crossed the internal reference length: the Triton engine kept the rebuilt table frozen at that reference size, so later positions read past its end — output collapsed to a repeated "!" (observed on deepseek-moe-16b-chat in both --triton and --triton-sequential). The table now grows with the actual sequence length (prompt plus generated tokens), in both Triton modes and from the very first forward pass; outputs are now byte-identical to the PyTorch engine on the affected model. Models that ship a pre-built rotary table are unaffected.

• Browsing speech models on the hub works again. neurobrix hub --category offered two category names the registry never served (AUDIO, SPEECH) and was missing the three speech categories the registry actually uses; filtering by any speech category ended in a registry error. The command's category choices now match the registry vocabulary (TTS, STT, AUDIO_LLM, alongside the existing categories), in upper or lower case.

Changed

• The --triton engines now cache freed GPU blocks by default. The device allocator's free-list pool (previously opt-in) is on for every run: freed blocks are reused for later requests instead of going back to the driver one by one, and a failed allocation flushes the pool and retries before reporting out-of-memory. Measured on a ~8,300-token prompt with a 30B mixture-of-experts model on one V100-32G: the cold run (load + prompt + first token) drops from 152.5 s to 90.1 s, with byte-identical output. NBX_ALLOC_POOL=0 restores the previous behaviour. The default engine is unaffected.

• Hub catalog correction: Whisper-Large relabeled V3 → V2. The model previously listed as openai/Whisper-Large-V3 on the hub is in fact a large-v2-class artifact: fingerprint verification on the hub-served file itself found 80 mel bins and 99 language tokens (large-v3 uses 128 mel bins and adds Cantonese), with Whisper-V3-Turbo checking out as genuine v3-class under the same test. The entry is now openai/Whisper-Large-V2 — imports use the new address; a copy already imported under the old name keeps working unchanged, since only the label was wrong, never the weights or the transcription behavior.

• Small single-token quantized projections now fill the GPU. The int4 matrix-vector kernel tiled its outputs in fixed blocks of 64 columns, so projections with few outputs — above all the per-layer expert-routing projection of mixture-of-experts models (128 outputs) — launched two thread blocks on an 80-multiprocessor GPU and ran latency-bound at ~3 GB/s. The tile width now narrows automatically until the launch reaches the hardware's occupancy target, read from the architecture configuration file; large projections keep their proven width untouched, and the kernel's byte-parity oracle is proven at every width. Measured under the locked protocol with interleaved arms: +4.3% at short context (38.4 → 40.1 tokens/s on the 30B reference row), outputs byte-identical across arms at all three contexts. NBX_DGEMV_ADAPT=0 restores the fixed width.

• Mixture-of-experts generation gets a second wave of speedup — the expert down-projection now spreads across the whole GPU. The first rewrite of the single-token expert pipeline (below) left its final projection running on far too few thread blocks to fill the GPU (32 programs for 80 multiprocessors, ~46 GB/s of weight streaming). That projection now places each expert on its own grid row — eight times the parallelism, ~248 GB/s — and completes the expert sum in a separate fixed-order reduction, still deterministic by construction; the gated projections also adopt the narrower tile the profile ladder selected. Measured under the locked protocol with interleaved arms: +23.2% / +21.3% / +20.3% at short / 4,164 / ~8,300 tokens on top of the first wave — the 30B reference row reaches 38 tokens/s on one V100. Outputs byte-identical at short and 4,164; the ~8,300 row moved once, pass-stable (a greedy near-tie under the new reduction order, same class as the first wave). NBX_MOEV_DSPLIT=0 restores the fused down-projection; NBX_MOE_VEC=0 still restores the pre-rewrite path entirely.

• Mixture-of-experts generation is dramatically faster — the single-token expert pipeline was rebuilt end to end. At one token, the previous path sorted a trivial routing list through a three-stage alignment, ran three grouped matrix kernels whose every 16-row tile carried a single real row, applied the activation in a separate pass, and summed the expert outputs in another. Two kernels now do all of it: one computes each active expert's gated projections with the int4 weights dequantized on the fly and the activation applied in the epilogue; the second walks the experts in a fixed order and folds the router-weighted sum directly into its accumulator — deterministic by construction, with no intermediate buffers. Measured under the locked protocol with interleaved arms: +27.6% / +24.1% / +20.8% at short / 4,164 / ~8,300 tokens — the 30B reference row crosses 30 tokens/s on one V100 for the first time. Dense-expert models and multi-token prefill keep the previous path. NBX_MOE_VEC=0 restores it everywhere.

• The per-token weight projections run a rewritten vector kernel — all model generation on the Triton engine gets a uniform speedup. The previous matrix-vector kernel accumulated into a large register tile (64 KB per thread block, forcing spills) and under-filled the GPU at small output widths. The rewrite follows the structure of production decode kernels on this hardware class: a small group of output rows per program, wide vectorized weight reads, one small accumulator reduced chunk by chunk in fixed order (deterministic by construction). Measured under the locked protocol with interleaved arms at three context lengths: +7.1% / +6.4% / +6.1% (short / 4,164 / ~8,300 tokens), outputs byte-identical everywhere. NBX_MV_VEC=0 restores the previous kernel.

• Decode attention on Volta now runs a purpose-built vector kernel — large models generate significantly faster, and the gain grows with context. The previous decode attention computed scores through general matrix kernels shaped for tensor cores this hardware generation cannot use for that pattern. The new kernel follows the structure every production decode kernel uses on this silicon (per-thread fused multiply-adds over vectorized cache reads, online softmax, a deterministic fixed-order combine): measured under the locked protocol with interleaved arms, +5.7% at a short context, +16.6% at a 4,164-token context, +30.1% at ~8,200 tokens, outputs byte-identical to the previous path at all three lengths. Born with a float64 correctness oracle, a determinism proof, and a counted route-activation test; NBX_DECODE_VEC=0 restores the previous path.

• The benchmark protocol now locks GPU clocks for the whole campaign instead of merely logging them. The measurement harness pins the SM frequency (default 1380 MHz, chosen to hold within this machine's thermal envelope), samples it during every repetition, and refuses the campaign outright if the lock ever breaks mid-run — clock drift is killed at the source rather than documented after the fact. Locked numbers sit ~10% below boost-clock numbers; reports record the lock frequency so like is always compared with like.

• The opt-in flash-decoding attention kernel (NBX_FLASH_DECODE=1) now defaults to its best measured configuration (a 10-configuration sweep found the former default block size ran 4× slower on this hardware). The kernel remains off by default: re-measured under the pinned protocol it is still slower than the standard attention path at every context length tested, and the measurement notes at the flag site record the full numbers.

• Decode attention no longer copies the KV cache to read it. The KV cache hands out views whose padding makes them non-contiguous in memory, and the attention entry point materialised those views — copying the entire visible K and V for every layer of every generated token, a cost that grows with context length. The attention math path now walks the views directly through stride-aware matrix kernels (verified bit-identical, with a test proving no hidden copy remains), while kernel paths that genuinely need flat memory still get it. Measured under the pinned protocol on the 30B row: +2.4% at a short context, +11.1% at a 4,164-token context (16.7 → 18.5 tokens/s), outputs byte-identical.

• Large-model decoding on the Triton engine is substantially faster again: the per-step host work no longer scales with the size of the stored graph. A traced model's graph carries every operation of the full forward pass, but a replayed decode step executes only a small fraction of it. Three pieces of per-step bookkeeping nevertheless walked the entire stored graph on every generated token — scanning ~70,000 tensor entries to find a handful of inputs, and restoring ~70,000 arena slots one Python assignment at a time. The input scan is now computed once per model, the restore is a single bulk operation, and the replay input check pairs its lists at C level. Measured on a 30B MoE model on one V100-32G under a pinned protocol (5 repetitions per campaign, clocks logged on every run; the before/after comparison spans two sequential same-day campaigns whose logged drift bound, 1–2%, is two orders below the effect): 14.0 → 23.8 tokens/s at a short context (+69%), 11.1 → 16.7 tokens/s at a 4,164-token context (+50%), outputs byte-identical in every repetition. The equivalent input-scan on the PyTorch engine is cached the same way.

• Sampling with top_k is dramatically faster on the Triton engine. The top-k cut used to be applied by masking: rejected tokens were set aside but the vector stayed at full vocabulary width, so every later stage — the sort, the softmax, the cumulative sum, the draw itself — paid for 150,000 entries when at most top_k of them (typically 20 to 50) could ever be chosen. The draw was the worst of these: it samples by a method that needs one random number per candidate, so it was drawing 150,000 random numbers to pick a single token. The candidates are now collected first and everything after runs over that short list, which also removes the sort entirely because the candidates already arrive in order. Measured on a 30B model over a 152k-token vocabulary, the sampling stage of each decoding step fell from 43.5 ms to 3.0 ms.

  The distribution sampled from is unchanged — both forms are exact samplers for the same set of probabilities. What changes is which random numbers get consumed, so a run with a fixed seed will now produce different, equally valid, text than the same seed did before. Nothing has ever promised that seeded output stays identical across versions, but if you depend on a recorded sample, re-record it.

Fixed

• Long-context text generation is now run-to-run deterministic on Volta. Long prompts (roughly 4,000 tokens and beyond) occasionally produced one of two different completions for the same input at temperature 0. The cause was traced link by link: the prompt-reading phase routed its attention to a kernel whose specialisation for these shapes has a known race on this hardware generation, making every run's internal state microscopically different; one close-call token decision then landed differently in about one run in ten. That phase now computes attention in a small number of deterministic chunks instead — verified byte-identical at every generation step across repeated runs — and reads the prompt about 25% faster as a bonus. Very large spatial attention (video-scale models) is outside this fix's cost envelope and keeps its previous behavior; the remaining scope is recorded in the debt registry.

Fixed

• Top-k sampling returned wrong tokens on the Triton engine. When sampling with top_k over a large vocabulary, the operation that selects the k highest-scoring tokens could return values that were not the true top k, together with token identifiers that did not match the values they were returned with. Generated text was still fluent, which is what made this hard to notice: the sampler was choosing from a corrupted candidate set rather than failing outright. Anyone who ran Triton-engine generation with top-k sampling in 0.5.1 should regard that output as unreliable and re-run it.

  Two things had to be true at once for the fault to appear, which is why it escaped earlier testing: the vocabulary had to be large enough to be processed in several passes, and the number of candidates those passes produced had to not be an exact power of two. Combinations that happened to land on a power of two were correct, so a spot check could easily pick a clean case and conclude the operation was sound.

  Greedy decoding was never affected — it does not use this operation at all. Mixture-of-experts routing was never affected for any published model: routing takes a single-pass path that skips the faulty step entirely, for any model with 256 experts or fewer, and the largest published model has 128. The PyTorch engine was never affected in any mode; it uses PyTorch's own implementation.

  The operation is now checked on every release against an independent reference, across vocabulary sizes, candidate counts and seeds, with the returned values, the returned identifiers, and the agreement between the two verified separately.

• Minutes-long stall on the first token when combining a large vocabulary with a large top_k on the Triton engine. The final selection step is generated as one fully unrolled routine whose compilation cost grows steeply with the number of candidates it has to sort — on the tested hardware, 15 seconds, 48 seconds, and then over two and a half minutes as that number doubled. top_k of 64 on a 152k-token vocabulary landed in the worst part of that curve and looked like a freeze. The candidate set is now kept small enough for the step to stay in the fast region, by scanning the vocabulary in fewer, wider passes. The selected tokens are unchanged; only tokens with exactly equal scores may now be broken in a different order. Beyond the range where this remains practical, the engine now reports a clear error instead of appearing to hang.

Added

• Unreachable-code elimination at load time: operations no output depends on are skipped instead of executed. This complements the build-side pruning shipped earlier — it also covers models built before that landed, and models installed from the registry that cannot be rebuilt locally. Operations that write in place, and random draws, are always kept: their effect is not visible in the graph's data flow.
• Exact identity elimination: operations that return their input unchanged — a slice covering a whole axis, a reshape to the shape a tensor already has — are removed from the executed sequence, and every consumer reads the original tensor. Validated byte-identical across the whole model zoo. Arithmetic that merely looks like an identity (multiplying by one, adding zero) is deliberately left in place: it is not exact in floating point, and belongs to a separate pass with a different acceptance rule.
• Horizontal matmul fusion now rewrites the graph rather than only planning: several projections reading one activation with different constant weights become one wider matrix multiply plus column reads, built from ordinary operations both engines already run.
• Two opt-in graph optimisation passes (both off by default; with their flag unset the engines execute exactly what they executed before). Common-subexpression elimination recognises an operation whose result was already computed with the same inputs and reads the first result instead of recomputing it. Horizontal matmul fusion recognises several projections that read the same activation with different constant weights — the classic query/key/value shape — so the engines can run them as one wider matrix multiply instead of several narrow ones. The first is validated byte-identical across the whole model zoo; the second ships its planner and its measurement contract while its own byte gate runs.

Fixed

• Warm serving of CPU-streamed models survives multiple requests: large models served with per-request weight reloading (CPU-streamed placement on a single GPU) crashed on the second request with a device mismatch at the first layer-norm — the weight-streaming hooks installed on the first load were not re-armed after the between-request unload. The unload path now uninstalls the streaming state so every reload re-installs it (both engines).
• Text-only prompts can drive speech generation on omni models: a text-only request with --mode audio on a build declaring the generative-speech contract now runs end-to-end (both engines); previously it was refused for lacking an image/audio input even though the requested OUTPUT was speech.

Removed

• Dead vendored kernel-utility package (kernels/utils) that could not be imported (missing internal modules) and had no consumers.

Added

• Per-instance serving daemon channel: NBX_SOCKET_PATH gives a daemon (and every client that targets it) an isolated socket, pid and log path, so independent daemons can coexist on one machine. Default behavior without the variable is unchanged.

• Frozen-plan replay on autoregressive decode (Triton mode): the per-token decode graph of KV-cached language models can now be recorded once and replayed as direct kernel launches, cutting the per-step launch overhead. The KV append writes at a device-resident position counter, MoE token routing tables are built on the GPU (removing one host sync per expert layer per step), and every replayed bucket is verified byte-equal against the normal path before being adopted — outputs are byte-identical by construction. Opt-in via NBX_TRITON_REPLAY=1 + NBX_REPLAY_KV_DECODE=1; the Qwen3-Coder-30B int4 reference row moves from 2.08 to 4.86 tok/s (×2.34) on one V100-32G with identical answers.

• CUDA-graph launch of recorded decode plans (Triton mode): a recorded decode graph can additionally be instantiated as a single CUDA graph per length bucket and launched as one call per token, removing the per-kernel submission cost on top of frozen-plan replay. Byte-identical outputs (per-bucket verification unchanged); any capture error falls back loudly to direct replay. Opt-in via NBX_REPLAY_GRAPH=1; measured on the Qwen3-Coder-30B int4 reference row: 4.86 → 5.78 tok/s (+18.8%; cumulative ×2.78 over the pre-replay baseline) on one V100-32G with identical answers.

• Warm serving keeps KV cache buffers across requests (Triton mode): the KV cache is now owned by the executor and reset in place between requests instead of being reallocated per request — removing per-request allocation churn and keeping recorded replay plans valid across a serving session.

• Device-resident scalar arithmetic on Triton mode: binary ops (add/mul/div/sub) with a 0-d GPU scalar operand now compute the scalar in-register instead of syncing it to the host — outputs stay byte-identical (host-binding arithmetic reproduced exactly), and per-step host syncs disappear from diffusion timestep chains (Ming-class image generation: 1,196 syncs per request removed), making those graphs eligible for frozen-plan replay.

• Step cache extended to autoregressive image-generation legs: the opt-in diffusion step cache now also covers the image-generation denoise loop of omni/multimodal models (same configuration channels, same skip-rate reporting, both engines).

• Failure diagnostics for Triton op errors: a failed Triton op now reports each tensor argument's device, pointer, and shape in the error message, so placement bugs adjudicate themselves from the traceback.

• Persistable autotune selections for replay runs: the kernel configurations selected during a recorded Triton-mode run are saved per hardware architecture and re-applied on later runs, so replayed graphs never re-tune — recorded-run start on the Sana row drops 56 to 46 s with byte-identical output. Stored configurations are validated against the kernel's current search space before use; anything stale simply re-tunes.

• Quantized-execution kernel family (first increment): pure-Triton int4 groupwise dequantize-GEMV kernels for LLM decode, gated byte-for-byte against their own dequantize-then-multiply reference (same math, same accumulation order — proven across the target decode shapes). Weight traffic drops 4x vs fp16 by construction; kernels are not yet wired into the engine (the quantized-artifact format decision precedes integration).

• Weight-storage encoding compatibility gate: builds that declare a packed weight-storage encoding are checked against the engine's supported set at load time — an unsupported scheme is refused with a clear message (naming the scheme and the fix) before any weight I/O, and encoded variants always ship next to their full-precision build.

• Encoded-weight execution on Triton mode (dense LLMs): builds using the packed weight-storage encoding now run end-to-end — the loader assembles per-tensor triplets under the graph keys, the transpose marker is zero-cost (the packed layout is already the transposed orientation), decode routes through the byte-gated fused dequantize-GEMV and prefill through a transient dense dequantize. First validated model produces the same greedy answer as its full-precision build at a quarter of the weight bytes.

• Encoded-weight execution for MoE models on Triton mode: the fused grouped-GEMM over experts learns in-register decompression — per-expert packed triplets ride three pointer tables and the B-tile is rebuilt each K-iteration with the tier's canonical fp32 dequantization, byte-identical to a dequantize-then-grouped-GEMM reference kernel (proven per expert shape, decode and prefill, both routing arms). First validation: the 30B-A3B coding model's encoded variant generates coherent text on a SINGLE V100-32G (17 GB of weights vs 57 GB full precision).

• Compiled-mode refusal for encoded-weight builds: running an encoded variant without --triton refuses at weight-load with a message naming the supported path and the full-precision build published next to the variant — never a silent fallback.

• Launch-tuple census diagnostic: NBX_REPLAY_TUPLE_CENSUS=1 (with replay enabled) records two steady decode steps' kernel-launch tuples and prints the varying surface per sequence — the measurement instrument behind extending replay to token-decode graphs. Observation only; never replays.

• Teacher-forced scoring mode: --set global.score_mode=1 turns a Triton-mode text request into one prefill with logits at every position and returns a per-window NLL record — the perplexity instrument for build-variant quality clauses.

• Decode address stabilization (replay groundwork, opt-in): under the decode-replay development flag, token-decode steps carve every transient allocation from a per-sequence bump slab recycled each step, and attention reads bucket-padded KV views with a persistent additive length mask (padding proven byte-neutral: identical greedy output with and without). Measured: the per-step varying launch surface collapses from 3,428 tuples to the 144 KV-append write offsets plus one sampling seed — the address-stability prerequisite for replaying token decode.

Fixed

• Step-cache override precedence: an explicit --set global.step_cache_* value of 0 now correctly overrides a process-level environment pin instead of silently falling through.

• Diffusion step cache (opt-in, per-model): models configured with a step_cache threshold reuse the previous denoising prediction on low-change steps (relative-L1 signal, bounded consecutive skips, first/last steps always computed) — Sana-1024 on Triton mode drops from 24.6 to 17.5 s/image at 29.5 dB PSNR with 8/20 steps skipped. Outputs are approximate by design; the pass is inactive unless explicitly enabled and reports its skip rate on every run.

• Frozen-plan replay for Triton mode (opt-in): NBX_TRITON_REPLAY=1 records a graph's kernel-launch sequence once and replays it as direct C-launcher calls with byte-identical outputs — diffusion steps on Triton mode run up to 2.2x faster (Sana-1024: 52.6 to 24.4 s/image, outputs byte-identical). Ineligible graph classes (KV-cache decode, streaming flows) automatically keep the normal path.

• Heterogeneous multi-GPU placement hardening: placement estimates no longer bind any symbolic dimension below its traced extent, block packing accounts per-device conversion costs and reserves per-GPU activation headroom — large omni models (Qwen3-Omni class) now run multi-GPU on mixed 16G/32G rigs where the vendor stack cannot load.

• Vertical fusion pass (opt-in): NBX_OPTIM_FUSION_VERTICAL=1 fuses matmul + activation (SiLU/GELU) pairs into single Triton kernel launches with byte-identical outputs, reducing per-step launch count on Triton mode (measured +1.2% on diffusion steps where the pattern dominates).

• Warm-serving text answers for multimodal builds: text-mode requests against multimodal-family models (Qwen3-VL class) now return the answer text in the serving response instead of a raw passthrough; the request mode is honored and, absent a mode, a produced text surface is detected automatically.

• Per-token streaming for vision-language models: the serving daemon's streamed generate now emits its per-token events for VLM builds on both execution engines — time-to-first-token is measurable over the RPC for image+text requests, same event semantics as plain LLM decoding.

• Image input over the serving daemon: generate accepts image_path; the preprocessing is the exact brick the CLI cold path uses (one source of truth), so warm and cold requests see identical model inputs.

Fixed

• Speech-mode requests can no longer be silently degraded: a mode=audio request on a speech-capable build is guaranteed to run the flow that carries the speech leg — previously a text-only prompt could be routed to the plain text path and return tokens with no audio and no error. Missing prerequisites now fail loudly.

• Warm daemon output resolution for multimodal builds: the request mode now reaches the output writer, so audio/image answers save with the right extension instead of erroring; the speaker selection is forwarded to the speech leg (it was accepted by the CLI and dropped by the daemon).

• Constant folding at load (opt-in): with NBX_OPTIM_CONST_FOLD=1, subgraphs computable from model parameters alone are executed once at weight-bind time — through the same kernels the hot path would have used, so outputs stay byte-identical — and their results become load-time constants, leaving the per-forward hot loop. Applies to both execution engines; conservatively skips symbolic shapes, in-place-aliased tensors, fused-op inputs, and streaming (zero3/lazy) placements, which refuse with a clear message. Off by default while the zoo-wide byte gate and the fold cache land.

• Token streaming over the serving RPC: generate accepts stream: true and the daemon pushes one event per decoded token on the same connection before the final response, on every execution mode. DaemonClient.generate_stream(...) yields the events — time-to-first-token is now measurable (and measured, in the benchmark harness) instead of estimated. Non-streaming requests are byte-identical to before.

• Optimization analysis engine (report-only): a new analysis module maps optimization opportunities across every installed package graph — constant-foldable subgraphs, exact algebraic identities, duplicated computations, dead code, layout candidates and kernel-fusion sites — each labeled with the gate class a future transformation must pass (byte-preserving vs drift-gated). Nothing is transformed: the sweep (python -m neurobrix.core.optim.sweep) only writes a dated findings report per model plus a synthesis table. Findings are honest to symbolic dimensions: bounds frozen at trace values are surfaced as package-coverage suspects instead of being reported as removable.

Fixed

• Non-interactive license acceptance: neurobrix import accepts --accept-license (or NBX_ACCEPT_LICENSE=1) so gated models can be installed from scripts, CI and Docker; without a terminal the CLI now explains both options and the license URL instead of failing silently, and license errors from the registry are relayed with the server's message.

The historic version: NeuroBrix reads, sees, hears, SPEAKS, DRAWS and codes. Three generative omni models — speech out, image out, conditional-flow-matching voice out — each validated on all four execution modes and published on the hub.

Added

• Conditional-flow-matching voice synthesis for multimodal builds whose speech contract declares a CFM token2wav chain, on ALL FOUR execution modes: the language model's answer conditions a codec-token autoregressive stage (additive text-embedding + normalized hidden-projection merge), then a conformer flow encoder, a guided cosine-grid flow-matching decoder and a neural vocoder — all packaged components, with per-speaker conditioning constants precomputed into the package. Both engines share the seeded draw AND initial-noise frontiers, so the codec tokens are byte-coupled across all four modes; verified end-to-end by independent speech-to-text on every mode's WAV. Adds the NBX_SPEECH_DUMP diagnostic (default-off) capturing the leg's bisection points (span ids, speech codes, mel, waveform) on both engines.
• Generative image output (--mode image) for multimodal builds that declare an image-generation contract, on ALL FOUR execution modes: the language model conditions a flow-matching diffusion stack that lives entirely inside the package (learned query tokens as a package constant, condition bridge, denoiser and VAE as traced components), with guided sampling through the internal CFG protocol and scheduler factory. Both engines draw the initial latents from ONE shared seeded gaussian frontier — bit-identical noise, so runs are byte-reproducible per engine and cross-engine differences can only come from kernel numerics. Builds without the contract refuse cleanly.
• Generative speech output (--mode audio) for multimodal builds that declare a speech contract, on ALL FOUR execution modes: speaker selection, seeded reproducible sampling (one shared draw path — each engine's audio output is byte-reproducible run to run, and the two engines stay draw-for-draw coupled until a genuinely flat sampled distribution makes the pick engine-sensitive), and WAV output at the model's declared sample rate. Builds without the speech contract refuse cleanly with an explanatory message.
• Per-draw speech sampling diagnostics (NBX_DRAW_DIAG): log the top-4 candidates, the top-2 margin and the chosen code at every speech sampling step, on both engines through the one shared draw path — the adjudication tool for cross-engine code divergences.
• Per-step decode diagnostics (NBX_DECODE_TOPK): both execution engines can log the top-4 next-token logits and the top-2 margin at every decode step — the measuring tool for adjudicating rare greedy-decoding token flips between engines.
• Reproducible speech seed from the package: speech-enabled builds now carry their default RNG seed in the package defaults; the CLI --set global.seed overrides it per request (seed 0 is a valid value). Packages without a declared seed refuse with a clear message instead of silently falling back.

Changed

• The speech leg executes every codec-embedding lookup and the per-step residual-code head projection as graph components (a single-step predictor component invoked once per residual code group), removing all raw weight access from the flow layer. Both execution engines run the same graphs by construction; the sampling frontier is unchanged. Requires a package built with the matching speech contract — older speech packages refuse cleanly at the contract check, as before.

Fixed

• Half-precision creation-op fill sentinels: full-class ops that carry a mask sentinel next to an explicitly recorded dtype no longer overflow when the hardware dtype policy narrows that dtype — the scalar now clamps to the resolved target dtype's finite range (numerically inert for attention-mask sentinels), on BOTH the compiled and the sequential execution paths through one shared rule.
• Sequential-mode resample ops traced from scale factors: upsample ops recorded with a null output size and float scale factors now recompute the size from the live input length (mirroring the compiled wrapper), and float scale factors in op argument lists are no longer integer-coerced by the shape resolver (a 1/480 downsample scale became 0, crashing the vocoder source resample).
• Pinned-noise reproducibility on the compiled path: the shared seeded RNG pin (NBX_FORCE_RAND_SEED) now also covers compiled execution — all execution modes draw stochastic vocoder noise from one shared stream, making pinned runs byte-reproducible per engine.
• ATen alias ops in Triton modes: numpy-style ATen aliases (multiply, divide, subtract, comparison aliases) recorded by vendor code now canonicalize to their base ops in one shared map — kernel dispatch, op classification and precision rules all treat an alias exactly like its canonical op.
• Text-mode answers on speech-enabled multimodal builds: builds whose language-model graph exposes an auxiliary mid-stack output (used by the generative-speech leg) produced garbled text in text-only requests — the runtime picked the auxiliary output as the final hidden states by position. Hidden-state extraction now prefers the output the build NAMES as principal (both execution engines); text answers are restored byte-identically to their pre-speech baselines.
• Ming-Lite-Omni decode is now deterministic (greedy) as the vendor intends: the packaged sampling default made every run stochastic; with the corrected package, all four execution modes produce identical answers token-for-token on text, image, and audio inputs.
• Corrupted vertical band on short-schedule video generation (Triton engine, V100): the flash-attention kernel's known instability on pre-Ampere GPUs turned out to be structurally biased, corrupting a fixed window of tokens that classifier-free guidance then amplified into a visible band. Affected attention shapes now take the deterministic attention path on V100-class hardware (hardware-profile driven): the band is gone, the same video renders byte-identically across repeated runs for the first time, and generation is faster on the affected configuration.
• Serving large-vocabulary models no longer fails at warmup with a spurious GPU out-of-memory error (Triton engine): the softmax kernel used at sampling loaded the whole vocabulary row in one register tile; at 150k-token vocabularies the kernel launch itself silently required ~2 GiB of GPU memory for register spill, and could fail on a busy device even with gigabytes reported free. Vocabulary-scale rows now use a bounded-tile streaming softmax whose launch needs no extra memory at any vocabulary size; smaller rows keep the existing kernel unchanged.
• Warm serving now actually keeps weights loaded between requests: the persistent-serving flag was never forwarded to the per-component executors, so the daemon silently unloaded the full weight set after its warmup and re-loaded it during the first request — on large multi-GPU models (57 GB MoE sharded across 4 GPUs) that mid-request reload could crash the request with a GPU illegal-access error, and in all cases defeated the purpose of warm serving. The daemon now serves warm requests without any reload, which also makes the first response dramatically faster.
• Cross-GPU tensor transfers in the fused MoE path now use the shared synchronized transfer helper: the previous local copy skipped the source-device synchronization barrier, leaving a latent read-ordering race on multi-GPU expert placements.

Changed

• 0.4.1 replaces 0.4.0 (packaging hygiene). Shipped files referred to internal build tooling by name in comments and error messages; those references now read "the build toolchain". No functional change to any execution path.
• Model-registry annotation lookup: the optional per-component flag registry is now located via the NBX_MODEL_REGISTRY environment variable or a .nbx_registry pointer file (development setups); a configured-but-missing location raises instead of being silently ignored. Deployed installs without a registry are unchanged (documented defaults apply).

Added

• Large mixture-of-experts vision-language models (Qwen3-VL-MoE class, 30B-scale): text, image and video understanding run on packages of this family, sharded across GPUs automatically. The placeholder-mask convention is now discovered from the packaged graph instead of assumed, so lineages that index hidden states with a flat mask and lineages that scatter with an expanded one both work from the same runtime code.

Fixed

• Scatter operations with mixed precision inputs: when automatic mixed precision promotes only one side, the source now follows the destination's precision instead of failing (both execution engines).
• Sorting on the Triton engine: models whose graphs sort tensors at runtime (windowed vision transformers of the Ming lineage) now run on the Triton engine — the radix-sort kernel failed to build on current Triton releases and its wrapper mis-initialized the returned order indices. Sorted values and order indices are now byte-equal to the reference, including tie stability.

Added

• Mixture-of-experts models with several routers per layer (Ming lineage): models that pick experts through a different gate per input modality now run on the fused expert path, on both execution engines. The modality blending stays visible in the model graph — the fused expert primitive consumes its result and learns nothing model-specific. Every already-supported MoE model keeps a bit-identical expert binding and execution order. Two latent defects in that shared path are fixed along the way: the binding was not reproducible across processes, and the fused operation could be scheduled before the operation that produces its input.

Fixed

• Multi-head-latent attention on the Triton engine (DeepSeek-V2 class): attention with asymmetric query/value head widths now computes correctly end-to-end, and three deep correctness fixes land with it — a protected float16 downcast (out-of-range sentinel values saturate to the finite extreme instead of overflowing to infinity, matching the PyTorch engine's behavior), concatenation of strided tensor views, and cross-device transfers of non-dense tensor views on multi-GPU placements (previously read out of bounds at pipeline boundaries). All four execution modes produce byte-identical output on the reference code-generation prompt.

Added

• Staged-splice multimodal understanding (MiniCPM-o lineage). A second vision-language contract joins the vlm flow on both execution engines: models whose vision tower feeds a perceiver projection stage and whose language model splices modality embeddings in-graph (bool-mask scatter over placeholder runs) now run text, image and audio understanding on one build — detected from the packaged graphs, never from model names. Includes adaptive-slice image preprocessing (LLaVA-UHD class), a universal fast-tokenizer runner for vendor tokenizer classes over standard serializations, a new Triton masked-scatter kernel composition, and correctness fixes with reach beyond this lineage: convolution/pooling argument broadcast for single-element geometry lists, a pooling-kernel fix for non-contiguous inputs (silent wrong values on transposed inputs), and a placement-solver reserve applied uniformly so near-capacity models degrade to multi-GPU plans at solve time instead of failing at runtime.

• Video understanding for omni multimodal models (Qwen3-Omni lineage). neurobrix run --input-video … --prompt … answers questions about a video clip with these builds — completing four-modality understanding (text, image, audio, video) on a single build, on both execution engines. Frames are sampled at the vendor's fps contract (--fps overrides), the vision tower runs the clip as a temporal patch grid, and rotary positions carry real timestamps (temporal scale from the sampling rate), so any clip length or frame rate works on one build. All four execution modes produce byte-identical answers on the reference clip.

• Audio understanding for omni multimodal models (Qwen3-Omni lineage). neurobrix run --input-audio … --prompt … now answers questions about an audio clip with these builds: the audio tower's windowed encoder runs natively on both execution engines with variable-length mel features (no fixed 30-second padding), and the SAME build still serves text-only and image prompts. Audio spans get 1-D rotary positions matching the vendor semantics; chunk counts, window paddings and feature lengths all follow the input's true duration — any clip length works on a single build. Sequential mode reproduces the vendor pipeline token-for-token on the reference clip; compiled mode is byte-identical to sequential; the Triton engine matches its own op-by-op reference byte-for-byte. Includes a fix for a latent memory-safety bug in the fused MoE kernel's final projection (out-of-bounds routing-weight reads on padded expert blocks) that could crash multi-expert models regardless of modality.

• Image understanding for DeepStack multimodal models (Qwen3-Omni lineage). neurobrix run --input-image … --prompt … now answers questions about an image with these builds: the vision tower's multi-scale DeepStack features are routed into the language model's early layers on both execution engines, and the SAME build still serves pure-text prompts (image-less requests take the KV-cached text path automatically). Sequential mode reproduces the vendor pipeline token-for-token on the reference input; compiled mode is byte-identical to sequential; the Triton engine matches its own op-by-op reference byte-for-byte. Includes a deterministic boolean-mask compaction kernel (nonzero) and boolean-mask advanced-indexing support (aten::index / aten::index_put) on the Triton engine, covering zero-size masks.

• Terminal coding agent mode. Tool-capable chat models can now work as a terminal coding agent: neurobrix chat --agent (interactive) or neurobrix run --mode agent --prompt "<task>" (one-shot). The model reads and writes files, runs shell commands, and iterates until the task completes — entirely inside a sandboxed working directory (canonical-path jail, network isolation by default, per-command timeout, turn bound). Tool-call syntax is read from the model's embedded chat template (XML and JSON dialects supported) — never hardcoded per model; models opt in via their family's supported-modes configuration. Works cold (in-process engine) and warm (serving daemon), on both execution engines. Every session records a full transcript (markdown + JSONL) with per-turn decode outcomes, tool calls, and results. New flags: --workdir, --max-turns, --bash-timeout, --allow-network, --approve-all, --yolo, --transcript-dir.

• Per-token decode progress diagnostic. NBX_DECODE_PROGRESS=<file> appends a timestamped line per decoded token (both engines) — buffer-immune pace visibility for long or CPU-offloaded decodes.

Added

• Image-generation models are now published for the multimodal category and run on ALL FOUR execution modes (first: Janus-Pro-7B). The Triton engine now supports image-autoregressive generation end-to-end (--triton and --triton-sequential): the pre-projection hidden states are captured on both Triton substrates, the guidance-conditioned decode feeds the aligned image-token embeddings through the full-context path, and classifier-free guidance routes through the unified CFG engine. Cross-engine agreement on the supported hardware: identical greedy argmax at the first step, hundreds of identical greedy tokens before floating-point near-ties diverge, and visually indistinguishable renders.

Fixed

• Mixture-of-experts models run dramatically faster in the eager (sequential) execution path. A cache-cleanup call fired after every expert layer, forcing a full GPU synchronization dozens of times per token — the cause of pathologically slow prefills on large MoE models. It was unnecessary (the compiled path never did it, and genuine memory pressure is already handled by the allocator's reclaim-and-retry). A 57 GB MoE that previously took hours now completes in minutes, with no change in memory safety or output.

• Large MoE models now run efficiently across multiple GPUs on the Triton engine. When a model's experts are spread across GPUs, the per-layer combine step previously copied results to the CPU, summed them token-by-token in Python, and copied them back — once per layer, per GPU. It is now done entirely on-device. A 57 GB mixture-of-experts model spread over four GPUs renders roughly an order of magnitude faster than the previous single-GPU offload path, with output identical to the compiled engine.

• A memory-cleanup call in the lazy component-placement strategy no longer raises on phase end (an undefined variable would have crashed it if that path ran with loaded weights).

• Running an autoregressive model on a single pinned GPU no longer fails with an "invalid device ordinal" error. Position indices for attention could be created on a GPU index that isn't visible in the current process, aborting the run; they now fall back to the active device. Only affects runs pinned to a subset of GPUs; unaffected runs are byte-for-byte unchanged.

• Mixture-of-experts language models packaged under any family now run correctly. Expert routing was only activated for models in the LLM family; a MoE backbone inside a multimodal package silently ran with just the handful of experts recorded at packaging time, producing near-random output. Routing now activates from the model's declared expert configuration, whatever the family.

• Language models with multi-plane rotary positions (M-RoPE) decode on every execution mode. The runtime feeds the position input at the rank the model's graph declares — text-only requests supply the three identical planes automatically, on both the PyTorch and Triton engines.

• Two operator-coverage fixes for newer architectures: indexing operations that take a list of index tensors no longer fail shape resolution, and index_add participates in mixed-precision type promotion (previously crashed on a half/float mismatch).

• --seed now governs Triton-engine autoregressive sampling. The run-scoped random stream was armed for diffusion runs but not for autoregressive decoding, so temperature/top-p sampling on the Triton engine was not reproducible from the seed (text and image models alike). The same seed now yields the same token stream on both Triton modes; seedless runs keep the previous behaviour.

• Guidance-free image generation (--cfg 1.0) no longer crashes. The published Janus-Pro-7B package baked the guided two-stream batch shape into an attention position operation, so requesting a single unguided stream failed on every execution mode. The republished package binds the batch dimension symbolically — guided (--cfg > 1) and unguided runs both work, and the guided output is byte-identical to the previously published package's.

• Serving a Triton model no longer breaks on the second request. A run of warm daemon requests could fail on request two: the Triton execution path bound model weights only once (at compile), so after a request's cleanup released them the next request ran against empty weight slots and crashed inside the rotary-embedding math. The Triton path now re-binds weights on every execution, matching the PyTorch engine. Two related second-request faults are fixed in the same pass: a cached rotary-constant reference into freed memory, and a completed generation whose token list wasn't read back correctly by the serving text extractor.

• A broken tool call no longer ends an agent session silently. When a model emitted a function block without its required wrapper (a rare decode slip), the agent parser treated it as a finished answer and stopped mid-task. It is now recognized as a malformed call, which triggers the existing one-shot format reminder so the model re-emits it correctly.

• Pre-tokenized requests now work on the Triton engine's serving path. Daemon chat/completion requests pass pre-tokenized ids; only the PyTorch engine consumed them — the Triton engine silently tokenized an empty prompt instead and failed deep in a RoPE kernel with a cryptic compile error (arange's end argument must be greater than the start argument). The Triton flow now mirrors the pre-tokenized input path exactly, and both engines raise a clear error on an empty prefill instead of crashing mid-graph.

• Generation now stops at the template's stop tokens in every mode. End-of-sequence token lists from the model's configuration are normalized identically in both engines (single id, list, or absent); previously a list-valued eos could be missed, letting decode run past the answer.

• Warm agent runs refuse a mismatched --model. If a serving daemon is running a different model than the one requested, the agent command now errors clearly instead of silently using the daemon's model.

• Serving no longer crashes on the second request of a session. The compiled and Triton hot loops eliminate weight-transpose operations at compile time by pre-transposing the stored weight; the pre-transposed state is now stamped in tensor metadata and re-read on every recompile, so a second request (which recompiles against the already-transposed weights) no longer computes with a stale layout — previously this crashed the language-model head with a shape mismatch.

• Hybrid CPU-offload placements now execute correctly in the compiled engine. When the placement plan keeps a large component's weights in CPU memory and streams them to the GPU block-by-block inside a mixed-placement plan, the compiled execution path previously never engaged the streaming machinery and crashed on the first weighted operation. The streaming installer is now engaged per-component from the placement plan, matching the other execution modes. First beneficiary: 30B-class mixture-of-experts models on a single 32 GB GPU.

• Large models no longer fall back to CPU execution when a GPU plan exists. Two placement-selection defects combined to route a 57 GB mixture-of-experts model to CPU on a machine with four GPUs: the viability check counted CPU-offloaded weights as if they occupied GPU memory (rejecting valid hybrid plans), and missing hardware-profile statistics could penalize the GPU-offload strategy below the CPU last-resort in the ranking. GPU-compute strategies now use placement-aware memory accounting and can never rank below CPU execution. Verified across all 44 installed models (placement plans unchanged for models that already worked; byte-identical outputs).

Added

• First vision-language model support: GLM-4.1V-9B-Thinking. NeuroBrix now runs image-understanding models end-to-end: give it an image and a question (--input-image + --prompt) and it answers in text, with the model's full reasoning trace. Images are processed at their native resolution (no fixed crop), and the answer is validated identical across all four execution modes. Supporting engine additions, all usable by future models: a pure-Triton 2D image resampling kernel (bicubic/bilinear, bit-matched to the reference), integer floor-division and dtype-aware cumulative-sum kernels, and chat templates that ship as separate files are now packaged into the model automatically.

• Groundwork for vision-language (image-understanding) models. The engine can now bind a model's symbolic dimensions from the contents of an input tensor, not only from its shape — required by modern dynamic-resolution vision models, whose image-grid geometry travels as data. Both execution engines implement it identically, and existing models are provably unaffected (byte-identical regression outputs). The image preprocessing module gains the matching native_patch_grid type (dynamic-resolution patch grids, bit-exact with the vendor preprocessing), used by the upcoming GLM-4.1V integration.

Changed

• --input-image preprocessing is now a universal, reusable module. The image preprocessing behind --input-image (image-to-video conditioning and CLIP-encoder views) moved from ad-hoc code inside the CLI into a dedicated ImageInputProcessor with a dependency-light numpy core, the same pattern already used for audio input. Outputs are bit-identical to before — validated by unit tests and a byte-identical video anti-regression battery across the image-to-video model zoo — and the new module is the foundation for upcoming vision-language (image-understanding) model support, where image preprocessing types like dynamic-resolution patch grids will plug in alongside the existing ones.

• Faster repeated requests on the serving daemon. The server was re-reading and re-parsing every model component's on-disk layout files on every single request in order to set up spatial tiling — 100ms to several seconds of pure per-request overhead, even though those files never change while the daemon is running. That setup now happens once and is reused across requests, so a warm server answers repeat requests without the per-request file-parsing cost. The first request and one-shot CLI runs are unchanged.

• Lower GPU memory use in the Triton engine when a model is close to running out of memory. The Triton execution engine now reclaims the memory of already-finished intermediate results sooner when the GPU is near its capacity, keeping the memory high-water mark down so memory-heavy models are less likely to hit an out-of-memory error. Crucially this is pressure-sensitive: when the GPU has plenty of free memory the engine keeps its fast batched-release behavior unchanged, so models that already fit comfortably see no change in speed. The point at which it starts reclaiming early is read from the hardware profile (not hardcoded), so it adapts per GPU. Two related reclamation gaps were also closed: a rarely-used no-cache text-decode path that kept every step's intermediates alive for the whole pass, and an attention mask cache that grew without bound across many sequence lengths in a long-running server (now released at model-unload boundaries).

Fixed

• 4K image generation now completes in the Triton engine (Sana 4K). The final image-decode step used to run out of GPU memory on a 32 GB card. Two fixes: (1) a large intermediate result in the decoder's residual pathway is now written in place into a buffer that was about to be released, instead of allocating a fresh 8 GB block the memory heap could no longer serve contiguously; (2) as a general safety net, when a GPU allocation fails the engine now reclaims memory held by already-finished intermediate results and retries once before reporting out-of-memory — this covers any model running close to the memory limit, not just Sana. A 4096×4096 image that previously crashed at the decode step now renders end to end (~7.5 minutes on one V100).

• Corrected GPU memory planning for image models that use a high-compression autoencoder (such as Sana). When estimating how much GPU memory each part of a model needs, the planner assumed a fixed 8x spatial compression from the autoencoder. Models whose autoencoder compresses more aggressively (Sana's is 32x) had the memory need of their main network over-estimated by up to 16x, which could push that network onto slower CPU-offloaded placement — and, in the Triton engine, prevent the run from starting at all. The planner now reads the true compression factor from the model itself, restoring single-GPU placement for these models. (The largest 4K Sana configuration still stops at a separate, already-tracked memory limit during the final image-decode step.)

• Language models now decode correctly in the op-by-op reference mode beyond the trace length. In the step-by-step reference execution mode, autoregressive text models could crash once generation ran past the length used when the model was prepared, because the rotary position table was being narrowed to the current step's length while the decode step still asked for an absolute position further along. The reference mode now keeps the full position table available, matching the primary execution mode, so decoding continues correctly to arbitrary lengths. Verified end to end on TinyLlama-1.1B-Chat (identical greedy output across all execution modes) with no change to image generation.

• Internal consolidation / correctness hardening (both engines). Three duplicated internal patterns, each carrying a latent bug class, now route through a single brick per engine. (1) Device-string parsing in the Triton flow handlers: six divergent copies disagreed on multi-GPU placement strings — one crashed outright, three silently selected GPU 0, and the fast path of the rest picked the LAST device of a comma list instead of the primary; one shared parser now resolves the first named device in every form. (2) Flow-level compute-dtype resolution now reads the resolved allocation plan instead of the model's declared dtype — the plan is what actually executes (a declared dtype the hardware cannot run natively is resolved to a supported one), and one copy had been silently falling back to defaults because it walked a plan attribute that never existed. (3) Stage-boundary memory cleanup in the flow handlers now follows the memory manager's sync-before-free discipline on both engines: the device is synchronized before garbage collection drops the last references, so an in-flight kernel can never read freed memory (the illegal-address class), and the cache/pool flush happens after the collect.

• Broadcast bias in batched matrix-multiply-add (Triton engine). A bias tensor with a size-1 row or column dimension (e.g. [batch, 1, N]) was read with its memory stride instead of broadcasting, picking up values past the intended bias row/column. Every size-1 bias dimension now broadcasts correctly.

Performance

• Batched matrix-multiply launches (Triton engine). bmm now issues a single batched kernel launch for the whole batch instead of a Python loop of per-batch (and, on the decode path, per-row) launches. LLM decode attention, which runs batch = heads matrix multiplies per generated token, drops from thousands of kernel launches per token to two; measured 47-74× faster on decode-shaped batches and 4.6× on prefill-shaped batches (V100). Results are value-identical to the previous path on the batched-matmul route (bit-equal under a pinned kernel config) and within float reassociation tolerance (max relative difference ~1e-4) on the former per-row decode route.

Silent-failure hardening (zero-fallback sweep)

• A failed tiled residual-chain merge now stops the run. Previously, when a band-streamed residual-chain merge failed inside the op-level tiling engine — or its chain weights could not be resolved — the un-merged input tensor was silently substituted and the run continued to a wrong final output. Both failure paths now raise with the chain and op named in the message.
• Always-on NaN/Inf gate on the diffusion loop state. Both execution branches now verify the latent state is finite once per denoising step (a single isfinite scan at the step boundary — no per-op overhead) and stop with the component, step, and timestep named, instead of a debug-only print that let corrupt runs continue. The per-op output guard keeps its NaN and positive-infinity contract (negative infinity is a legitimate value — attention masks, log-underflow — so it is left to the step-boundary state gate, where it unambiguously signals corruption).
• fp16-overflow protection can no longer be silently disabled. The per-component flag registry raises when its file exists but is unreadable or malformed (an absent registry still legitimately resolves to defaults), and the fp32-pin resolution in the placement solver propagates read failures instead of returning an empty protection set.
• Text-to-speech diffusion parameters are now required. The next-token-diffusion flow (both execution branches) refuses to run when its semantic keys (guidance scale, latent scale/bias, scheduler shape) are missing from the model package instead of inventing defaults, matching the behavior of the other audio flows.
• Classifier-free guidance input batching no longer swallows resolver errors. A genuinely absent optional conditioning port is still skipped; any other resolution failure now propagates instead of silently feeding batch-1 conditioning to a batch-2 forward.

NeuroBrix 0.3.0 — The Video Release

The video generation family is complete: ten video models — text-to-video and image-to-video — validated in all four execution modes (PyTorch sequential/compiled, Triton sequential/compiled), plus Allegro-TI2V at native 720×1280×88-frame resolution. All eleven are published on the NeuroBrix Hub and install with one command:

neurobrix import Wan-AI/Wan2.1-T2V-1.3B
neurobrix run --model Wan2.1-T2V-1.3B --prompt "a red fox walking in a snowy forest" --num-frames 49

Highlights

• Video family (11 hub models): Wan2.1-T2V-1.3B, CogVideoX-2b, SANA-Video-2B-720p, CogVideoX-5b-I2V, Mochi-1-preview, Wan2.1-I2V-14B-480P, Wan2.2-I2V-A14B (dual-denoiser, 28B), Wan2.1-VACE-1.3B, Open-Sora-v2, Allegro, Allegro-TI2V. Image-to-video conditioning via --input-image, classifier-free guidance, deterministic seeded sampling.
• 5D spatio-temporal VAE tiling — native-resolution and long-clip video encode AND decode on a single 32 GB GPU (a 156 GiB encode plans as 24 tiles of ~10.6 GiB), with seamless blending and byte-identical untiled paths.
• Multi-GPU component placement for 14B+ models, including dual-expert lifecycle with one-way expert eviction (Wan2.2's 2×14B denoisers).
• New Triton kernels & engine hardening — max_pool3d, conv3d temporal chunk-streaming under memory pressure, complex-dtype rotary support, bool-op broadcasting, per-branch numerical parity for guided models (0.9999+ correlation vs the PyTorch reference at native scale).
• Scheduler parity — Euler-family float-timestep spacing bit-identical to the reference pipelines; runtime-pure UniPC including flow matching; CogVideoX DDIM variant; inverted-sigma flow support.
• Accuracy & memory fixes across all four execution modes — see the CHANGELOG for the full list.

The audio (11 models), LLM, image, multimodal, and upscaler families from 0.2.1 are unchanged and re-verified.

Install: pip install neurobrix==0.3.0 · Docs: https://neurobrix.es/docs · Hub: https://neurobrix.es

NeuroBrix 0.2.1

First publicly usable release of the universal inference runtime — not final, just usable by people. One engine, any model, any modality, zero model-specific code.

Highlights

• 4-mode execution matrix green on orpheus-snac, hat-l-x4, Flex.1-alpha, deepseek-moe-16b — each validated in a clean-room venv (none of the 14 vendor libraries installed), greedy and cross-mode-consistent. The four modes are two independent branches × two variants: PyTorch (--sequential op-by-op oracle, --compiled fused torch+cuDNN/cuBLAS) and Triton (--triton-sequential, --triton — NeuroBrix @triton.jit kernels, zero torch).
• Audio, LLM, image, and upscaler families run end-to-end.
• New aten::im2col Triton kernel (HAT OCAB) → hat-l-x4 now 4/4.
• Flex/FLUX CLIP-pooler advanced-index fix (_meta_index) → Flex.1-alpha triton coherent.
• R30 triton-sequential cross-device (pipeline-parallel) → deepseek-moe-16b 4/4.
• Zero Outsider: SNAC traced into orpheus's .nbx (no runtime HF download); internal tokenizer/mel/g2p/filterbank runners; engine runs from the .nbx with only torch (compiled) / triton+NBXTensor (triton).
• Engine docs aligned to the real CLI (9 families, 4 modes, upscalers).

Documented debt — Qwen3-30B-A3B --sequential

Validated by construction, not executed. The original frozen-seq-dim view crash is eliminated by re-tracing (graph carries {mul,s0,s1}, 0 frozen views), and correctness is established via compiled mode (same graph, "Paris.") plus the SymbolicShapeResolver shared by both paths. The op-by-op sequential oracle was not run to completion on 30B (≈115k ops × per-op dispatch is impractically slow). This is honest debt — not counted as a 4/4 pass. Qwen3 ships/validated in compiled. The other four matrix models are 4/4.

Install

pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install neurobrix

Full details: CHANGELOG.md. Model licenses are each vendor's own (engine is Apache-2.0; it carries no model-license machinery).

NeuroBrix v0.2.0 — P-PRISM-NEVER-REFUSE v2 closed (2026-05-15)

162 commits since v0.1.6 (2026-04-20). Minor version bump per SemVer: features added are rétrocompatibles, no breaking API changes. Major acquis closes the P-PRISM-NEVER-REFUSE v2 mandate with 14/16 hardware × mode cells validated end-to-end on Sana 1.6B 4Kpx diffusion (4096×4096 output).

Highlights

• Doctrine R34 model-agnostic ratified and audited clean — zero active hardcode-by-model-name violations across runtime, kernels, strategies, dispatchers.
• Doctrine R35 "Prism never refuses" implemented and empirically validated — cascade strategy covers every legitimate hardware × model combination down to cpu_execution fallback.
• 14/16 cells matrice hardware × mode validated end-to-end on Sana 4Kpx (single 32 GiB, single 16 GiB, 2× 16 GiB; compiled, sequential, triton, triton_sequential). The 2 ⏸ cells remain on upstream-blocked CPU triton modes (triton-cpu PyPI wheel pending).
• 18× speedup on 32g triton mode: 1443 s → 79.81 s wall on Sana 4Kpx via DC-AE residual chain band-streaming (each of 9 VAE chains streams band-by-band, peak transient drops from 6.7 GiB to 1.6 GiB per chain).

Hardware coverage

Wall times measured on Sana 1.6B 4Kpx single-step at 4096×4096 spatial output. CPU triton cells wait on upstream triton-cpu (meta-pytorch project) shipping PyPI wheels.

Architecture

Hybrid CPU+GPU per-component placement

core/strategies/lazy_sequential.py + core/runtime/executor.py now correctly route mixed-device plans through strategy.execute_component so a CPU component consuming a GPU producer no longer stalls in implicit transfer or raises device-mismatch. Unlocked 16g compiled + sequential on models whose VAE alone exceeds the GPU budget.

Intra-component multi-device cascade

The single_gpu strategy correctly picks cuda:0 on 2× 16 GiB hardware when a component fits there post-S5 chain tiling. No solver extension required — the cascade naturally handles the case.

DC-AE residual chain tiling — band-streamed (R30 dual-branch)

• Compiled mode: band_streamed_chain_torch materialises the residual chain band-by-band in PyTorch ATen, writes back to T_base in place.
• Triton mode: band_streamed_chain_nbx (143 lines, R33-pure) mirrors the algorithm using NBX wrappers + NBXTensor methods only.
• Mode-aware dispatcher in core/module/tiling_engine.py selects the variant per graph_executor.mode.
• Chain wrapper is default-ON on triton modes; NBX_TRITON_CHAIN_WRAPPER=0 reverts to the c9d2581 skip path as emergency rollback.

Edge-padding + halo-offset wrapper math fix (_tiled_conv2d_spatial_*)

Two coupled bugs in the 4 tiled-conv wrappers (_tiled_conv2d_spatial_torch, _tiled_conv2d_spatial_nbx, _fused_upsample_conv2d_torch, _fused_upsample_conv2d_nbx):

• Edge-band double-padding (max(0, -in_read_start) already provides pad_h; the extra + pad_h if is_top_band term over-padded by pad_h rows).
• Internal-frontier halo offset on the conv_band read side (output slice was offset by halo_top rows on every internal band). Validated bit-exact via scripts/microtest_tiled_conv2d_small_scale.py sweeping (kh ∈ {1,3,5}, pad_h ∈ {0,1,2}, tile_factor ∈ {1,2,4,8}) at 1024² and 2048² — cos=1.0000 max_abs=0.0000 universally.

NBXTensor.getitem negative-slice fix — ROOT FIX for chain wrapper

NBXTensor.__getitem__ slice path used k.start or 0 which silently left negative starts untouched (-2 or 0 evaluates to -2, not 0). For t[-N:], the resulting narrow(dim, -N, length) produced an _offset = parent_offset - N*stride → data_ptr() pointed BEFORE the cudaMalloc'd block → Triton's cuPointerGetAttribute rejected the pointer with "Pointer argument (at 0) cannot be accessed from Triton (cpu tensor?)".

The Sana 4Kpx VAE chain wrapper's halo_carry[:, :, -top_size:, :] triggered this at the 3rd iteration of every 4096² chain. Fix restores the universal Python/torch slice contract:

start = k.start if k.start is not None else 0
stop = k.stop if k.stop is not None else shape[dim]
if start < 0: start = max(0, shape[dim] + start)
if stop < 0:  stop  = shape[dim] + stop

8-line fix, 4 cells unlocked. Latent bug across the codebase, now closed.

Triton-CPU integration stage 1

triton-cpu (meta-pytorch project) is not on PyPI today and requires build-from-source. Stage 1 ships the install gate (raises clean ImportError with actionable message when missing), coverage docs, and the activation flow. CPU triton + CPU triton-sequential cells remain ⏸ until upstream ships PyPI wheels — automatic unblock then.

Doctrine ratified

• R34 model-agnostic strict: structural pattern detection, no hardcode-by-model-name in runtime / kernels / strategies / dispatchers.
• R35 Prism never refuses: cascade fully implemented to cpu_execution fallback; every legitimate plan finds a strategy.
• R30 dualité runtime: the 4 execution modes (compiled, sequential, triton, triton_sequential) cover the 14 cells symmetrically; band_streamed_chain_* mirror restored chain coverage on triton path.
• R33 zero torch in triton/: preserved; chain wrapper NBX variant uses only NBX wrappers + NBXTensor methods, no torch import on the execute path.
• R29 inspectable artefacts: every cell ✓ has a coherent red apple PNG in docs/verdicts/p_triton_chain_cpu_pointer/ and docs/verdicts/p_nbx_tiled_conv2d_small_scale/.

Known limitations (transparency)

• 2/16 cells ⏸ : CPU pure × triton and CPU pure × triton_sequential blocked on absence of triton-cpu PyPI wheel (meta-pytorch/triton-cpu, build-from-source only at v0.2.0 release time). Backlog item P-TRITON-CPU-UPSTREAM-WHEEL-FOLLOWUP unblocks automatically when upstream ships wheels.
• Op-level cross-device split (Gap B) out of scope of v2 mandate: for models where a single op exceeds per-device VRAM. Backlog item P-OP-LEVEL-CROSS-DEVICE-SPLIT opens when a concrete model demands it (no Sana 4Kpx need it now that VAE fits 16 GiB post-S5 tiling).

Validation artefacts

The mandate v2 verdict and per-cell R29 artefacts are committed:

• docs/verdicts/p_prism_never_refuse_v2_closed.md — full verdict, sub-chantier history, commit hashes, anti-régression table.
• docs/verdicts/p_triton_chain_cpu_pointer/ — 6 coherent red apple PNGs (32g compiled / 32g triton / 16g compiled / 16g triton / 16g triton-seq / 2× 16g triton / 2× 16g triton-seq).
• docs/verdicts/p_nbx_tiled_conv2d_small_scale/ — microtest logs + 16g compiled anti-regression PNG + 32g triton baseline PNG.

Key commits

Sub-chantiers and bugfixes that shaped this release:

Tag

• p-prism-never-refuse-v2-closed posted on commit 77571b7 (mandate closure), pushed to origin and gitlab.
• v0.2.0 semver release tag posted on this commit, pushed to both remotes.

Upgrading from v0.1.x

No breaking API changes. The chain wrapper is default-ON on triton modes (set NBX_TRITON_CHAIN_WRAPPER=0 to fall back to the v0.1.x behaviour). Other internal changes are transparent to call sites.

Fifth release since 0.1.5 on April 15. Four substantive commits hardening the multi-GPU Triton path and closing the long-standing Item 3 perf/VRAM trade-off, plus a regression-harness overhaul.

Highlights

• Multi-GPU Triton hardened — 163 of 211 kernel-launch sites were latently broken. kernels/wrappers.py's documented _set_device contract was honoured at only 48 of 211 production launch sites. On single-GPU this was invisible (active device always matched the tensor), but any multi-GPU Prism plan with a component on a non-active device crashed argmax, bmm, addmm, softmax, rms_norm, and most elementwise wrappers. Fix promotes _set_device to a single source of truth in nbx_tensor.py and inserts the guard at every red site. 211 / 211 green. Phase 2 multi-GPU strategy matrix now 8 / 8 cells green.

• Zero3 block-wise ratchet pipelining (native + triton). Qwen3-30B-A3B-Thinking-2507 on a single 16 GB V100 now runs under bounded VRAM (two blocks of weights + KV + activations). Overlaps H2D(N+1) with compute(N) on a dedicated transfer stream. Prefill dramatically faster than the per-op slow path. Polymorphic over torch.Tensor + CompiledSequence and NBXTensor + TritonSequence.

• Item 3 closed — in-kernel fp16 → fp32 tile promotion for mm / bmm / addmm on pre-Ampere (Path A'). New PROMOTE_B: tl.constexpr flag on matmul_kernel / addmm_kernel: the b tile is cast to a.dtype after tl.load, before tl.dot. Bit-exact. On TinyLlama-1.1B v100-16g --triton: peak VRAM 4.74 → 2.54 GB (-46 %), decode 9.94 → 5.04 s on 8 tokens (-49 %), character-identical output across 30-token greedy decode. On Qwen3-30B zero3: -64 MB, -5 %.

• NBX_FORCE_STRATEGY env var for deterministic Prism strategy selection. Short-circuits the score cascade. Unknown strategy → RuntimeError listing the 9 valid values. Valid but unavailable for the device count → RuntimeError. Fits but under-performs the auto-selected one → still used (zero silent fallback). Enables per-strategy matrix regression testing.

• Regression harness tracked in git + 4 ::native audio fails resolved. Prior diagnoses had attributed the fails to "tokenizers env-drift" without sourced stderr; this release captured actual stderr and rooted 3 of them in interpreter selection (venv vs system Python user-site) plus one genuine aten::cudnn_batch_norm regression on Kokoro-82M (marked xfail with follow-up scoped in docs/follow-ups/). New pytest_configure hook auto-detects $VIRTUAL_ENV. Harness goes from 4 failed / 11 passed / 11 xfailed / 14 skipped to 0 failed / 14 passed / 12 xfailed / 14 skipped.

Full changelog

See CHANGELOG.md §0.1.6.

Install

pip install neurobrix==0.1.6

Wheel sha256 7fca8534dd5c87d6dc455ad206618cda95f165460e74e8b6a944e8745a97e410, sdist sha256 be3258ebd8eae326d24059054ad59de2fb352df5e143bbf9f7bff558224fc673.

No release notes were published for this version.

No release notes were published for this version.

No release notes were published for this version.

No release notes were published for this version.

No release notes were published for this version.

No release notes were published for this version.