NBXGrammar

What it is

NBXGrammar compiles a JSON Schema, or a description of a tool-call format, into a matcher that gives at every decoding step the mask of the tokens a model may write next. It is the engine behind response_format: json_schema and forced tool_choice in the NeuroBrix OpenAI-compatible server.

Written in Rust with the standard library only, loaded from Python with ctypes: no binding library.

Example

import json, numpy as np, nbxgrammar as G

grammar = G.Grammar.json_schema(json.dumps({"type": "object", "properties": {"city": {"type": "string"}},
                                            "required": ["city"]}))
vocab = G.Vocab(token_bytes, eos_ids)        # each token id's bytes (None for a special token), the end-of-sequence ids
matcher = G.Matcher(grammar)
mask = matcher.mask(vocab)                    # int32 words, ceil(vocab / 32): bit v % 32 of word v // 32 = token v allowed
matcher.advance(vocab, token_id)              # the token the sampler chose

From the README.

What it enforces

• JSON Schema, a subset: objects (properties, required; no other property is generated), arrays (items, minItems, maxItems), strings, numbers, integers, booleans, null, enum, const, anyOf whose alternatives start differently, $ref into $defs. Any other constraining keyword is refused by name, never ignored; title, description, default and examples are read as annotations.
• Descriptions:literals, schemas, sequences, choices by a literal prefix, repetitions with a separator, free text up to a terminator, and exact token ids: what a chat template's tool-call format is made of.
• Whitespace: at most one space where JSON allows whitespace inside a value, never a newline, and none around a JSON root.

Measured on Qwen3's vocabulary (151 936 tokens, Apple M4 Pro): a mask in about 15 µs median inside JSON, 73 µs in free text.

Latest version