NeuroBrix Engine

The inference runtime: run AI models for text, images, audio and video on your own hardware.

version 0.6.5

pip install neurobrix

Linux, Windows and macOS. Validated on NVIDIA and Apple GPUs; AMD and Intel GPUs are targets. An integrated Intel GPU is described in the hardware profile, not driven.

Licence: Apache 2.0

Download and versions Release notes GitHub GitLab PyPI Documentation

What it is

The engine imports a model from the hub in the .nbx format, keeps it loaded in a local daemon, and runs it from the command line or through an OpenAI-compatible API. It needs no PyTorch: models run on the Triton engine, on the nbxtensor library.

Example

neurobrix import TinyLlama/TinyLlama-1.1B-Chat-v1.0 --accept-license
neurobrix serve --model TinyLlama/TinyLlama-1.1B-Chat-v1.0
neurobrix chat
neurobrix stop

Import a small language model, serve it, chat with it, stop the daemon.

Go further

The models the engine runs, by family and by card size.
The benchmarks measured by the engine team.
Install options, platforms and what is new.