This model requires you to accept the Apache 2.0 license terms before download.
Description
TinyLlama 1.1B chat — greedy decode contract. Published under its Hugging Face repository name (TinyLlama/TinyLlama-1.1B-Chat-v1.0); the earlier record TinyLlama/TinyLlama-1.1B-Chat carried an invented name and is hidden in favour of this one.
Tags
At a glance
| Kind | language model |
|---|---|
| Precision | bfloat16 |
| Container size | 2.1 GiB |
| Weights inside | 2.0 GiB |
| Components | 2 |
| Licence | Apache 2.0 |
| Origin | TinyLlama/TinyLlama-1.1B-Chat-v1.0 @ fe8a4ea1ffed |
What is inside the container
A .nbx container packs every part of TinyLlama-1.1B-Chat-v1.0 — 2 components with weights, graphs and runtime settings — so one file is all the runtime needs.
| Component | Blocks | Layers | Weights |
|---|---|---|---|
| model | 0 | 22 | 1.9 GiB |
| lm_head | 0 | — | 125 MiB |
Default settings
The values the container ships with. They apply when you do not pass your own.
- Max length
- 2048
- Temperature
- 0
- Top-p
- 1
- Top-k
- 0
Memory needed
The weights of TinyLlama-1.1B-Chat-v1.0 take 2.0 GiB in bfloat16. That is the floor for the memory the runtime has to place, spread over the devices it finds; activations come on top and grow with the size and length you ask for. A measured peak per device class is not published for this model yet.
Verify what you downloaded
Every download carries its SHA-256, and the file endpoint sends it in the X-NBX-SHA256 header. The container is 2.1 GiB (2,208,408,007 bytes).
$ sha256sum TinyLlama-1.1B-Chat-v1.0.nbx
312f09f686c306de6e1dd17e53e2dbb816eda68f8360f5226eb2cd457ebd2102
Questions
What is TinyLlama-1.1B-Chat-v1.0?
A language model by TinyLlama, distributed as a bfloat16 .nbx container and licensed Apache 2.0.
How big is the download?
2.1 GiB, of which 2.0 GiB are weights.
How do I run it?
Install the runtime from the download page, then use the commands under “CLI Usage” below: neurobrix import TinyLlama/TinyLlama-1.1B-Chat-v1.0, then neurobrix serve --model TinyLlama-1.1B-Chat-v1.0.
CLI Usage
$ neurobrix import TinyLlama/TinyLlama-1.1B-Chat-v1.0
$ neurobrix serve --model TinyLlama-1.1B-Chat-v1.0
$ neurobrix run --prompt "your prompt here"
$ neurobrix stop