Ternair loads Microsoft's BitNet b1.58 checkpoints — including the real 2B-4T — and serves them behind an OpenAI-compatible API. Pure Python, no C++ toolchain, one command to serve.
# install
pip install git+https://github.com/Simonc44/Ternair
# serve a real BitNet model
ternair serve --model microsoft/bitnet-b1.58-2B-4T --port 8080
# query it
curl localhost:8080/v1/chat/completions \
-d '{"messages":[{"role":"user","content":"Hello!"}]}'A base-3 codec packs five ternary weights into a single byte — 1.6 bits per value, 20% denser than standard 2-bit packing.
No build step, no CUDA toolkit — convert once, then serve from anywhere.
pip install git+https://github.com/Simonc44/Ternair
Pure Python. No C++ compiler, no CUDA toolkit — just pip.
ternair import-bitnet --source ./bitnet-2b4t --output ./ternair-2b4t
Bit-exact conversion of the trained weights into a 4× smaller file.
ternair serve --model ./ternair-2b4t --port 8080
An OpenAI-compatible API, ready for your existing client code.
Same trained weights, same architecture — a lighter file and a server built in.
A small, dependency-light stack — from quantization to production metrics.
{−1, 0, +1} weights with per-row absmean quantization — the same recipe as BitNet b1.58.
Base-3 codec: five trits per byte, 1.6 bits per value — 20% smaller than standard 2-bit packing.
One prefill pass, one token per decode step, correct RoPE position offsets, streaming generation.
/v1/completions, /chat/completions, /batch, /metrics — zero external services.
PyTorch, NumPy, Triton GPU, and a native C++ CPU kernel — with automatic fallback per device.
Requests, tokens per second, latency percentiles — ready for production monitoring.
Try it with a tiny demo model, or pull the real 2B-4T checkpoint straight from Hugging Face.
# install Ternair
pip install git+https://github.com/Simonc44/Ternair
# quick sanity check with a tiny model
ternair demo
# download and serve the real 2B-4T model
hf download microsoft/bitnet-b1.58-2B-4T --local-dir ./bitnet-2b4t
ternair import-bitnet --source ./bitnet-2b4t --output ./ternair-2b4t --storage packed
ternair serve --model ./ternair-2b4t --port 8080Apache 2.0. Issues and pull requests are welcome on GitHub.