Colibri is a free, open-source C engine that runs frontier-scale Mixture-of-Experts AI models, from a 7-billion-parameter OLMoE up to a 2.8-trillion-parameter Kimi K3, on a machine you already own, by treating your disk as an extension of RAM and streaming only the experts a token actually needs. It jumped roughly 960 stars in a single day on GitHub trending today, on top of nearly 30,000 total, right as its newest release, v1.11.0, shipped a bundled seven-engine Windows build.

  • Colibri needs no GPU: it runs entirely on CPU by streaming a model's expert layers from an ordinary NVMe drive instead of loading the whole thing into RAM or VRAM.
  • The engine ships as a single small C program plus a Python launcher; prebuilt binaries exist for Linux, macOS (Apple Silicon) and Windows, so no compiler is required.
  • Nine open model families run today, from a 7B OLMoE model that fits on a normal laptop to a 2.8T Kimi K3 that needs well over a terabyte of disk.
  • It is Apache 2.0 and free; the catch is that most of colibri's headline models still need hundreds of gigabytes of disk, so this tutorial sets up the small OLMoE model as your first real, working example.

The exact steps, start to finish

  1. Step 1. Check your machine is ready. 
    # confirm Python 3 and pip are installed
python3 --version
pip3 --version
    You need Python 3 for the launcher and the one-time model conversion; the engine itself is pure C and needs nothing else at runtime. No GPU and no compiler are required for this path.
  2. Step 2. Download the prebuilt engine for your OS. Grab the matching archive from the Releases page (v1.11.0 at the time of writing) and unpack it. 
    # Linux
mkdir colibri && tar xzf colibri-v1.11.0-linux-x86_64.tar.gz -C colibri && cd colibri
    macOS (Apple Silicon) uses the same two commands with the colibri-v1.11.0-macos-arm64.tar.gz archive instead. On Windows, right-click colibri-v1.11.0-windows-x86_64.zip, choose Extract All, then open the extracted folder in a terminal.
  3. Step 3. Confirm the engine is ready. 
    # Linux / macOS
python3 coli info

# Windows, from the extracted folder
coli.cmd info
    This should print an engine-ready confirmation. On Linux, if it complains about a missing shared library, install the OpenMP runtime with sudo apt install -y libgomp1 and try again.
  4. Step 4. Install the Python packages the model converter needs. 
    # same command on Windows, macOS and Linux
pip install numpy torch safetensors huggingface_hub
    These are only used once, to pull and convert the model. The engine that actually runs it afterward has zero Python dependencies at inference time.
  5. Step 5. Get the OLMoE converter script. It ships in the git repository rather than the release archive, so pull the one file you need directly: 
    curl -O https://raw.githubusercontent.com/JustVugg/colibri/HEAD/c/tools/convert_olmoe_merged.py
    No account or API key needed for this step: OLMoE is a fully public, ungated model on Hugging Face.
  6. Step 6. Download and convert your first model. This streams AllenAI's OLMoE checkpoint from Hugging Face one shard at a time, converts each shard to colibri's int8 format, and deletes the source as it goes, so it never needs more than one extra shard of disk space: 
    python convert_olmoe_merged.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ./olmoe_merged
    Expect roughly 7 GB to land in ./olmoe_merged when it finishes. The command is resumable: if it stops partway through, rerun the exact same line and it picks up where it left off.
  7. Step 7. Run a readiness check. 
    # Linux / macOS
COLI_MODEL=./olmoe_merged ./coli doctor

# Windows
coli.cmd doctor --model olmoe_merged
    doctor is read-only: it tells you exactly what is missing, whether that's files, permissions, or memory, before you try to load anything for real.
  8. Step 8. Chat with your first model. 
    # Linux / macOS
COLI_MODEL=./olmoe_merged ./coli chat

# Windows
coli.cmd chat --model olmoe_merged
    The dense layers load into RAM in a few seconds and you get a terminal prompt talking to a real 7-billion-parameter model running entirely on your own machine. Swap web in for chat to drive the same model from a browser dashboard with live token metrics instead of a plain terminal.
How colibri fits a 744B model into 25 GB of RAM A normal local LLM tool must load its entire model into RAM or VRAM before it will run. Colibri instead keeps a small resident core in RAM, around ten gigabytes, while its tens of thousands of routed experts stay on disk and stream up on demand, so total model size stops being limited by how much RAM or VRAM you own. A NORMAL LOCAL LLM Entire model must fit in RAM or VRAM before it will even load COLIBRI'S TIERED MEMORY VRAM (optional) hottest experts RAM, ~10 GB dense layers + cache Disk (NVMe) 19,456 experts, 370 GB Measured routing heat pulls the experts a token needs up the chain, one layer ahead Placement only ever changes speed. The model's answers stay identical either way. genztech.blog
Fig 1 A normal local LLM tool needs the whole model in RAM or VRAM at once. Colibri instead keeps a small resident core and streams the rest from disk, layer by layer, as the router asks for it.

What is colibri and why is it trending?

Colibri started as a one-person project on a 12-core laptop with 25 GB of RAM in July 2026, built around a simple bet: a 744-billion-parameter Mixture-of-Experts model only activates around 40 billion parameters per token, and only about 11 GB of those weights actually change from token to token. Instead of demanding the whole model sit in fast memory, colibri treats parameters as data to be staged exactly when the router proves it needs them, the same way a compiler's JIT only compiles the code paths that actually run. Two months later the project has nine working model families and a community benchmark log spanning laptops to six-GPU workstations. Yesterday's v1.11.0 release, which bundles all seven shippable engine binaries into one Windows archive, plus renewed interest in running frontier-class open weights without renting a data center, is what pushed it onto today's trending page.

RelatedUnsloth Setup: Run and Train LLMs on Your Own Machine

How do you install colibri on Windows, macOS, and Linux?

The path above uses colibri's recommended shortcut: a prebuilt archive for Linux, macOS Apple Silicon, or Windows, so nothing needs to compile. The only platform without a prebuilt shortcut is ARM64 Linux (AWS Graviton, Ampere, Raspberry Pi), where you build from source instead:

# ARM64 Linux only, or if you want a build tuned for your exact CPU
sudo apt install -y build-essential git python3
git clone https://github.com/JustVugg/colibri.git
cd colibri/c
./setup.sh

setup.sh checks your compiler and OpenMP install, builds the engine, and self-tests, printing something close to engine self-test: 32/32. On macOS a source build needs Xcode's command line tools plus brew install libomp; Windows has an MSYS2 UCRT64 path too, but the prebuilt zip is simpler and what this tutorial uses.

Which model should you actually run first?

OLMoE is the right starting point precisely because it is the smallest thing colibri supports: about 7 GB of disk, 8 GB of RAM, no GPU. That is what makes it possible to go from a bare machine to a working chat session in the time it takes to read this article. It is also not the point of the project: colibri's real pitch is a 372 GB GLM-5.2 container needing 16 GB of RAM, or a 1.6 TB Kimi K3 checkpoint needing 32 GB. None of the nine families need a GPU; one only ever makes streaming faster. Once OLMoE proves the pipeline works, the same coli chat --model <path> line loads any bigger family, coli auto-picks the matching engine from the model folder's config.json.

What are the gotchas before you rely on it?

Three things are worth knowing before you go further than this tutorial's OLMoE run. First, disk speed sets your token rate on the larger models: experts stream from storage on every forward pass, so a slow or shared drive can mean well under one token per second on a 744B model, and that is the documented, expected cost, not a bug to file. Second, OLMoE itself will feel like any other small local model, since at 7 GB it barely touches the disk-streaming machinery that makes colibri interesting; treat it as a smoke test for the install, not a demo of the real advantage. Third, on Windows the --repo conversion path can crash mid-run with a permission error while deleting a shard it just finished with, a file-locking quirk rather than lost work: rerunning against a fully downloaded checkpoint with --model <dir> instead of --repo sidesteps it entirely, and both flags ship in the same script.

RelatedOpenCode Setup: Install the Open-Source AI Coding Agent

Colibri's measured decode speed by hardware class Community-measured decode speed running the same 744B GLM-5.2 model ranges from about 0.05 to 0.1 tokens per second on a 25 gigabyte laptop-class box up to roughly 6 to 7 tokens per second on six RTX 5090 GPUs with the full model resident. GLM-5.2 (744B) DECODE SPEED, SAME ENGINE 25 GB dev box, cold 0.05-0.1 tok/s RTX 5070 Ti laptop 1.07 tok/s 128 GB CPU-only desktop ~1.8 tok/s 6x RTX 5090, full residency 5.8-6.8 tok/s Same int4 container, same engine. Only where the experts live changes. genztech.blog
Fig 2 · benchmark Community-measured decode speed for the full 744B GLM-5.2 model, on the same engine, from a 25 GB laptop-class box up to six RTX 5090s with every expert resident. Figures from colibri's own published benchmark log.

How does colibri compare with llama.cpp and Ollama?

Traitcolibrillama.cppOllama
Model must fit in RAM/VRAMNo, streams experts from diskYes, with partial GPU offloadYes
Largest model on a 25 GB boxUp to 744B, slowlyRoughly 30-70B quantizedRoughly 30-70B quantized
Runtime dependenciesPure C, zero depsC++ with CUDA/Metal backendsWraps llama.cpp behind a Go server
LicenseApache 2.0MITMIT

Colibri is not trying to replace llama.cpp or Ollama for models that already fit comfortably on your hardware, both remain faster and simpler for that case. It exists for the gap above them: models too large for any consumer RAM or VRAM budget, where the alternative is not a slower local run but no local run at all.

What to watch · 2026
  • More open model families. The maintainer names MiniMax as a candidate for the next sibling engine, arriving whenever someone measures it end to end and publishes the numbers.
  • Dual-SSD adoption. The weighted two-drive streaming mode already ships, but the project is still asking the community for broader independent read-bandwidth A/Bs before calling it settled.
  • Whether frontier labs keep releasing open weights. Colibri's entire premise depends on GLM, Kimi, Qwen and DeepSeek continuing to publish full, downloadable checkpoints rather than API-only access.

Our take

Colibri is doing something genuinely different from the crowded field of local-LLM runners: instead of competing on quantization tricks for models that already fit in memory, it questions whether a model needs to fit in memory at all. Treating VRAM, RAM and an NVMe drive as one hierarchy, and measuring rather than guessing which experts are hot, is a real systems idea with published numbers behind it, not a marketing claim. The honest caveat is that the headline use case, a 744B or 2.8T model on a normal PC, is currently slow enough on modest hardware to read as a proof of concept rather than a daily driver, and the project says so itself. What makes it worth setting up today is the 7B OLMoE path above: it costs 7 GB and about fifteen minutes, and leaves you holding a working pipeline for the same engine that, given enough disk, loads a model far beyond anything a single consumer GPU could otherwise touch.

Primary sources

Original analysis by GenZTech. Tool documentation: JustVugg/colibri on GitHub.