A new open-source inference engine called Colibrì has demonstrated the ability to run GLM-5.2, a 744-billion-parameter frontier AI model developed by China’s Zhipu AI, on a consumer PC with only 25 GB of RAM, no GPU required.
The project, released on GitHub on July 10 by developer vforno (JustVugg), achieves this by exploiting the Mixture-of-Experts (MoE) architecture of GLM-5.2. Instead of loading all 744 billion parameters into memory, which would normally require multiple Nvidia H200 GPUs, Colibrì divides the model into two tiers and streams the bulk of its weights from an NVMe SSD on demand.
About 17 billion parameters, the dense layers, attention mechanism, and shared embeddings, are quantized to 4-bit precision and stored permanently in RAM, occupying approximately 9.9 GB. The remaining 21,504 routed expert modules, totaling about 370 GB on disk, are read from storage only when the model’s router selects them for a given token. Unselected experts are never loaded.
An LRU cache keeps recently used experts in RAM, and a learning cache records which experts are most frequently called during a user’s interactions, automatically pinning them at startup. This means Colibrì actually gets faster the more it is used.
Performance is the trade-off
Running a frontier model on consumer hardware comes with severe speed constraints. On the developer’s test machine, a 12-core CPU with 25 GB RAM via WSL2, cold-start inference achieves approximately 0.05 to 0.1 tokens per second, or roughly one word every 10 to 20 seconds. An Apple M5 Max laptop reaches about 1.06 tokens per second, and a Ryzen 9 9950X with PCIe 5.0 NVMe manages around 0.28 tokens per second.
Multi-Token Prediction (MTP) speculation adds 2.2 to 2.8 tokens per forward pass once the cache is warmed. The developer is transparent about the speed trade-off: the goal is not real-time interaction but the ability to run frontier-class models at all on hardware that costs less than a single H100 GPU fan module.
The engine itself is written in approximately 1,300 lines of pure C with zero external dependencies, no Python, no PyTorch, no CUDA. It requires Linux or WSL2, an OpenMP-compatible GCC compiler, an AVX2-capable CPU, at least 16 GB of RAM, and an NVMe SSD capable of storing the ~370 GB model. An OpenAI-compatible API server is built in, allowing local applications to connect to the model as they would to any cloud provider.
GLM-5.2 is notable not only for its size but for its licensing: released under the MIT license, it is openly downloadable and has been shown to outperform Anthropic’s Claude Opus 4.7 and compete with Fable 5 on several benchmarks. The combination of an open-weight frontier model and Colibrì’s minimal hardware requirements has drawn attention from privacy advocates and researchers concerned about dependence on cloud-based AI providers.