A 28.9M-Parameter LLM Now Runs on an $8 Microcontroller

A language model with 28.9 million parameters is now generating text on an ESP32-S3, a microcontroller that costs about $8 and has 512KB of SRAM. It runs at roughly 9.5 tokens per second with no network connection. In the same week, a 9.36 million parameter text-to-speech model shipped under Apache-2.0 that beats several larger compact baselines in blind listening tests. Neither project is a scaled-down frontier model. Both are arguments that the interesting engineering has moved to where the memory is.

The instinct is to file these as hobby curiosities. That undersells what the ESP32 project in particular figured out, which is a memory-hierarchy trick borrowed from Google's Gemma line and applied somewhere nobody expected it to fit.

RelatedLaguna S 2.1: 8B Active Params, 70% Terminal-Bench

How does a 28.9M model fit in 512KB of SRAM?

It does not. That is the point.

Of the 28.9 million parameters, roughly 25 million never enter fast memory at all. They live in a flash lookup table using Per-Layer Embeddings, the architecture Google introduced with Gemma, and the model reads about 450 bytes per token out of it. The remaining few million parameters are the actual compute core, and that is small enough to sit in SRAM and run on every token.

The result is a three-tier split: the thinking core in fast SRAM, output processing in the 8MB PSRAM, and the bulk of the weights parked in 16MB of slow flash where their access pattern is sparse enough that latency stops mattering. Quantized to 4 bits, the whole thing is 14.9MB on disk. Throughput lands at 9.5 tokens per second end to end, 9.7 for pure compute, which means the flash reads cost about 2% of wall-clock time.

What can it actually do?

Very little, and the author is direct about it. The model was trained on TinyStories, so it produces short, simple fictional narratives. It cannot answer questions, follow complex instructions, or retrieve facts. Anyone expecting a pocket assistant will be disappointed within one prompt.

Judging it on capability misses what was demonstrated. The claim is not that 28.9M parameters are enough to be useful. It is that the ceiling on what fits in a class of hardware just moved, and the technique that moved it generalizes to models that were trained to do something.

Is the speech model the same kind of result?

No, and the difference is instructive. Inflect-Micro-v2 is 9.36 million deployable parameters, 37.53MB in FP32, built on a VITS-style end-to-end architecture with phoneme processing, monotonic alignment and an integrated neural waveform decoder producing 24kHz mono audio. It is a fixed-voice English system, not multi-speaker and not a cloning model, with speed, variation and seed as the exposed controls.

Where the ESP32 project is a systems result, this one is a quality result. It scores 66.2% human preference in community blind listening tests, a UTMOS22 naturalness score of 4.395, and 3.99% semantic word error rate on ASR intelligibility. It runs 6.28 times faster than real time on a four-thread CPU. In preference testing it beats KittenTTS Nano, Piper Low and Supertonic 3, several of which carry similar or larger weights.

One caveat belongs on the record: the model card states that the training corpus pipeline, filtering infrastructure and full optimization recipe are not part of the public release. Open weights under Apache-2.0, closed method. That is a common shape in 2026 and worth naming rather than glossing.

Related200 Startups Tell Trump: Do Not Ban Chinese Open-Weight AI

Why this matters more than the parameter counts suggest

Both projects attack the same assumption, which is that model quality and deployment cost move together. The ESP32 work shows that parameter count and working-set size can be decoupled by architecture. The TTS work shows that a sub-10M model can win on perceived quality against larger ones when the training is targeted rather than general.

For anyone building on hardware, that combination is the actionable part. A device with 8MB of PSRAM was recently a device that called an API. The interesting question for the next year is not whether tiny models get smarter. It is whether the Per-Layer Embedding trick holds up when applied to a model trained on something more demanding than children's stories, because that is the experiment that turns a demo into a deployment pattern.

Our take

The ESP32 model cannot answer a question and the TTS model speaks in one voice, so it would be easy to write both off. We would not. The thing worth paying attention to is that neither project got its result by making a model bigger or by waiting for better silicon. One rearranged where the weights live. The other spent its parameter budget on a narrow task instead of a broad one.

Those are both engineering moves available to anyone right now, on hardware that costs less than lunch. The frontier-model race will keep consuming the headlines and most of the capital. Meanwhile the floor is rising fast enough that the assumption behind a great many product decisions, namely that inference has to happen somewhere else, is quietly expiring.

Original analysis by GenZTech, based on the project repository and model card published by their respective authors. Source: github.com/slvDev/esp32-ai