Magnitude Setup: Free Local Models for Your AI Agent

Magnitude is a free, open source inference server that profiles your computer, tells you which local AI models will actually run well on it, then downloads, tunes, and wires them straight into the coding agent you already use. It added 686 stars in a single day on GitHub's trending page and has crossed 3,100 total, and the pitch is simple: stop paying per-token API bills and stop guessing whether your laptop can run a given model.

• One CLI, one command to install, installs from npm as @magnitudedev/cli and runs on macOS, Linux, and Windows through WSL.
• It profiles your hardware first, chip, memory, bandwidth, and only recommends models that will run acceptably, with an estimated tokens-per-second for each.
• It auto-configures your agent harness: Claude Code, OpenCode, Cline, Codex, Hermes, Pi, OpenClaw, and Oh My Pi are all supported, or you can use Magnitude's own built-in harness.
• Everything runs fully offline after the model downloads: no API keys, no token costs, no rate limits, and it is Apache 2.0 licensed so you can inspect or modify it.

The exact steps, start to finish

1. Step 1. Check your machine is ready.

   # macOS / Linux
   node --version
   npm --version
   
   # Windows: Magnitude requires WSL
   wsl --status
   node --version
   npm --version

   Magnitude needs Node.js and npm to install (it ships as an npm package), and on Windows it only runs inside WSL, not natively. If wsl --status errors out, install WSL first from an elevated PowerShell with wsl --install, reboot, then re-check.
2. Step 2. Install the CLI.

   $ npm i -g @magnitudedev/cli

   This pulls the magnitude command onto your PATH. No account, signup, or API key is needed for this step or any step below, Magnitude's whole point is running models locally without a vendor relationship.
3. Step 3. Confirm it installed.

   $ magnitude --help

   Every Magnitude command accepts --help; if this prints the command list instead of "command not found," the install worked.
4. Step 4. Run interactive setup and pick a model.

   $ magnitude setup

   This is the core of the tool: it profiles your CPU, GPU, and memory, then shows you a catalog of models it has determined will actually run on your specific machine, each with an estimated tok/s. Pick one and Magnitude downloads and tunes it (speculative decoding and concurrency settings included) for your hardware automatically.
5. Step 5. Confirm the model installed.

   $ magnitude models status

   Lists every model you have installed and where it lives on disk (Magnitude checks the standard Hugging Face cache locations, HF_HUB_CACHE, HF_HOME, or ~/.cache/huggingface/hub by default).
6. Step 6. Connect it to your coding agent.

   $ magnitude connections add <harness> [--set-model <model-id>] [--install-skill]

   Swap <harness> for whichever agent you run: Claude Code, OpenCode, Cline, Codex, Hermes, Pi, or Oh My Pi. This step rewrites that harness's own model configuration to point at your local Magnitude server, no manual endpoint editing required. Run magnitude connections list first if you want to see which harnesses it detected on your system.
7. Step 7. Have your first real, offline conversation. Open the harness you just connected (or run magnitude setup's built-in harness) and send it a real prompt. Magnitude loads the model into memory on that first request and unloads it again once the harness goes idle, so nothing sits in RAM burning battery between prompts. A working setup answers with no API error, no vendor rate-limit message, and no network request leaving your machine.

What is Magnitude and why is it trending?

Magnitude, from the startup Magnitude Dev, is an inference server rather than a chat app: it has no UI of its own beyond its CLI and instead plugs into agents you already run, Claude Code, OpenCode, Cline, Codex, Hermes, Pi, OpenClaw, and Oh My Pi are all listed as supported connections. The pitch that is driving its trending spike is the gap it fills between "just install Ollama" and actually getting good results: an agent that tells you to run Ollama has no idea what GPU you have, how much VRAM is free, or which quantization will fit without swapping to disk. Magnitude profiles the machine first and only shows models with a real shot at running well, with an estimated tokens-per-second attached to each one, then handles tuning details like speculative decoding and concurrency itself. It launched its CLI on npm and the latest tagged release, @magnitudedev/[email protected], shipped September 2, 2026.

RelatedOpenCode Setup: Install the Open-Source AI Coding Agent

How do you install Magnitude on macOS, Linux, and Windows?

Installation is identical everywhere once npm is available:

# macOS and Linux, native
npm i -g @magnitudedev/cli
magnitude setup

# Windows, inside WSL (no native Windows support yet)
wsl
npm i -g @magnitudedev/cli
magnitude setup

On macOS, Apple Silicon gets Metal acceleration automatically once you pick a model; on Linux, Magnitude's release assets ship separate builds for CPU, Vulkan, and CUDA 11.8/12.9, and setup detects which one your GPU needs. Windows users must run every command from inside a WSL2 shell, there is currently no native Windows binary, only the WSL path documented in the project's own README. If you would rather skip the interactive browser entirely, the maintainers recommend simply handing your coding agent this exact prompt from their README: "Set up local models for me with the Magnitude CLI. Install it with npm i -g @magnitudedev/cli (or my package manager), then run magnitude docs onboarding and follow the instructions." That runs the whole flow, model pick, download, and harness connection, end to end with your agent driving.

What are the gotchas before you rely on it?

A few things worth knowing going in. First, Windows support is WSL-only today; if you live in a native PowerShell or cmd workflow, budget time to set up WSL2 first, it is not a drop-in Windows binary. Second, there is no fixed minimum hardware spec published anywhere, Magnitude's whole model is "we'll tell you what fits," which means a machine with 8GB of RAM and one with 64GB get genuinely different catalogs, do not assume a listicle's minimum-spec claim applies to you. Third, larger models, especially the CUDA-accelerated release builds, run 400 to 650MB just for the runtime binary before any model weights download, so a slow or metered connection changes the calculus. Fourth, there is no Docker image documented in the current README or docs site, so containerized deployments are not an officially supported path yet. None of these are deal breakers, they are the difference between a smooth fifteen-minute setup and a surprised one.

RelatedOpenMontage Setup: Turn Your AI Agent Into a Video Studio

Ollama and LM Studio remain the better fit if you want to manage models yourself or need a visual chat window. Magnitude wins specifically for people whose actual goal is "make my existing coding agent run on local models without me becoming a quantization expert," which is a narrower but currently underserved job.

Our take

The actual insight behind Magnitude is not "local models are good now," that argument has been won for a while. It is that picking the right local model for a specific machine, then wiring it into whatever agent you already use, is still genuinely annoying, and nobody had automated that whole chain end to end. Ollama solved distribution, LM Studio solved discovery for non-technical users, and Magnitude is trying to solve the harder problem of "what should actually run on this exact laptop and how do I make Claude Code use it without hand-editing a config file." The WSL-only Windows story and the total absence of Docker support are the two things standing between this being a nice trick and being genuinely production-ready for teams. Worth a fifteen-minute install if you are already paying real money in API bills for routine agent tasks.

Original analysis by GenZTech. Tool documentation: magnitudedev/magnitude on GitHub.