Unsloth Setup: Run and Train LLMs on Your Own Machine

Unsloth is a free, open source desktop app that runs and trains AI models entirely on hardware you already own, and it is one of today's fastest-climbing repositories on GitHub trending with roughly 572 stars added in a day on top of 72,996 total. That combination is the whole story: Ollama and LM Studio made local inference easy, but neither lets you fine-tune a model from the same window. Unsloth does, and the download is a 42 MB installer rather than a Python environment.

• The current beta is v0.1.800, shipped August 14, 2026, with native installers for Windows, macOS, Ubuntu, generic Linux and ARM64.
• There are three ways to install it: the Tauri desktop app, a self-hosted web UI called Unsloth Studio, and a pip-installable Python package called Unsloth Core.
• Training claims are the reason it exists: 2x faster fine-tuning with 70% less VRAM, with LoRA, QLoRA, full fine-tuning, GRPO, DPO and FP8 all supported.
• One command, unsloth start claude, points Claude Code, Codex, OpenCode or OpenClaw at a model running on your own box.

What is Unsloth and why is it trending?

Unsloth started in November 2023 as a Python library that made LoRA fine-tuning faster and cheaper, and for two years that is all it was: a dependency you imported inside a Colab notebook. The repository now describes itself as "the first desktop app to run and train models," and that pivot is what is pulling stars. The August 14 release, tagged v0.1.800-beta, added support for Qwen3.8-27B running locally on 17 GB of RAM through the project's Dynamic GGUF quantizations, plus a 1-bit build of the 2.4-trillion-parameter Qwen3.8 for people with unreasonable hardware.

RelatedOpenWork Setup: Open-Source Claude Cowork in 15 Minutes

The model list is the other draw. Kimi K3, MiniMax-H3, DeepSeek-V4, Gemma 4, Muse Glimmer and FLUX all have first-party guides, and the app handles diffusion, embedding and text-to-speech models rather than text-only LLMs. Hardware coverage is unusually broad for a young desktop app: CPU, Apple Silicon, NVIDIA, AMD, Intel and multi-GPU setups all have supported paths, with training working on RTX 30, 40 and 50 series, Blackwell, DGX Spark and Apple Silicon via MLX.

How do you install Unsloth Desktop on Windows, macOS and Linux?

The desktop app is the recommended path and needs no toolchain. Download the installer for your platform from the v0.1.800-beta release page: Windows is a 41.8 MB .exe, macOS a 43.0 MB .dmg, Ubuntu a 44.6 MB .deb, generic Linux a 44.2 MB AppImage, and ARM64 Linux a 42.8 MB tarball. If you would rather not click through a browser, the project publishes one-line installers:

# macOS, Linux, WSL
curl -fsSL https://unsloth.ai/install.sh | sh

# Windows PowerShell
irm https://unsloth.ai/install.ps1 | iex

The same command updates an existing install, which is worth remembering while the app is still on beta version numbers. Both scripts pull PyTorch by default; if you only ever plan to run GGUF files and never train, you can skip that download entirely with UNSLOTH_NO_TORCH=1 placed before the shell invocation.

How do you run Unsloth Studio, the web UI?

Studio is the same product served over HTTP, which is what you want if the GPU lives in a machine under your desk rather than the one on it. After running the installer above, launch it:

$ unsloth studio -p 8888

By default it binds to 127.0.0.1 and is reachable from that machine only. The GGUF inference backend is chosen automatically, but you can force it under Settings, System, GGUF inference engine, or set it before the first install:

# pick the backend before installing: cpu, cuda, rocm, vulkan or auto
export UNSLOTH_LLAMA_CPP_BACKEND=vulkan
curl -fsSL https://unsloth.ai/install.sh | sh

The Vulkan path exists specifically for older AMD cards the project has no ROCm PyTorch wheels for, namely Polaris and RDNA 1. On those, GGUF chat runs on the GPU while training stays unavailable. Note that Vulkan accelerates inference only, never training. macOS ignores the variable and always uses the Metal build.

How do you point Claude Code or Codex at a local model?

This is the feature most people will actually use daily. Start Unsloth, load a model, open your project folder, then run one command:

$ unsloth start claude

Swap claude for codex, hermes, openclaw or opencode depending on your agent. Under the hood this serves the loaded model through OpenAI-compatible and Anthropic-compatible endpoints, so the agent does not know it stopped talking to a vendor. If you want to keep your paid frontier model for hard work and offload the cheap turns locally, there is a subagent mode:

unsloth start claude --as-subagent --model unsloth/model-GGUF:quant

There is also a Docker image if you would rather keep the whole thing contained:

RelatedStrix Setup: Run an AI Penetration Tester on Your Code

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

How does it compare with Ollama and LM Studio?

Read that table honestly. For pure inference, Ollama and LM Studio are more finished products with fewer sharp edges, and Unsloth can export to both formats anyway. The reason to install Unsloth is the training column, plus the breadth of model types. If you never intend to fine-tune anything, you are choosing a beta app to do a job two stable ones already do well.

What are the gotchas before you rely on it?

Five things worth knowing. First, the version number is honest: v0.1.800-beta is beta, and the repository carries over 1,250 open issues against 73,000 stars. Treat it as a tool you experiment with, not one you build a production pipeline on this month. Second, the licensing is dual: the core Unsloth package is Apache 2.0, but the Studio UI is AGPL-3.0, which matters if you plan to embed or offer it as a hosted service. Third, server-side tools including Python and terminal code execution run as your user and are on by default, so anyone who reaches the server with the API key can execute code on that machine. Pass --disable-tools whenever you expose it.

Fourth, publishing Studio on a public URL forces an admin password prompt in the terminal before the link goes up, and a headless run without one shuts down after an hour unless you set it. Use the UNSLOTH_STUDIO_PASSWORD environment variable rather than a literal --password flag, which lands in your shell history and process list. Fifth, when you bind to all interfaces, Unsloth contacts ifconfig.me and check-host.net to work out your public address and whether the port is reachable. Both are third parties. Set UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK=1 if that is not acceptable on your network.

Our take

The interesting claim here is not speed, it is scope. Every local AI app of the last two years has been an inference shell: download a GGUF, chat with it, wire it into something. Training stayed on the other side of a wall made of CUDA versions, Python environments and a Colab tab you were afraid to close. Unsloth putting a fine-tuning run behind the same window as a chat box is a genuinely different product category, and it explains why a two-year-old library is trending again as if it just launched.

What we would not do yet is move a workflow onto it. Beta version numbers and 1,250 open issues are not a reason to skip the download, but they are a reason to keep Ollama installed. The honest recommendation: install it for the training features, keep whatever you already use for serving, and revisit the serving question when the beta tag comes off. The 42 MB download costs you nothing to find out whether local fine-tuning is finally practical on your hardware, and for a large number of people it now is.

Original analysis by GenZTech. Tool documentation: unslothai/unsloth on GitHub.