LOCAL INFERENCE × CODING AGENTS
Pick one engine to load the model, then connect one harness to give it files, tools, and an agent loop. The model stays on your Mac; the two apps talk over a local HTTP endpoint.
STEP 1 / SERVE THE MODEL
All three can expose a local API. Ollama minimizes setup, llama.cpp exposes the most knobs, and LM Studio adds a visual model manager.
LEAST SETUP
Import a downloaded GGUF with a Modelfile, or pull an existing Ollama model. Its launch command can configure supported coding harnesses for you.
FROM ./models/your-model.gguf
PARAMETER num_ctx 65536brew install ollama
ollama create local-model -f ./Modelfile
ollama run local-modellocalhost:11434MOST CONTROL
Point the server directly at a GGUF. Give it a stable alias for harness configuration and enable Jinja templates for tool calls.
brew install llama.cpp
llama-server \
-m ./models/your-model.gguf \
--alias local-model \
-c 65536 \
--jinjalocalhost:8080VISUAL WORKFLOW
Install and open LM Studio once to enable its CLI. Then find and load a model, set a memorable identifier, and start the local server.
lms get
lms load <model-key> \
--identifier=local-model \
--gpu=max \
--context-length=65536
lms server start --port 1234localhost:1234STEP 2 / PICK A HARNESS
Claude Code can target llama.cpp's Anthropic-compatible endpoint, but that path is less complete and less documented than Ollama or LM Studio. Use it as an experiment, not the default.
THE AGENT LAYER
Start the engine first. Then give the harness the matching base URL and the exact model identifier exposed by that engine.
MINIMAL, EXTENSIBLE
Add any OpenAI-compatible local server in ~/.pi/agent/models.json, start Pi, then choose the model with /model.
curl -fsSL https://pi.dev/install.sh | sh{
"providers": {
"local": {
"baseUrl": "http://localhost:8080/v1",
"api": "openai-completions",
"apiKey": "local",
"compat": {
"supportsDeveloperRole": false,
"supportsReasoningEffort": false
},
"models": [{
"id": "local-model",
"name": "My local model",
"reasoning": true,
"contextWindow": 65536,
"maxTokens": 8192
}]
}
}
}POLISHED AGENT UX
Ollama has the shortest setup. LM Studio exposes the Anthropic endpoint Claude Code expects. Use a large context window for tool definitions and project files.
curl -fsSL https://claude.ai/install.sh | bash
ollama launch claude --model local-modelexport ANTHROPIC_BASE_URL=http://localhost:1234
export ANTHROPIC_AUTH_TOKEN=lmstudio
export CLAUDE_CODE_ATTRIBUTION_HEADER=0
claude --model local-modelexport ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_AUTH_TOKEN=llama.cpp
export ANTHROPIC_API_KEY=""
claude --model local-modelOPEN SOURCE HARNESS
Ollama can launch it directly. For llama.cpp or LM Studio, add a custom OpenAI-compatible provider and change only the base URL.
curl -fsSL https://opencode.ai/install | bash
ollama launch opencode{
"$schema": "https://opencode.ai/config.json",
"provider": {
"local": {
"npm": "@ai-sdk/openai-compatible",
"name": "Local model",
"options": {
"baseURL": "http://localhost:8080/v1"
},
"models": {
"local-model": {
"name": "My local model"
}
}
}
}
}KEEP THIS HANDY
Pi and OpenCode use the OpenAI-compatible URL. Claude Code uses the Anthropic base URL where the engine supports it.
11434http://localhost:11434/v1http://localhost:114348080http://localhost:8080/v1http://localhost:80801234http://localhost:1234/v1http://localhost:1234THE CONTEXT TAX
The GGUF file is only the starting point. Context, the KV cache, tool definitions, and the harness itself all need memory. A model that barely fits may slow down or fail once an agent scans a repo.