TL;DR: If Docker Desktop is on your machine, a local LLM server is too — Docker Model Runner ships inside Docker Desktop 4.40+ and serves an OpenAI-compatible API that Cline, Continue.dev, and Aider connect to with three lines of config. It passes llama.cpp’s --jinja flag for you, so tool calling works out of the box. The catch it kept: a 4,096-token default context that silently strangles agents.

What you’ll be able to do after this guide:

• Turn the Docker install you already have into a model server on http://localhost:12434/engines/v1, pulling coding models as OCI artifacts (docker model pull ai/qwen3-coder) or straight from Hugging Face (docker model pull hf.co/...)
• Fix the 4K default context before it breaks agent mode, with the one docker model configure command that persists across restarts
• Wire the endpoint into Cline (OpenAI Compatible provider), Continue.dev (openai provider with apiBase), and Aider (openai/ prefix) — the exact configs Docker’s own IDE-integration docs bless

Honest take: For a developer who already lives in Docker, Model Runner is now the lowest-friction local backend on this site — one checkbox and one pull, no separate app, and tool calling configured correctly by default where raw llama-server makes you find the flag yourself. Ollama still wins on model-library breadth and community troubleshooting mass, but the gap is a lot thinner than most people think.

---

The local backend you didn’t have to install

This is the fifth engine in our local-backend series, after Ollama, LM Studio, llama.cpp’s llama-server, and vLLM. Docker Model Runner (DMR) is the one with the unfair distribution advantage: it ships inside Docker Desktop 4.40 and later on macOS and Windows, and on Linux it’s an apt/dnf package (docker-model-plugin) for Docker Engine. Docker open-sourced the whole thing (Apache 2.0, docker/model-runner on GitHub), and it’s under active development — v1.2.8 landed August 12, 2026, ten days before this article.

Under the hood there’s no mystery: the default inference engine is llama.cpp, the same code path as Ollama, LM Studio, and llama-server. What’s different is the packaging philosophy. Models are OCI artifacts pulled from Docker Hub’s ai/ namespace with the same mental model as images — docker model pull, docker model ls, docker model rm — and they run as host processes with GPU access (Apple silicon on Mac, NVIDIA on Windows), not inside containers. A model stays loaded in memory until another model is requested or a 5-minute inactivity timeout unloads it, which is Ollama’s keepalive behavior with slightly different numbers.

Who should care: anyone whose machine already runs Docker Desktop for work and doesn’t want a second model-manager app; teams that want model versions pinned in compose.yaml next to the services that use them; and anyone burned by llama-server’s flag surface who still wants llama.cpp behavior.

Step 0 — Switch it on and find the port

DMR is included but not always listening where your coding tools need it. Three setups, tested August 22, 2026 against the current docs:

Docker Desktop (macOS/Windows): Settings → AI tab → check Enable Docker Model Runner. For tools running on your host (all three tools in this guide), also check Enable host-side TCP support — the default port is 12434. On Windows with a supported NVIDIA GPU, a third checkbox enables GPU-backed inference. The CLI equivalent:

docker desktop enable model-runner --tcp 12434

Docker Engine (Linux): install the plugin, and note that here TCP on port 12434 is enabled by default:

sudo apt-get update && sudo apt-get install docker-model-plugin
docker model version

That host-side TCP toggle is the setup mistake to make once and never again. Inside containers, DMR is always reachable at http://model-runner.docker.internal/engines/v1 — but Cline, Continue, and Aider are not containers. They live on your host and need the TCP endpoint. If your tool times out connecting to localhost:12434 while docker model run works fine in a terminal, the checkbox is off. Same family of failure as the Ollama connection-refused error, different cause.

Sanity-check the endpoint the way every OpenAI-compatible client will see it:

curl http://localhost:12434/engines/v1/models
# → {"object":"list","data":[{"id":"ai/qwen3-coder", ...}]}

The /engines/v1 prefix is the piece to memorize — it’s where DMR differs from every other backend in this series. The usual routes hang off it: /chat/completions, /completions, /embeddings, /models. (You may also see /engines/llama.cpp/v1/... in Docker’s docs; that’s the explicit form of the same thing.) No API key exists or is needed; when a tool insists on one, any non-empty string satisfies it.

Step 1 — Pull a model that can actually drive an agent

Models come from Docker Hub’s ai/ namespace — 99 repositories as of today, curated and packaged in multiple quantizations with the tag scheme {parameters}-{quantization}. The one to start with for agent coding:

docker model pull ai/qwen3-coder
docker model run ai/qwen3-coder "Write a Python function that merges two sorted lists."

Verified sizes and fit, from the Docker Hub model cards today:

VRAM / unified memory	Model	Size	Why
16 GB	ai/gpt-oss	11.3 GB	OpenAI’s 20B-class MoE in its native MXFP4; the strongest tool-caller that leaves KV-cache room at this tier
24 GB	ai/qwen3-coder	17.3 GB	Qwen3-Coder-30B-A3B (30.5B total, 3.3B active), 262,144-token native context, tool-calling support listed on the card. The default agent-coding pick
Anything else	docker model pull hf.co/<org>/<repo>-GGUF	varies	DMR pulls GGUF directly from Hugging Face, so the whole GGUF ecosystem is reachable without waiting for the ai/ namespace

The Hugging Face pull is the escape hatch that keeps DMR’s curated library from becoming a ceiling — docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF works exactly like an ai/ pull, per the official get-started docs. For the hardware side of the table — what card, how much memory, whether 30B-class MoE models are realistic on your box — that’s runaihome.com’s beat: their best local AI models by VRAM guide is the companion piece. On a RTX 3090-class card, both models above run comfortably.

Step 2 — Tool calling: the flag you don’t have to find

Cline’s agent loop and Continue’s agent mode live or die on function calling. In our llama-server guide, the single most important fact was that --jinja is required for tool use and nothing tells you it’s missing — the model just narrates JSON into the chat instead of acting, the failure we dissected in the Cline tool-use loop fix.

DMR handles this for you. Its llama.cpp backend appends --jinja to the server arguments by default for text models — it’s right there in the open-source config code, along with -ngl 999 (full GPU offload when a GPU exists). So a tool-calling-capable model pulled from the ai/ namespace does OpenAI-style tool calls through /engines/v1/chat/completions with zero flags, and Docker’s own IDE-integration docs wire it straight into Cline — an agent that won’t function without tool calling — as a supported configuration.

Two honest caveats. First, the flag being set doesn’t make a weak model a good agent — a small model with a correct template still emits malformed calls under pressure; ai/qwen3-coder and ai/gpt-oss are the two we’d trust, consistent with every other backend in this series. Second, DMR won’t referee capability the way Ollama does: Ollama scans the chat template and rejects tool requests with a 400 (the “does not support tools” error), while DMR — like raw llama-server — serves whatever the template produces. With a non-tool-trained model that means silent degradation, not a clean error.

Step 3 — Kill the 4K context before it kills agent mode

Here’s what DMR inherited from the llama.cpp lineage without fixing: the default context size for the llama.cpp engine is 4,096 tokens, straight from Docker’s configuration docs. Cline’s system prompt plus tool definitions plus a couple of open files blow past 4K before the model produces its first token. The symptom is the same slow-motion failure we documented for Ollama’s identical default: the agent forgets the task mid-run, re-reads files it already read, or loops — no error anywhere, because truncation isn’t an error.

The fix is one command, and unlike an env var it persists until you remove the model:

docker model configure --context-size 32768 ai/qwen3-coder

32K is the working floor we recommend for agent use on every backend; ai/qwen3-coder natively supports 262K, so the binding constraint is your memory, not the model — KV cache scales with the number you pick. To verify what the model supports versus what you configured, docker model inspect ai/qwen3-coder shows the card’s maximum. Reset with --context-size -1.

If you’d rather declare it than run it, the same knob exists in Compose, which is DMR’s quiet killer feature for teams — the model, its context, and its runtime flags live in the repo next to the services that use them:

models:
  coder:
    model: ai/qwen3-coder
    context_size: 32768

One more inherited behavior worth knowing: DMR also supports a vLLM backend (CUDA, safetensors) as an alternative engine, and there the context default inverts to the model’s full trained maximum — the loud-crash-instead-of-silent-truncation trade we covered in the vLLM guide. The llama.cpp engine is the default and the right choice on a single developer machine.

Step 4 — Wire in the three tools

These are the configurations from Docker’s own IDE-integrations documentation, cross-checked against each tool’s docs today.

Cline: OpenAI Compatible provider

Cline’s local-models docs still cover only Ollama, LM Studio, and Atomic Chat (there’s an open discussion asking for native DMR support), so the route both sides agree on is Cline’s OpenAI Compatible provider:

• Base URL: http://localhost:12434/engines/v1
• API Key: any non-empty string (e.g. dmr)
• Model ID: ai/qwen3-coder
• Model Configuration: set the context window to the value you gave docker model configure, so Cline’s truncation math matches reality; check Computer Use if you want browser tooling (it’s gated on function calling, which Step 2 established you have)

And turn on Use Compact Prompt in Settings → Features — Cline’s local-model docs recommend it for every local backend, and a leaner system prompt leaves more of your 32K for code.

Continue.dev: openai provider, apiBase with the full prefix

Continue has no dedicated DMR provider; Docker’s docs use Continue’s standard openai provider with a redirected apiBase — note the base includes /engines/v1, DMR’s whole path quirk in one line:

models:
  - name: Qwen3-Coder (Docker Model Runner)
    provider: openai
    model: ai/qwen3-coder
    apiBase: http://localhost:12434/engines/v1
    apiKey: dmr
    roles: [chat, edit, apply]

Agent mode gates on the tool_use capability, which Continue autodetects from the endpoint; if agent tools stay greyed out, add capabilities: [tool_use] to the block (capabilities are additive — the override can’t break autodetection).

Aider: OpenAI-compatible env vars

Same pattern as every OpenAI-compatible backend, with DMR’s base URL:

export OPENAI_API_BASE=http://localhost:12434/engines/v1
export OPENAI_API_KEY=dmr
aider --model openai/ai/qwen3-coder

The double prefix in openai/ai/qwen3-coder looks wrong and isn’t — openai/ routes Aider’s LiteLLM layer to the generic OpenAI-compatible path, and ai/qwen3-coder is the literal model ID DMR serves. Docker’s docs show exactly this form. You’ll get Aider’s usual warning about unknown context window and costs for unrecognized models; silence it with a .aider.model.metadata.json (max_input_tokens: 32768, costs at 0), same as for llama-server. Aider’s search/replace edit loop is the least tool-call-dependent of the three, making it the most forgiving client if you’re testing a smaller model — and if the blocks themselves fail to apply, that’s its own failure mode, not DMR’s.

The five-backend decision table

	Docker Model Runner	Ollama	llama.cpp server	LM Studio	vLLM
Best for	Devs already on Docker; models in Compose	One dev, zero fuss	One dev who wants every knob	One dev who wants a GUI	Concurrent agents, team boxes
Install	Already in Docker Desktop 4.40+	Separate app	Build/download binary	Separate app	pip/uv on Linux
Model source	Docker Hub ai/ (OCI) + HF GGUF	Own library + HF	Any GGUF	Own catalog (GGUF/MLX)	HF safetensors
Tool calling	On by default (--jinja auto)	Automatic, template-gated	Manual --jinja flag	Automatic	Two flags + per-family parser
Context default	4K — configure once, persists	4K (truncates silent)	4K (truncates silent)	4K per-model setting	Model max (OOMs loud)
Base URL quirk	/engines/v1 prefix	/v1	/v1 (Continue: bare port)	/v1	/v1
The catch	Desktop needs TCP toggled on; youngest troubleshooting corpus	Wrapper hides the knobs	You manage everything	Closed-source shell	Linux-first, VRAM-greedy

When to skip it

Skip DMR if you don’t otherwise use Docker — installing Docker Desktop just to get a llama.cpp wrapper is strictly more moving parts than installing Ollama, and when something misbehaves, Ollama’s three-years-deep pile of GitHub issues and Reddit threads is worth real debugging hours. Skip it on machines where Docker Desktop’s own resource footprint already hurts. And the standing verdict from our Cursor + Ollama guide applies to every backend here: local models remain a privacy and cost play, not a capability play — hosted frontier models still win long agentic chains.

Take it seriously if Docker Desktop is already running on your machine every day — you’re one checkbox, one pull, and one configure command away from a tool-calling local backend, which is less setup than any other engine in this series. For the open-source side of the stack beyond Docker’s curation, aifoss.dev tracks the FOSS tooling; if your GPU lives in the cloud instead, a RunPod box running Docker Engine gets the same plugin, with TCP already on by default.

FAQ

Does the base URL need /v1 or /engines/v1? /engines/v1 — for all three tools. It’s the one backend in this series where copying a config from an Ollama or vLLM tutorial and only changing the port will fail. (Inside a container, swap the host: http://model-runner.docker.internal/engines/v1.)

Is tool calling really on by default? For text models on the default llama.cpp engine, yes — DMR’s backend config appends --jinja automatically (it’s in the open-source repo), which is the flag raw llama-server users must set by hand. Whether the model emits well-formed calls is a separate question; stick to ai/qwen3-coder or ai/gpt-oss for agent work.

Why did Cline lose track of the task even though the model loaded fine? Almost certainly the 4,096-token default context. Run docker model configure --context-size 32768 ai/qwen3-coder and set the same number in Cline’s model configuration. Silent truncation, not the model, is the usual culprit — same disease as Ollama, same cure.

Can I use models that aren’t in Docker Hub’s ai/ namespace? Yes — docker model pull hf.co/<org>/<repo>-GGUF pulls GGUF directly from Hugging Face. The ai/ namespace (99 repos today) is curation, not a wall.

Does my model run inside a container? No. DMR runs inference as a host process with direct GPU access (Apple silicon on macOS, NVIDIA on Windows/Linux); models are packaged and distributed as OCI artifacts but not executed in containers. That’s why performance matches other llama.cpp wrappers rather than paying a virtualization tax.

Sources

• Docker Model Runner — official repository (Apache 2.0)
• Docker Model Runner releases (v1.2.8, Aug 12 2026)
• DMR llama.cpp backend config — --jinja default in source
• Get started with DMR — Docker docs
• DMR configuration options (context-size defaults) — Docker docs
• DMR IDE integrations (Cline, Continue, Aider configs) — Docker docs
• Run LLMs locally with Docker Model Runner — Docker blog
• ai/qwen3-coder model card — Docker Hub
• ai/gpt-oss model card — Docker Hub
• Cline running models locally — official docs
• Cline OpenAI Compatible provider — official docs
• Continue.dev OpenAI provider (apiBase override) — official docs
• Aider OpenAI-compatible API — official docs

Last updated August 22, 2026. Pricing and features change frequently; verify current state before purchasing.