How to Use Continue.dev for Local AI Code Completion

Introduction to Continue.dev

Continue.dev is an open-source AI code assistant that plugs directly into your favorite IDE, bringing the power of large language models to your fingertips without sending your code to third-party servers. Unlike cloud-only alternatives such as GitHub Copilot, Continue.dev lets you choose where your model runs — locally on your machine, on a self-hosted server, or through a hosted API. This flexibility makes it a compelling choice for developers who care about privacy, latency, and cost.

At its core, Continue.dev works as a VS Code and JetBrains extension that intercepts your coding workflow and offers intelligent completions, chat-based assistance, code edits, and even whole-function generation. When paired with a local model runtime like Ollama, LM Studio, or llama.cpp, you get a fully offline AI pair programmer that never leaves your machine.

Why Local AI Code Completion Matters

Before diving into setup, it is worth understanding why running AI code completion locally has become such a hot topic among professional developers.

Privacy and Data Sovereignty

When you use a cloud-based completion tool, every keystroke and snippet you write may be transmitted to external servers. For teams working on proprietary codebases, regulated industries, or client projects under NDA, this is a non-starter. Local inference keeps your source code, prompts, and context entirely on your hardware.

Lower Latency

Cloud completions depend on network round trips. A local model running on a capable GPU or even a modern Apple Silicon Mac can often return completions faster than a remote API, especially for short inline suggestions.

Cost Control

Hosted AI services charge per token. Heavy users can rack up significant monthly bills. Once you have the hardware, local inference is effectively free at the margin, making it ideal for students, hobbyists, and cost-conscious startups.

Model Choice and Customization

With Continue.dev you are not locked into a single vendor's model. You can swap between specialized coding models like DeepSeek-Coder, Qwen2.5-Coder, StarCoder2, or general-purpose models like Llama 3.1, depending on the task.

Prerequisites

To follow this tutorial you will need:

A supported IDE: VS Code (recommended) or JetBrains IntelliJ/PyCharm/WebStorm
A machine with reasonable specs — at least 16GB RAM, though 32GB or more is ideal for larger models
A GPU is helpful but not strictly required; Apple Silicon Macs work well via Metal acceleration
Ollama installed for easy local model management (we will use it in this tutorial)
Node.js 18+ if you want to build or contribute to Continue.dev itself

Installing Ollama as Your Local Model Runtime

Ollama is the simplest way to run local LLMs. It handles model downloading, quantization, and exposes an OpenAI-compatible API that Continue.dev can talk to out of the box.

Installing Ollama

On macOS and Windows, download the installer from the official Ollama website. On Linux, use the install script:

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation:

ollama --version

Pulling a Coding Model

For code completion, smaller specialized models tend to offer the best speed-to-quality ratio. Qwen2.5-Coder is an excellent choice:

# Pull a strong general coding model (7B parameters)
ollama pull qwen2.5-coder:7b

# Or a smaller, faster model for inline completions
ollama pull qwen2.5-coder:1.5b

# For chat and larger edits, a bigger model works well
ollama pull deepseek-coder-v2:16b

Once downloaded, you can test the model directly from your terminal:

ollama run qwen2.5-coder:7b "Write a Python function that reverses a linked list"

Ollama runs an API server on http://localhost:11434 by default. You can confirm it is running with:

curl http://localhost:11434/api/tags

Installing the Continue.dev Extension

In VS Code

Open the Extensions panel (Ctrl+Shift+X or Cmd+Shift+X), search for "Continue", and install the extension published by Continue. Alternatively, install it from the command line:

code --install-extension continue.continue

In JetBrains IDEs

Open Settings → Plugins → Marketplace, search for "Continue", and click Install. Restart your IDE when prompted.

After installation, you will see a new Continue icon in your sidebar (VS Code) or tool window (JetBrains). Clicking it opens the chat panel where you can interact with your local model.

Configuring Continue.dev for Local Models

Continue.dev stores its configuration in a file called config.json. In VS Code, this lives at ~/.continue/config.json. You can also open it directly from the Continue panel by clicking the gear icon.

Here is a complete example configuration that wires up Ollama for both chat and autocomplete:

{
  "models": [
    {
      "title": "Qwen2.5-Coder 7B (Chat)",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b",
      "apiBase": "http://localhost:11434",
      "contextLength": 8192
    },
    {
      "title": "DeepSeek-Coder V2 (Chat)",
      "provider": "ollama",
      "model": "deepseek-coder-v2:16b",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder 1.5B (Autocomplete)",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b",
    "apiBase": "http://localhost:11434"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text",
    "apiBase": "http://localhost:11434"
  },
  "allowAnonymousTelemetry": false
}

Let us break down the key sections:

models — A list of models available in the chat dropdown. You can switch between them on the fly.
tabAutocompleteModel — The model used for inline ghost-text completions as you type. Use a small, fast model here.
embeddingsProvider — Used for codebase indexing and semantic search features. The nomic-embed-text model is lightweight and effective.
allowAnonymousTelemetry — Set to false to keep everything local and private.

You will also need to pull the embeddings model:

ollama pull nomic-embed-text

Using Continue.dev in Your Workflow

Inline Tab Completion

Once configured, simply start typing in any source file. Continue.dev will show ghost-text suggestions in gray. Press Tab to accept the full suggestion, or Cmd+→ (macOS) / Ctrl+→ (Windows/Linux) to accept word by word. Press Esc to dismiss.

For example, type the following Python function signature and pause:

def merge_sorted_lists(a: list[int], b: list[int]) -> list[int]:

Continue.dev should suggest a full implementation. If the suggestion is weak, try adding a docstring or type hints — more context helps the model produce better completions.

Chat Panel

The chat panel is where you ask broader questions. You can reference files, code selections, or your entire codebase. Some useful commands:

@file path/to/file.py — Include a specific file in context
@codebase — Use semantic search over your indexed project
@docs — Reference indexed documentation
@terminal — Include recent terminal output

Example prompt:

@codebase Where is the database connection pool initialized, and how can I increase its max size?

Edit Mode

Select a block of code and press Cmd+I (macOS) or Ctrl+I (Windows/Linux) to open the inline edit dialog. Describe the change you want, and Continue.dev will propose a diff you can accept or reject.

For instance, select this JavaScript function:

function fetchUsers() {
  return fetch('/api/users').then(r => r.json());
}

Then type in the edit prompt:

Add error handling, a timeout of 5 seconds, and return an empty array on failure.

Continue.dev will generate a revised version you can review before applying.

Custom Slash Commands

You can define reusable prompts in your config. Add a customCommands section:

{
  "customCommands": [
    {
      "name": "tests",
      "description": "Generate unit tests for the selected code",
      "prompt": "Write comprehensive unit tests using pytest for the following code. Include edge cases:\n\n{{{ selectedCode }}}"
    },
    {
      "name": "review",
      "description": "Review selected code for bugs and improvements",
      "prompt": "Review the following code for bugs, security issues, and style problems. Be concise:\n\n{{{ selectedCode }}}"
    }
  ]
}

Now you can type /tests or /review in the chat panel with code selected.

Best Practices for Local AI Code Completion

Choose the Right Model Size

Model size directly affects speed and quality. A common strategy is to use two models: a small one (1.5B–3B) for instant tab completions and a larger one (7B–16B) for chat and edits. This balances responsiveness with reasoning depth.

Provide Rich Context

Local models have smaller context windows than frontier cloud models. Help them by writing clear type annotations, docstrings, and meaningful variable names. The more signal in your surrounding code, the better the suggestions.

Index Your Codebase

Run the @codebase indexing once per project. This builds a local vector store using your embeddings model, enabling Continue.dev to retrieve relevant files when you ask questions. Re-index when significant changes occur.

Quantization Matters

Ollama uses 4-bit quantization by default, which dramatically reduces memory usage with minimal quality loss. If you have abundant VRAM, you can pull less aggressive quantizations for slightly better output quality.

Keep Models Updated

The open-source coding model landscape evolves rapidly. Check for new releases monthly:

# Update an existing model
ollama pull qwen2.5-coder:7b

# List installed models
ollama list

Use a Dedicated GPU When Possible

While CPU inference works, a dedicated NVIDIA GPU with CUDA or an Apple Silicon Mac with Metal acceleration provides 5–20x speedups. For a 7B model, 8GB of VRAM is usually sufficient.

Combine with Cloud Models Selectively

Continue.dev supports multiple providers simultaneously. You can keep local models as your default and add a cloud model (such as Claude or GPT-4) for particularly complex refactors. Add another entry to the models array with the appropriate provider and API key.

Troubleshooting Common Issues

Completions Are Slow

Switch to a smaller autocomplete model. A 1.5B model on a modern laptop typically completes in under 200 milliseconds. Also ensure Ollama is using GPU acceleration by checking ollama ps — the PROCESSOR column should show GPU rather than CPU.

Suggestions Are Low Quality

Try a different model. Qwen2.5-Coder and DeepSeek-Coder currently lead open-source benchmarks for code. Also verify your contextLength setting is not too small, which truncates useful surrounding code.

Ollama Connection Errors

Make sure the Ollama service is running. On Linux you may need to start it manually:

systemctl start ollama
# Or run in foreground for debugging
ollama serve

Confirm the API is reachable:

curl http://localhost:11434/api/version

Conclusion

Continue.dev combined with a local model runtime like Ollama gives you a powerful, private, and cost-effective AI coding assistant that rivals commercial offerings. By carefully selecting models for autocomplete versus chat, providing rich context, and indexing your codebase, you can build a workflow that accelerates development without compromising data sovereignty. The open-source ecosystem around local coding models is improving rapidly, and Continue.dev positions you to take full advantage of it today. Install it, configure it with the examples above, and start coding alongside your own offline AI pair programmer.