← Back to DevBytes
Developer Tutorial August 19, 2026 8 min read

How to Build a CLI Coding Assistant with Ollama

How to Build a CLI Coding Assistant with Ollama

Local AI coding assistants are transforming how developers work. Instead of sending proprietary code to cloud-based LLMs, you can run a capable model entirely on your own machine. Ollama makes this remarkably easy by bundling model weights, inference engines, and a clean API into a single tool. In this tutorial, you'll build a fully functional CLI coding assistant that can answer questions, explain code, generate snippets, and refactor files — all running locally.

What Is Ollama?

Ollama is an open-source runtime for running large language models locally. It abstracts away the complexity of quantization, GPU acceleration, and model loading. With a single command, you can pull models like Llama 3, CodeQwen, DeepSeek Coder, or Mistral and interact with them through a REST API or CLI. For developers, the most compelling feature is the /api/chat endpoint, which exposes a streaming chat interface that mirrors the OpenAI API style.

Why Build a Local Coding Assistant?

Prerequisites

Before you start, make sure you have the following installed:

After installing Ollama, pull the model:

ollama pull qwen2.5-coder:7b

Verify the server is running:

ollama serve

By default, Ollama listens on http://localhost:11434. You can test it with curl:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen2.5-coder:7b",
  "messages": [{"role": "user", "content": "Say hello"}],
  "stream": false
}'

Project Setup

Create a new directory and initialize a Node.js project:

mkdir ollama-coder
cd ollama-coder
npm init -y
npm install commander chalk ora node-fetch

We'll use commander for argument parsing, chalk for colored output, ora for spinners, and node-fetch for HTTP requests. Create the main file:

touch index.js

Building the Core Client

First, create a module that wraps the Ollama chat API and handles streaming responses. Create a file named ollamaClient.js:

const OLLAMA_URL = process.env.OLLAMA_URL || "http://localhost:11434";

async function chatStream({ model, messages, onToken }) {
  const res = await fetch(`${OLLAMA_URL}/api/chat`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ model, messages, stream: true }),
  });

  if (!res.ok) {
    throw new Error(`Ollama error: ${res.status} ${res.statusText}`);
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";
  let fullText = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop();

    for (const line of lines) {
      if (!line.trim()) continue;
      const json = JSON.parse(line);
      const token = json.message?.content || "";
      if (token) {
        fullText += token;
        onToken?.(token);
      }
    }
  }

  return fullText;
}

module.exports = { chatStream };

This function reads the newline-delimited JSON stream that Ollama returns, parses each chunk, and invokes a callback for every token. This gives users a real-time typing effect in the terminal.

Defining the System Prompt

A coding assistant is only as good as its system prompt. Create prompts.js:

const SYSTEM_PROMPT = `You are an expert programming assistant embedded in a developer's terminal.
You provide concise, correct, and practical answers about software engineering.

Guidelines:
- Answer in the same language the user writes in.
- When showing code, always use fenced code blocks with the correct language tag.
- Prefer modern, idiomatic solutions.
- If asked to refactor or explain a file, reference specific lines when helpful.
- If a request is ambiguous, make a reasonable assumption and state it.
- Never invent APIs or libraries that do not exist.`;

function buildMessages(userInput, fileContext = null) {
  const messages = [{ role: "system", content: SYSTEM_PROMPT }];

  if (fileContext) {
    messages.push({
      role: "user",
      content: `Here is the content of a file named \`${fileContext.name}\`:\n\n\`\`\`${fileContext.lang}\n${fileContext.content}\n\`\`\``,
    });
    messages.push({
      role: "assistant",
      content: "Understood. I have the file context. What would you like me to do with it?",
    });
  }

  messages.push({ role: "user", content: userInput });
  return messages;
}

module.exports = { SYSTEM_PROMPT, buildMessages };

Notice how file context is injected as a prior user-assistant exchange. This pattern keeps the actual question clean while still giving the model full visibility into the file.

Building the CLI Interface

Now wire everything together in index.js. We'll support several commands: an interactive chat, a one-shot question, and a file-aware mode.

#!/usr/bin/env node

const fs = require("fs");
const path = require("path");
const { Command } = require("commander");
const chalk = require("chalk");
const ora = require("ora");
const { chatStream } = require("./ollamaClient");
const { buildMessages } = require("./prompts");

const DEFAULT_MODEL = process.env.OLLAMA_MODEL || "qwen2.5-coder:7b";

const program = new Command();

program
  .name("ollama-coder")
  .description("A local CLI coding assistant powered by Ollama")
  .version("1.0.0");

program
  .argument("[prompt]", "One-shot question for the assistant")
  .option("-m, --model ", "Ollama model to use", DEFAULT_MODEL)
  .option("-f, --file ", "Include a file as context")
  .option("-i, --interactive", "Start an interactive chat session")
  .action(async (prompt, options) => {
    if (options.interactive) {
      await runInteractive(options);
    } else if (prompt) {
      await runOneShot(prompt, options);
    } else {
      program.help();
    }
  });

async function loadFileContext(filePath) {
  if (!filePath) return null;
  const abs = path.resolve(filePath);
  const content = fs.readFileSync(abs, "utf-8");
  const ext = path.extname(filePath).slice(1) || "text";
  return { name: path.basename(filePath), lang: ext, content };
}

async function runOneShot(prompt, options) {
  const fileContext = await loadFileContext(options.file);
  const messages = buildMessages(prompt, fileContext);

  const spinner = ora("Thinking...").start();
  let firstToken = true;

  try {
    await chatStream({
      model: options.model,
      messages,
      onToken: (token) => {
        if (firstToken) {
          spinner.stop();
          firstToken = false;
        }
        process.stdout.write(token);
      },
    });
    if (!firstToken) process.stdout.write("\n");
  } catch (err) {
    spinner.fail(chalk.red(err.message));
    process.exit(1);
  }
}

async function runInteractive(options) {
  const readline = require("readline");
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  let fileContext = await loadFileContext(options.file);
  const history = [];

  console.log(chalk.cyan("ollama-coder interactive mode. Type 'exit' to quit."));
  console.log(chalk.gray(`Model: ${options.model}`));
  if (fileContext) {
    console.log(chalk.gray(`Context file: ${fileContext.name}`));
  }
  console.log();

  const ask = () => {
    rl.question(chalk.green("> "), async (input) => {
      const trimmed = input.trim();
      if (!trimmed || trimmed === "exit") {
        rl.close();
        return;
      }

      if (trimmed.startsWith("/file ")) {
        const newPath = trimmed.slice(6).trim();
        fileContext = await loadFileContext(newPath);
        console.log(chalk.gray(`Loaded ${fileContext.name}`));
        return ask();
      }

      const messages = buildMessages(trimmed, fileContext);
      const spinner = ora("Thinking...").start();
      let firstToken = true;

      try {
        const reply = await chatStream({
          model: options.model,
          messages,
          onToken: (token) => {
            if (firstToken) {
              spinner.stop();
              firstToken = false;
            }
            process.stdout.write(token);
          },
        });
        history.push({ role: "user", content: trimmed });
        history.push({ role: "assistant", content: reply });
        console.log("\n");
      } catch (err) {
        spinner.fail(chalk.red(err.message));
      }
      ask();
    });
  };

  ask();
}

program.parse();

Make the file executable and link it globally:

chmod +x index.js
npm link

Now you can use the assistant from anywhere:

# One-shot question
ollama-coder "How do I debounce a function in JavaScript?"

# With file context
ollama-coder -f src/utils.js "Refactor this to use async/await"

# Interactive session
ollama-coder -i

Adding a Refactor Command

Let's add a dedicated refactor subcommand that reads a file, asks the model to improve it, and optionally writes the result back. Add this before program.parse():

program
  .command("refactor")
  .description("Refactor a file using the assistant")
  .argument("", "Path to the file to refactor")
  .option("-o, --output ", "Write result to a file instead of stdout")
  .option("-m, --model ", "Ollama model to use", DEFAULT_MODEL)
  .option("--write", "Overwrite the original file with the refactored version")
  .action(async (file, options) => {
    const fileContext = await loadFileContext(file);
    const instruction =
      "Refactor the following file for readability, performance, and modern best practices. " +
      "Preserve all existing behavior. Return ONLY the refactored code in a single fenced code block, " +
      "with no additional explanation.";

    const messages = buildMessages(instruction, fileContext);
    const spinner = ora("Refactoring...").start();
    let result = "";

    try {
      result = await chatStream({
        model: options.model,
        messages,
        onToken: (token) => {
          spinner.stop();
          process.stdout.write(token);
        },
      });
    } catch (err) {
      spinner.fail(chalk.red(err.message));
      process.exit(1);
    }

    // Extract code from fenced block
    const match = result.match(/[\w]*\n([\s\S]*?)/);
    const code = match ? match[1].trim() : result.trim();

    if (options.write) {
      fs.writeFileSync(path.resolve(file), code + "\n", "utf-8");
      console.log(chalk.green(`\n\nUpdated ${file}`));
    } else if (options.output) {
      fs.writeFileSync(path.resolve(options.output), code + "\n", "utf-8");
      console.log(chalk.green(`\n\nWrote ${options.output}`));
    }
  });

Usage:

# Preview refactored code
ollama-coder refactor src/legacy.js

# Save to a new file
ollama-coder refactor src/legacy.js -o src/legacy.refactored.js

# Overwrite in place (use with caution!)
ollama-coder refactor src/legacy.js --write

Best Practices

Extending the Assistant

Once the foundation is in place, you can extend it in several directions:

Conclusion

Building a CLI coding assistant with Ollama gives you a private, fast, and infinitely customizable AI pair programmer that lives in your terminal. With just a few hundred lines of JavaScript, you've created a tool that streams responses, understands file context, supports interactive sessions, and can even refactor code in place. The real power comes from extending this foundation — adding retrieval, tool use, and editor integration to match your specific workflow. Because everything runs locally, you can iterate freely without worrying about API costs or data leakage, making this one of the most practical entry points into local AI development.

— Ad —

Google AdSense will appear here after approval

← Back to all articles