Agentic Ollama

Discussion & Solutions
1

[HELP] can some one tell how i can make ollama hosted deepseek or qwen model use in vscode to chat and read my local files and generate code for me in python or fullstack program i am try to make? the yt video not tell how i can make it read and write local file. pls pls help some one.

2

You can use Cline to get your stuff done, or use Qwen. Both of them are extensions just install and try them.

3

in vs code open copilot and go to agent or model manager there you can select a list of model providers and select what you want and you are good to go

4

:wrapped_gift: Your Personal Copilot — Free, Private, Runs on YOUR Machine

Local AI that chats, reads your files, writes code, and never phones home.

:world_map: The Full Picture

Everything YouTube skipped: Ollama + DeepSeek/Qwen in VS Code with actual file read/write — from zero to fully working.

:light_bulb: Why This Hits Different

No monthly fees. No data leaving your laptop. No rate limits ever.
Qwen2.5-Coder-32B now ties GPT-4 on benchmarks — running locally.
One-time GPU investment → unlimited AI coding forever.

:white_check_mark: What You Unlock

  • :robot: Chat with AI inside VS Code
  • :open_file_folder: AI reads your entire project (the missing piece)
  • :writing_hand: AI creates and edits files directly
  • :snake: Python, React, FastAPI, SQL — all of it
  • :high_voltage: Autocomplete that keeps up with Copilot
  • :locked: Offline-capable, fully private

@vinhson asked:
“[HELP] can some one tell how i can make ollama hosted deepseek or qwen model use in vscode to chat and read my local files and generate code for me in python or fullstack program i am try to make? the yt video not tell how i can make it read and write local file. pls pls help some one.”

Got you. Here’s the complete answer. :backhand_index_pointing_down:

:hammer_and_wrench: FULL SETUP — Zero to Working

STEP 1 → Install Ollama

The engine that runs AI models locally.

🍎 macOS
  1. https://ollama.com/download → download Mac version
  2. Drag to Applications
  3. Terminal: ollama --version
  4. Version shows? Done.
🪟 Windows
  1. https://ollama.com/download → run the .exe
  2. PowerShell: ollama --version
  3. Version shows? Done.
🐧 Linux 
curl -fsSL https://ollama.com/install.sh | sh

Auto-starts on boot. Done.

STEP 2 → Grab a Coding Model

Pick one based on your GPU:

GPU Memory Model Command
8GB Qwen 7B ollama pull qwen2.5-coder:7b
12GB DeepSeek V2 ollama pull deepseek-coder-v2
24GB Qwen 32B ollama pull qwen2.5-coder:32b
No GPU Qwen 1.5B ollama pull qwen2.5-coder:1.5b
🤔 How to check GPU memory
  • Windows: Task Manager → Performance → GPU
  • Mac: Apple menu → About This Mac
  • Linux: nvidia-smi

STEP 3 → Install the VS Code Extension

Two options. Pick based on what you need:

:a_button_blood_type: Continue — Chat + Edit + Autocomplete

Best for: asking questions, inline edits, code completion

  1. VS Code → Extensions (Ctrl+Shift+X)
  2. Search “Continue” → install (by Continue.dev)
  3. Sidebar → Continue icon
  4. Auto-detects your Ollama models :white_check_mark:

Usage:

  • Ctrl+L → open chat
  • @file src/app.py → AI reads that file
  • @codebase → AI searches whole project
  • Highlight code + Ctrl+I → inline edit
⚙️ Manual config (if auto-detect fails)

Create ~/.continue/config.yaml:

models:
  - name: Qwen Coder
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - autocomplete

Restart VS Code.

:b_button_blood_type: Cline — Full Agent Mode (Reads AND Writes Files)

This is what you’re looking for. AI that actually creates/edits files on disk.

  1. Extensions → search “Cline” (by saoudrizwan)
  2. Install → click Cline sidebar icon
  3. Settings:
    • API Provider: Ollama
    • Base URL: http://localhost:11434
    • Model: your installed model

Usage: Just describe what you want:

“Create a Flask API with JWT auth and save it to src/api.py”

Cline plans → creates file → writes code → shows diff → you approve.

🔐 Safety permissions

Control what AI can do:

Permission Recommendation
Read project files :white_check_mark: Enable
Edit project files :warning: Enable with approval
Run terminal commands :warning: Enable with approval
Read outside workspace :cross_mark: Keep off

Every edit shows a diff preview first. You approve or reject.

STEP 4 → Enable Codebase Search (The Missing Piece)

This lets AI see your whole project — what tutorials skip.

For Continue:

ollama pull nomic-embed-text

Add to ~/.continue/config.yaml:

models:
  - name: Embeddings
    provider: ollama
    model: nomic-embed-text
    roles:
      - embed

Restart VS Code. Now @codebase where is auth implemented? actually works.

For Cline:

Already built-in. Just ask:

“Look at my src folder and explain the structure”

It reads whatever files it needs.

STEP 5 → Speed Tweaks

⚡ Make responses faster

Windows (PowerShell admin):

[Environment]::SetEnvironmentVariable("OLLAMA_KEEP_ALIVE", "-1", "User")
[Environment]::SetEnvironmentVariable("OLLAMA_FLASH_ATTENTION", "1", "User")

Mac/Linux — add to ~/.bashrc or ~/.zshrc:

export OLLAMA_KEEP_ALIVE=-1
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_NUM_PARALLEL=2
  • KEEP_ALIVE=-1 → model stays loaded
  • FLASH_ATTENTION=1 → up to 3x faster
  • NUM_PARALLEL=2 → handles multiple requests
🧠 Bigger context window (see more files)

Ollama defaults to 4096 tokens. Your models handle way more.

ollama create qwen-32k -f - <<EOF
FROM qwen2.5-coder:7b
PARAMETER num_ctx 32768
EOF

Use qwen-32k in extension settings. DeepSeek-V2 supports 160K tokens — entire codebases in one chat.

:snake: Code Generation Examples

Python/FastAPI:

“Create a FastAPI app with SQLAlchemy User model, JWT auth, CRUD endpoints. Save to src/main.py”

React/Next.js:

“Build a dashboard component with Tailwind, fetches /api/stats, loading states, error handling. TypeScript.”

SQL:

ollama pull sqlcoder

“Tables: users(id, email), orders(id, user_id, total). Query: users with 3+ orders last month.”

:bar_chart: What You Can Run

Setup Model Speed
8GB GPU Qwen 7B ~50 tok/s
12GB GPU DeepSeek V2 / Qwen 14B ~35 tok/s
24GB GPU Qwen 32B ~25 tok/s
CPU only Qwen 1.5B ~8 tok/s
M1/M2/M3 Mac Same tiers ~30 tok/s

:vs_button: Local vs Copilot

Local Ollama GitHub Copilot
Monthly cost $0 $10-39
Privacy 100% local Cloud
Rate limits None Yes
Offline Yes No
Quality (32B) GPT-4 tier GPT-4

$400 GPU pays for itself in ~10 months. Then free forever.

:link: Resources

📚 Official Docs
📖 Tutorials
🔧 Extensions
📊 Benchmarks & Models

:rocket: Quick Start

# 1. Install Ollama (links above)

# 2. Get model
ollama pull qwen2.5-coder:7b

# 3. Get embeddings (for codebase search)
ollama pull nomic-embed-text

# 4. Install VS Code extension
#    → Continue (chat/autocomplete)
#    → Cline (file read/write agent)

# 5. Code with AI

File READ + WRITE → Cline
Chat + Autocomplete → Continue
Both? Install both.

Free. Private. Unlimited. Runs on your hardware. :fire:

5

Hello, I just bought an RTX 5080 16GB VRAM, is it ok to try Qwen 32B or should I stick to DeepSeek V2?

6

  • Go for Qwen 32B if you are doing coding or complex logic. Use a 3.5bpw EXL2 or a GGUF Q3_K_L to keep it entirely on the GPU. The intelligence gain over smaller models is very noticeable.

  • Stick to DeepSeek V2 (or Qwen 14B) if you want instant responses and a large context window (e.g., feeding it long documents).

[!TIP] Since you have a Blackwell-series card (5080), look for GGUF or EXL2 quants specifically. Using LM Studio or Ollama will allow you to easily slide the “GPU Offload” bar to see exactly how many layers you can fit before you run out of VRAM.