# Run AI Chat Assistant Completely Offline On Your Own Device ⭐

**URL:** <https://onehack.st/t/run-ai-chat-assistant-completely-offline-on-your-own-device/308241>\
**Category:** Tutorials & Methods\
**Tags:** programming, privacy, ai\
**Created:** [July 10, 2025, 9:28pm UTC](https://onehack.st/t/run-ai-chat-assistant-completely-offline-on-your-own-device/308241 "2025-07-10T21:28:57Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aina](https://onehack.st/user_avatar/onehack.st/aina/32/167469_2.png) [@Aina](https://onehack.st/u/Aina)\
**Post date:** [July 10, 2025, 9:28pm UTC](https://onehack.st/t/run-ai-chat-assistant-completely-offline-on-your-own-device/308241/1 "2025-07-10T21:28:57Z")

</div>

Run a Local AI Chat Assistant Completely Offline (No Internet Needed) ⭐

**Unlock the full power of AI chat assistants** by running them locally—completely offline, with no need for cloud services like OpenAI or Claude. This guide walks you through setting up an offline AI assistant using **Open WebUI** and **Ollama** , ensuring privacy, full control, and freedom from internet dependency.

 ![image](https://onehack.st/uploads/default/original/3X/5/8/58f993663a6856756c76cef8571d070dd24041c9.jpeg)

* * *

### 🧠 **What You’ll Set Up**

- **Open WebUI** – A sleek web interface for LLMs
- **Ollama** – Local LLM model runner (like llama3.2, llama3.2-vision)
- **Docker** – Containerized environment to isolate your setup

* * *

### 🔧 **System Requirements**

**🖥 Hardware:**

- CPU: 4+ core processor
- RAM: Minimum **8GB** , ideally **16GB+**
- Disk: 10GB+ free space + space per model
- GPU (optional): NVIDIA CUDA / AMD ROCm for faster inference

**🧰 Software:**

- OS: Linux (Ubuntu 20.04+), macOS (M1/M2+), or Windows 10/11 (WSL2)
- Docker: [https://www.docker.com/](https://www.docker.com/)
- Ollama: [https://ollama.com/](https://ollama.com/)
- Browser: Chrome, Firefox, Safari, Edge

* * *

### ⚙ **Setup Instructions**

**1. Install Dependencies**  
Install both Docker and Ollama, then verify Ollama is working:

```bash
curl http://localhost:11434/api/version

```

**2. Run Open WebUI via Docker:**

```bash
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

```

Access it at: `http://localhost:3000`

* * *

### 👤 **Create Admin Account**

Once WebUI loads, set up your admin profile. Enter your email, name, and password.

* * *

### 🧩 **Download and Use a Model**

You’ll need to pull a model before chatting:

```bash
ollama pull llama3.2

```

OR via WebUI:

1. Click “Select a model”
2. Search and pull “llama3.2” or “llama3.2-vision”

Once downloaded, select the model to activate your assistant.

* * *

### 🔍 **Choosing the Right Model**

| Model | Size | Best Use | Notes |
| --- | --- | --- | --- |
| **llama3.2** | ~4GB | Chat, code, writing | Text-only |
| **llama3.2-vision** | ~8GB | Image + Text | Needs more RAM/VRAM |

* * *

### 💬 **Interact with the Chatbot**

Start chatting! Ask questions, solve problems, or write content—all locally.

**Example prompts:**

- “Explain quantum computing in simple terms.”
- “What is the capital of Indonesia?”
- “Summarize this paragraph.”

* * *

### 🛠 **Troubleshooting Tips**

- Docker container not starting?

- Ollama not connecting?

* * *

### 🚀 **Advanced Features**

**📚 Use Your Own Documents (RAG)**

Add PDFs, DOCX, or MD files to your local KB and configure:

```json
{
  "rag_enabled": true,
  "chunk_size": 500,
  "chunk_overlap": 50,
  "document_lang": "en"
}

```

**🌐 Web Search with SearXNG**

Run this command:

```bash
docker run -d --name searxng -p 8080:8080 \
  -v searxng-data:/etc/searxng searxng/searxng

```

Then enable and configure in Open WebUI settings.

**🔐 Role-Based Access Control**

Assign user roles:

| Role | Permissions |
| --- | --- |
| Admin | Full system access |
| Power User | Model & RAG mgmt |
| Basic User | Chat only |

* * *

### 🖼 **Multimodal Input Support**

Use **llama3.2-vision** for image-based queries.

```bash
ollama pull llama3.2-vision

```

Upload an image and prompt:  
“What’s happening in this photo?”

Open WebUI will return both image and text analysis.

* * *

### ✅ **Conclusion**

With **Open WebUI + Ollama** , you can:

- Use **LLMs offline**
- Ensure **privacy** and **data control**
- Explore **custom AI applications**

* * *

### 🌐 **Resources & Official Links**

1. [Open WebUI](https://openwebui.com/)
2. [Ollama](https://ollama.com/)
3. [Docker](https://www.docker.com/)
4. [SearXNG](https://github.com/searxng/searxng)
5. [Ubuntu](https://ubuntu.com/)
6. [Windows WSL2 Setup](https://learn.microsoft.com/en-us/windows/wsl/install)
7. [macOS Homebrew](https://brew.sh/)
8. [NVIDIA CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)
9. [AMD ROCm](https://rocm.docs.amd.com/en/latest/)
10. [Openverse Image Source](https://openverse.org/image/a9e0bff3-3272-4c3a-bbd4-92e542e1f41a)
11. [GitHub – open-webui/open-webui](https://github.com/open-webui/open-webui)
12. [GitHub – ollama/ollama](https://github.com/ollama/ollama)
13. [curl Documentation](https://curl.se/docs/)
14. [Systemd Docs](https://www.freedesktop.org/software/systemd/man/systemctl.html)
15. [Visual Studio Code (for developers)](https://code.visualstudio.com/)
16. [JSON Validator](https://jsonlint.com/)
17. [Markdown Guide](https://www.markdownguide.org/)

* * *

This method offers a full-stack AI assistant experience without ever connecting to the cloud—ideal for developers, researchers, and privacy-conscious users.

## ENJOY & HAPPY LEARNING! ❤