Home / AI

Artificial Intelligence

AI that runs in your own house, not in the cloud.

You can have a capable conversational assistant without sending a single word to outside servers. Two pieces are enough: Ollama to run the models and Open WebUI to talk to them comfortably from the browser. Here is an overview for starting from scratch.

Why local?

  • Privacy. Your data never leaves the house: no prompts logged on somebody else’s server.
  • No subscriptions. Once the model is downloaded, using it is free and unlimited.
  • It works offline. The assistant is there even when the internet is not.
  • You actually learn. You get hands-on with models, context, quantisation, GPU versus CPU.

The trade-off? A local model is usually smaller than the commercial giants: excellent for a great many tasks, less so for the most extreme reasoning. For most everyday uses, though, it is surprising how little you need.

The two building blocks

The engine

Ollama

Downloads, manages and runs language models locally with a single command. Think of docker, but for LLMs: ollama run llama3 and you are up. It also exposes an HTTP API, so other programs can use it.

Runs on Linux, macOS and Windows · CPU or GPU.

The interface

Open WebUI

An elegant web chat in front of Ollama: history, multiple models, users, document upload (RAG) and search. It is the “ChatGPT-like” experience, but served by your own hardware and under your own control.

Installs in a container · reachable from the browser.

How they fit together

Browser ─▶ Open WebUI ─▶ Ollama ─▶ model (llama3, mistral, qwen…)
            (the chat)    (the engine)  (the “weights” downloaded locally)

Open WebUI does not “think”: it only displays the conversation and forwards the questions to Ollama through the API. Ollama loads the model into memory and generates the answer. All on the same machine (or the same home network) — in my case a dedicated VM on the Proxmox server.

Where to start, in practice

  1. Install Ollama and pull a lightweight model to test the water (a 7–8B, say).
  2. Start Open WebUI in a container and point it at Ollama’s address.
  3. Open the browser, pick the model, and start chatting.
  4. When you need more: try bigger models, upload documents, connect tools.

The next step: AI + home automation

The goal I enjoy most is connecting this local brain to the house: a voice assistant that understands natural-language commands and turns them into actions in Home Assistant — all without shipping my voice to anyone. Intelligence and home automation, both strictly at home.

Are you putting together a local setup and want to compare notes? Write to me.