Norwegian reference for local AI — registers, plans and honest tests.
Short, practical advice for local AI. The column grows — each tip is a card with tags and a date.
Last updated: · 7 tips
Start with the program that actually runs models: LM Studio if you want to click and chat, Ollama if you want commands and a local API. Only when something runs stably is it worth comparing model families. Otherwise you waste time on downloads you never get to test.
Local AI fits when private data, offline use, low latency or many repeated runs matter more than maximum reasoning. The cloud can still win on deep research and hard logic. The useful question is whether the local model is good enough for this exact task.
Think of the stack as the model file, the place you find it (typically Hugging Face), the runtime (LM Studio or Ollama), and the workflow around it — folder, prompt, checklist and output. Under the hood llama.cpp shows up too, and MLX on Apple Silicon.
On Hugging Face: purpose, size, license, hardware people run it on, which modalities it supports, and whether quantized files exist (often GGUF). Ignore the rest until you need it. The license often decides whether you can use the model commercially.
Rule of thumb: small models (~2–4B) for older laptops; mid-range (~7–12B) around 16 GB; larger (~26–35B) need more RAM or VRAM. Q4 is a sensible first choice to fit. Q8 keeps more quality but needs more memory. Use the memory filter for rough estimates.
Before fine-tuning: pick a boring, repeated job. Put ten real or made-up sample files in one folder. Ask the model to produce one reusable file (memo, checklist, report). Run it ten times, tighten the prompt, and make a simple eval. Chat answers are nice; an artefact you reuse changes the workflow.
Let a local model read private drafts, flag problems and clean them before any cloud round. Use the cloud when the answer needs heavier reasoning. Let a human approve what matters before it is sent. Keep local context windows narrow — large context costs memory.
Inspired by two episodes of The Startup Ideas Podcast. One episode was sponsored by Google. We recommend no vendor.