The quoted numbers are the interesting part. 380 GB just for weights plus several GB of context per user means this only pays off at Revolut’s scale. What stood out to me is that the article credits the biggest gains to NVFP4 quantization, splitting prefill and decode, and context tuning rather than adding GPUs. Those tricks carry over to much smaller setups too.

ARM is a non-issue these days for the usual self-hosted stuff; most popular images ship arm64 builds (check the tags on Docker Hub before you commit to one). For a box at your mum’s, I’d honestly consider staying on macOS + Docker or OrbStack rather than Asahi, purely because remote fixes are easier when the OS is the boring one. Add Tailscale so you can reach it without opening ports. 8 GB is plenty for things like a photo backup, an ad blocker, or a small search engine like SearXNG; it’s tight only if you try to run local LLMs on it.

What’s stuck for me after trying too many things:

• Ollama (or llama.cpp server if you want control over flags) as the model server, bound to localhost only
• Open WebUI as the front-end; it also handles OpenAI-compatible APIs, so local and hosted models live in the same history
• LiteLLM in between once you have more than one backend: one endpoint, per-key budgets, and logs
• SearXNG as the search tool for anything agent-ish; keep concurrency low or upstream engines start blocking it

Biggest lesson: RAM decides what you can run, and document upload/embeddings in Open WebUI eat more memory than chat does. On small boxes, point embeddings at an API or give it 4 GB.