Running 180B Models at Home on a 2015 Dell Workstation
What this is: the implementation guide for my local LLM box, the hardware and the serving stack behind it. A retired Dell Precision T7910 workstation got two new CPUs and three used datacenter GPUs, then power, cooling, RAM, and software fights followed. The box now serves a 180B-parameter Qwen hybrid at 524,288 tokens of context across two 262K slots, and a 284B DeepSeek, both through llama.cpp. Environment used here: Windows Server 2022 and Ubuntu on the same T7910, dual Xeon E5-2697A v4, three Tesla V100 32 GB (PG500-216, TCC mode), llama.cpp forks compiled from source against CUDA 12.8. Windows Server is the reference side and where both forks were first built; since 2026-09-08 production serving runs on the Ubuntu side of the dual boot, from .sh launchers under transient systemd units, with logs going to journald instead of a file. Where a figure came from one side and not the other, the text says which. Build window: about nine weeks, mid-June to September 2026. ...