Best Mini PCs for Local LLMs

Something genuinely unusual happened to mini PCs in 2025. A category that had spent a decade being “adequate for office work” acquired machines that run 70-billion-parameter language models — models that no consumer graphics card, at any price, can load.

The reason is unified memory. Instead of a small, fast pool of VRAM separate from system RAM, these machines have one large pool shared between CPU and GPU. Capacity stops being the constraint. Bandwidth becomes it.

This guide is about which of those machines are worth buying, and — just as importantly — when a mini PC is the wrong answer.

Why a mini PC can run what a graphics card cannot

The arithmetic is worth stating plainly, because it is counter-intuitive.

A 70B model at 4-bit quantisation needs roughly 42 GB for weights, plus several gigabytes of context. An RTX 5090 — the fastest consumer graphics card available — has 32 GB. It cannot load the model.

A Framework Desktop with 128 GB of unified memory can allocate up to 96 GB to the GPU. It loads the model with room to spare.

The price of that is speed. Full detail is in How Much VRAM Do You Need for Local AI?, but the short version:

theoretical tokens/second ≈ memory bandwidth ÷ model size in memory

Six tokens per second is roughly reading pace. Whether that is acceptable depends entirely on what you are doing with it — excellent for batch summarisation, poor for interactive coding.

The specifications

The recommendations

Best overall

The Framework Desktop is the mini PC we recommend most often for local AI, for a combination of reasons that no single specification captures.

The silicon — AMD’s Ryzen AI Max+ 395 — gives you sixteen Zen 5 cores, a 40-compute-unit RDNA 3.5 GPU, and up to 96 GB of that memory addressable by the GPU. That is enough for a 70B model at 4-bit with context to spare.

What separates it from machines using identical silicon is everything around it: Framework publishes repair documentation, sells replacement parts individually, and has an actual track record of supporting hardware years after release. On a machine whose memory is soldered and therefore permanent, longevity of support is not a soft factor.

5 GbE networking is a genuinely useful detail if the machine is going to serve models to other devices on your network.

Buy the 128 GB configuration. Memory is soldered; there is no upgrade path, and 64 GB puts 70B out of reach permanently.

Best value on the same silicon

Identical silicon, conventional mini-PC chassis, usually less money.

If the specification is what you are buying and you do not need Framework’s repairability commitment, this is the pragmatic choice. Wi-Fi 7 is a nice addition; the drop to 2.5 GbE networking is the main functional difference and matters only if you are serving models across a network.

The trade is support and firmware track record, which is shorter and less proven. For a machine you expect to run for five years, that is worth weighing rather than dismissing.

Best for CUDA development

If your production target is NVIDIA hardware, this is a different kind of purchase.

128 GB of CUDA-addressable memory at around 240 W, running the same stack as datacentre Blackwell. You develop against a 70B-class model locally and deploy without an architecture change. Two units link over 200 GbE to pool memory for larger models.

At 273 GB/s it is a development machine, not a serving one. Priced well above the AMD alternatives, and for the specific use case of “build against the stack you will ship on” it is the only thing that does the job.

Best for smaller models

Not every local AI machine needs to run 70B.

Twelve Zen 5 cores in a chassis that idles around 10 W makes an excellent quiet desktop that also runs a 14B model when you want one. At 120 GB/s the shared memory pool is a fraction of Strix Halo bandwidth, so treat model running as a capability rather than the purpose.

If your work is mostly development with occasional model use, this is a better-balanced machine than a Strix Halo box you would be paying a large premium to under-use.

Best homelab node

Included here for contrast, because it is the mini PC that is not an AI machine.

No discrete GPU, integrated graphics that are not a serious accelerator, and a half-height PCIe slot. What it has instead is two 10 GbE SFP+ ports, three NVMe slots and up to 96 GB of DDR5 in a one-litre chassis — which makes it an outstanding virtualisation and services node.

If your plan is to run models and everything else on one small machine, this is the wrong half of that plan. Pair it with something that has memory bandwidth. More on this in Best Mini PCs for Homelabs.

What to look for

Memory capacity, and that it is soldered. Every Strix Halo machine has soldered memory. The configuration you buy is permanent. 128 GB is the difference between reaching 70B and not.

Memory bandwidth, quoted honestly. 256 GB/s for Strix Halo, 546 GB/s for an M4 Max, 819 GB/s for an M3 Ultra. This number sets your token rate and nothing else changes it.

How much memory the GPU can actually address. Total installed memory is not the same as the GPU allocation. On Strix Halo under Windows the practical ceiling is around 96 GB of 128; Linux allows more flexibility. On macOS roughly 70% of total is available by default.

Thermal behaviour under sustained load. Inference is a constant load, not a bursty one. A machine that boosts well for thirty seconds and then throttles will disappoint. This is the specification we can least assess from a data sheet, and where independent testing matters most.

Networking, if the machine will serve. If other devices will query models on this box, 2.5 GbE versus 5 GbE is a real difference when moving large payloads.

When a mini PC is the wrong answer

If your models fit in 32 GB. A discrete GPU will run them several times faster for similar money. Buy the graphics card.

If you do image generation. Diffusion models are compute-bound rather than bandwidth-bound, and they fit comfortably in 16 GB. Integrated graphics are not competitive here, and the ROCm ecosystem gap is at its widest in image workflows.

If you want to fine-tune. Fine-tuning needs both memory and compute, and CUDA tooling dominates. A unified-memory machine will do it slowly, if at all.

Continue your research

• Best AI Workstations — the full spectrum, including discrete-GPU machines
• How Much VRAM Do You Need for Local AI? — size your requirement first
• Best Mini PCs for Homelabs — when the workload is services rather than models
• Best GPUs for Local AI — the faster path, if your models fit