Ollama Hardware Requirements: What You Actually Need

Ollama’s great strength is that it hides the complexity of running language models locally. Install it, type ollama run llama3.1, and it works.

That same abstraction is why hardware questions about it get confusing answers. Ollama will not tell you that it has quietly pushed half your model onto the CPU; it will just be slow. This page explains what it needs, what happens when it does not get it, and how to check what is actually going on.

The floor, and the actual requirement

Ollama itself is tiny — a Go binary and a model store. It has essentially no requirements. The models have the requirements, and Ollama’s job is to fit them onto whatever hardware you have.

Its default models are 4-bit quantised, which is why the numbers below are lower than the raw parameter counts suggest. The full arithmetic is in our guide to how much VRAM you need for local AI; the summary for Ollama specifically:

The VRAM column includes context and runtime overhead — it is what you actually need, not just the file size.

Ollama’s own documented minimum

Ollama’s documentation states that you should have at least 8 GB of RAM for 7B models, 16 GB for 13B and 32 GB for 33B models. Those figures are for CPU inference and represent the point below which the model will not load, not the point at which it becomes pleasant to use.

The thing that catches everyone out: partial offload

This is the single most important behaviour to understand.

When a model does not fit entirely in GPU memory, Ollama does not fail. It calculates how many of the model’s layers will fit, loads those onto the GPU, and runs the rest on the CPU. Generation then proceeds at something close to the speed of the slowest part.

The result is a performance cliff that looks like this in practice:

• All layers on GPU — fast. Tens of tokens per second on a modern card.
• A few layers on CPU — noticeably slower, often by half.
• Most layers on CPU — five to fifteen times slower.
• Entirely on CPU — one to five tokens per second for a 7B model, and effectively unusable for anything larger.

Because Ollama does this silently, people conclude their GPU is not being used at all, or that the model is broken. Usually the model is 2 GB too big.

How to check what is actually happening

Run a model, then in another terminal:

ollama ps

The output includes a PROCESSOR column. 100% GPU means everything is resident on the accelerator. Anything like 62%/38% CPU/GPU tells you exactly how much has spilled.

If you are seeing a split you did not intend, the fixes in order of preference are: use a smaller quantisation, reduce the context length, or close whatever else is holding GPU memory — a browser with hardware acceleration can hold a surprising amount.

GPU support by platform

NVIDIA

The best-supported path by a wide margin. Ollama requires compute capability 5.0 or newer, which means Maxwell (GTX 900 series) and anything since. In practice any RTX card works without configuration beyond having a current driver installed.

For AI work the card’s VRAM is what matters, not its gaming position. A 16 GB card two tiers down will run models an 8 GB flagship cannot touch.

AMD

Supported through ROCm on Linux and Windows, covering recent Radeon RX and Radeon PRO cards. It works, and it is more setup than NVIDIA — expect to check the supported-GPU list, and on Linux to deal with ROCm packaging.

Some unsupported cards can be coaxed into working by overriding the reported architecture version, which is a well-known community workaround rather than a supported configuration.

For a fuller treatment of the trade-off, see NVIDIA vs AMD for Local AI.

Apple silicon

Excellent, and the least configuration of any platform. Ollama uses Metal, and because Apple silicon has unified memory, the GPU can address most of the system’s RAM.

That makes memory capacity the whole story on a Mac. A 16 GB machine handles 7B–8B models. A 64 GB machine handles 32B comfortably. A 128 GB machine reaches 70B.

The rule of thumb on macOS is that roughly 70% of total memory is available to the GPU by default, so size accordingly.

Intel and integrated graphics

Integrated GPUs are generally not worth targeting. They share system memory at low bandwidth, and the result is usually slower than running on the CPU cores directly.

The exception is the new generation of high-bandwidth integrated designs — AMD’s Strix Halo in particular — where the integrated GPU has a genuinely wide memory bus and is a serious inference target.

What about the CPU?

For GPU inference the CPU barely matters. It loads the model, handles tokenisation, and gets out of the way. Any modern processor is adequate.

Two situations change that:

CPU-only inference. Here it is entirely a memory-bandwidth problem. Core count helps up to a point — typically 8 to 16 cores — but the memory subsystem saturates and additional cores stop contributing. A dual-channel DDR5 desktop delivers perhaps 80–100 GB/s, which is a small fraction of any discrete GPU. Expect a few tokens per second on a 7B model.

Partial offload. When layers are split, the CPU portion runs at CPU speed, and a faster CPU with faster RAM genuinely helps. This is the one case where memory speed on the motherboard is worth paying for.

System RAM

Even with a capable GPU, system RAM matters more than people expect, because Ollama reads the model file into memory before transferring it to the GPU.

• 16 GB — workable for 7B–8B models with a GPU doing the work.
• 32 GB — the sensible default. Room for a 32B model plus a normal working environment.
• 64 GB — needed if you intend to offload large models, or run several at once.
• 128 GB — only necessary for CPU inference on 70B-class models, or on unified-memory machines where system RAM is the model memory.

We look at where the cutover actually falls in 64GB vs 128GB RAM for Local AI.

Storage

Less demanding than most people assume, in two respects.

Capacity matters more than speed. Models are large — a 70B model at Q4 is around 40 GB — and it is easy to accumulate several hundred gigabytes without noticing. Budget 500 GB to 1 TB if you intend to experiment.

Speed matters only at load time. Model weights are read once, then live in memory. A fast NVMe drive shortens the wait when switching models and does nothing for generation speed. Any decent NVMe drive is fine; a PCIe 5.0 drive buys you a few seconds off the load and nothing else. More on this in Best NVMe SSDs for AI Workloads.

Realistic configurations

Learning and small models — 8 GB VRAM, 16 GB RAM. Runs 7B–8B models fully on the GPU at a comfortable speed. Enough to know whether local AI is useful to you before spending more.

The practical default — 16 GB VRAM, 32 GB RAM. Covers 14B models with room to spare and 32B with a shorter context. This is where most people should land.

Serious local work — 24–32 GB VRAM, 32–64 GB RAM. 32B models fully resident with long context. A real step up in capability for coding and analysis.

70B-class — 128 GB unified memory, or 48 GB across two GPUs. The unified-memory route is cheaper, quieter and slower; the dual-GPU route is faster, louder and hungrier.

Getting more out of what you have

• Drop a quantisation level. Q4 to Q3 saves roughly 20% memory. Try it before buying hardware, and check whether output quality suffers on your actual task.
• Cut the context window. /set parameter num_ctx 4096 frees several gigabytes on a large model.
• Keep models loaded. OLLAMA_KEEP_ALIVE controls how long a model stays resident. Raising it removes the reload delay between prompts.
• Prefer a smaller model at higher precision. A 14B model at Q6 is usually better than a 32B crushed to Q3, and it fits in less memory.
• Check what else is using the GPU. Browsers, video calls and desktop compositors all hold VRAM.

What to read next

• Best GPUs for Ollama — cards ranked by what they let you actually run
• How Much VRAM Do You Need for Local AI? — the full memory arithmetic
• Best Mini PCs for Local LLMs — small machines that punch above their size
• Best AI Workstations — complete systems for serious local work