64GB vs 128GB RAM for Local AI: Which Do You Need?

This question has two completely different answers, and which one applies to you depends on a single fact about your machine: does the GPU have its own memory, or does it share the system’s?

Get that distinction right and the decision is straightforward. Get it wrong and you will either waste money on memory that sits idle, or permanently cap what your machine can run.

First: what system RAM actually does during inference

Worth establishing, because most of the confusion around this question comes from assuming system memory is doing more work than it is.

On a machine with a discrete graphics card, running a model involves three distinct pools:

VRAM holds the weights and the KV cache. Once the model is loaded, this is where inference happens. The KV cache — the growing store of attention keys and values for your conversation — lives here too, not in system RAM. People frequently assume long context spills into system memory. It does not; it competes for VRAM.

System RAM holds the model file on its way to the GPU, and then keeps a copy in the operating system’s page cache. This second part is the interesting one, and it is covered below.

Disk holds the model at rest, and is read once per load.

So during generation, system RAM is doing almost nothing. Which is why adding more of it does not make a model that already fits run any faster.

The one way system RAM does earn its place: the page cache

llama.cpp — and therefore Ollama and LM Studio — maps model files into memory rather than copying them in. The operating system’s page cache then holds those pages, and if there is room it keeps holding them after the model unloads.

The practical consequence: with enough free RAM, switching back to a recently used model is near-instant, because the weights never left memory. Without it, every switch re-reads tens of gigabytes from disk.

If you routinely alternate between three or four models, the difference between 64 GB and 128 GB is the difference between a five-second switch and a thirty-second one. That is a real quality-of-life gain, and it is the strongest everyday argument for extra RAM on a discrete-GPU machine.

Case one: you have a discrete graphics card

If your machine has an RTX 5090, a Radeon, or any card with its own VRAM, then model weights live on the card, not in system RAM.

That makes the answer simple: 64 GB is ample, and 32 GB is usually enough.

When to go to 128 GB anyway

Four legitimate reasons, and none of them is “for AI” in the general sense.

You offload large models to system RAM. If you routinely run models too big for your VRAM, the overflow lives in system memory. A 70B model at 4-bit needs about 42 GB, so 64 GB gets tight and 128 GB gives room. Understand what you are buying, though — the next section explains why offloading is slow regardless of how much RAM you throw at it.

You fine-tune with CPU offloading. This is the strongest technical case, and the least discussed. Training holds far more than weights: gradients match the parameter count, and an optimiser such as AdamW keeps two additional 32-bit states per parameter. For a 7B model that is roughly 14 GB of weights, 14 GB of gradients and around 56 GB of optimiser state.

Frameworks such as DeepSpeed ZeRO-Offload and paged optimisers exist to move that optimiser state into system RAM, so a 24 or 32 GB card can train something it otherwise could not. That state has to fit somewhere, and 56 GB does not fit alongside an operating system in 64 GB. If you intend to fine-tune, this is the reason to buy 128 GB.

You run several things at once. A model server, a vector database, a handful of containers and a development environment add up quickly.

You process large datasets. Data preparation, embedding generation and anything that loads a corpus into memory scale with the corpus, not the model.

If none of those describes you, the money is better spent on VRAM. More VRAM changes what you can run; more system RAM does not.

Why offloading is slow, in numbers

Since “just add system RAM and offload” is the usual suggestion, it is worth seeing the arithmetic that makes it a poor plan.

When part of a model sits in system memory, every forward pass moves data across the PCIe bus:

The offloaded portion is limited by the slowest link in that chain, and it is now in the critical path for every single token. That is roughly 28 times narrower than the card’s own memory.

This is why offloading typically costs five to fifteen times the speed rather than a modest percentage. Extra system RAM lets a bigger model load. It does nothing about the bus it now has to cross.

Case two: you have unified memory

This is a completely different situation.

On Apple silicon, AMD’s Strix Halo, or NVIDIA’s GB10, there is no separate VRAM. CPU and GPU share one pool, and system RAM is model memory.

Here, capacity is the entire specification. And because on every one of these machines the memory is soldered, the decision is permanent.

What each capacity gets you

On a unified-memory machine, buy 128 GB if 70B-class models matter to you. 64 GB will run a 70B model only by starving everything else, with no context headroom, and you cannot change your mind later.

The allocation is not the sticker figure — and it is adjustable

The middle column above is smaller than the first because the operating system reserves a share for itself. What is less widely known is that on every platform, that reservation can be moved.

macOS allocates roughly 70–75% of total memory to the GPU by default. The limit is a kernel parameter:

sudo sysctl iogpu.wired_limit_mb=<value>

Raising it lets a 128 GB Mac dedicate substantially more than the default to a model. Leave the system a genuine working margin — pushing it too far will make the machine swap, which is far worse than a slightly smaller model.

Strix Halo on Linux exposes the same idea through the amdgpu driver’s translation table. The BIOS sets a small dedicated carve-out, and the amdgpu.gttsize kernel parameter governs how much additional system memory the GPU may address. Linux is the more flexible of the two operating systems here.

Strix Halo on Windows is driver-controlled, through AMD’s variable graphics memory setting, and the practical ceiling lands around 96 GB of 128.

The point is that “how much can the GPU actually use” is a configuration decision, not a fixed property — and it is worth knowing before you conclude a machine cannot run something.

The bandwidth caveat

Capacity buys you the ability to load a model. It does not make it fast.

A 128 GB Strix Halo machine at 256 GB/s runs a 70B model at roughly six tokens per second. The same model on a 128 GB Mac Studio M4 Max at 546 GB/s runs at roughly thirteen.

So “should I buy 128 GB?” has a companion question: is the machine’s bandwidth high enough that a 70B model will be pleasant to use? If the honest answer is no, buying the memory to load it may not be money well spent — a 32B model on 64 GB, running two to three times faster, is often the better machine in practice.

The full reasoning is in How Much VRAM Do You Need for Local AI?

Speed, channels and the four-stick trap

Two details that matter more than most people expect, and one that costs real performance.

Memory channels set the ceiling for anything the CPU touches. Dual-channel DDR5-6000 delivers about 96 GB/s. For a machine doing CPU inference or running offloaded layers, that is the hard limit — and it explains why CPU-only inference is slow regardless of core count. A 19 GB model at 96 GB/s ceilings out around five tokens per second before any inefficiency.

Two large modules beat four small ones. This is the trap. Consumer platforms run two DIMMs per channel poorly: populating all four slots on a typical AM5 or LGA1700 board frequently forces the memory controller down from 6,000+ MT/s to somewhere in the 3,600–4,800 range.

So a 128 GB kit built from four 32 GB modules can end up materially slower than a 128 GB kit built from two 64 GB modules. If you are buying 64 GB now and might want 128 GB later, buy two 32 GB modules rather than four 16 GB ones — you keep the upgrade path and the clock speed.

The winner, by scenario

The verdict

With a discrete graphics card, 64 GB is the right answer for almost everyone, and 32 GB is genuinely sufficient if your models fit in VRAM. Go to 128 GB for one of three specific reasons — constant model switching, CPU-offloaded fine-tuning, or heavy concurrent workloads — and not simply because the machine is “for AI”.

With unified memory, buy the most you can afford, because the memory is the model capacity and it is soldered in. If 70B-class models are anywhere in your plans, 128 GB is not a luxury tier; it is the entry requirement.

And whichever you choose, buy it in two modules rather than four. The clock speed you keep is worth more than the slot you leave empty.

Continue your research

• How Much VRAM Do You Need for Local AI? — the underlying memory arithmetic
• Best Mini PCs for Local LLMs — where the unified-memory decision bites hardest
• Best AI Workstations — building around either architecture
• Ollama Hardware Requirements — what a specific runtime needs