This is a reference architecture, not a tested build — and it is the only one on this site that might conclude you should buy nothing.
The other eleven start from a blank sheet and specify a machine. This one starts from an inventory and diagnoses one, because the constraint it answers to is different: the hardware is already chosen, by somebody solving a different problem. A gaming PC. A work laptop. A machine built five years ago for something unrelated.
That inverts the method. Instead of deciding what to buy, you are finding the one component that is actually stopping you and working out whether relieving it is worth the money — including the case where it is not, and the case where nothing is stopping you at all.
Find the binding constraint first
Almost every machine has exactly one thing gating what it can run, and on almost every machine it is the same thing.
| Component | Does it gate local AI? |
|---|---|
| GPU VRAM capacity | Almost always. This is the answer. |
| GPU memory bandwidth | Decides speed once the model fits |
| System RAM | Only if the model does not fit in VRAM |
| CPU | Essentially never, once the model is on the GPU |
| Storage speed | Load time only, measured in seconds |
| PCIe generation | Barely — see below |
The reason is the arithmetic the VRAM explainer sets out: a model at Q4_K_M occupies roughly 0.6 GB per billion parameters, it has to fit in the card’s memory, and if it does not fit you are not running it — no amount of CPU or system memory changes that.
So the diagnosis is short. Find out how much VRAM you have. Everything else is a second-order question.
What your existing card already runs
| VRAM you have | Runs comfortably at Q4_K_M | Verdict |
|---|---|---|
| 6 GB | 3–4B | Genuinely limiting, but usable for small models |
| 8 GB | 7–8B | Enough to do real work and learn the tooling |
| 12 GB | 13–14B, tight context | A good position; most people stop here happily |
| 16 GB | 13–14B comfortably | The current sensible floor for new purchases |
| 24 GB | 32B | The tier worth upgrading to |
If you have 8 GB or more, you can start today without spending anything. An 8B model at Q4 is a genuinely capable assistant, and a great many people who believe they need new hardware have simply never run one.
That is the honest headline of this page, and it is why it exists.
If the diagnosis is more VRAM
-
Most VRAM per Dollar
NVIDIA GeForce RTX 3090
The upgrade worth making rather than the one that is merely available: 24 GB moves you from 14B to 32B models, which is the largest capability step at this money. Check three PCIe connectors and 340 mm of clearance before ordering.
-
Best Low-Power Option
NVIDIA GeForce RTX 5060 Ti 16GB
The undemanding upgrade for an older machine: 16 GB at 180 W, so a modest existing power supply is likely to cope and the card is short enough for most cases. Slower per token than the alternatives, and it fits where they do not.
-
Best Budget
NVIDIA GeForce RTX 4060 Ti 16GB
The same idea second-hand and cheaper — 16 GB at 165 W. Slow enough that you will notice, and still the difference between running 14B models and not running them.
Four things that can block an upgrade
If the diagnosis is “more VRAM”, the question becomes whether your existing machine can accept it. Four checks, all free, all better done before ordering.
Power supply connectors, not just watts. A 550 W unit with two 8-pin PCIe connectors cannot power a card that needs three, regardless of the wattage on the label. Count the physical cables you have, and prefer native ones over adapters.
Case clearance, with fans fitted. Measure the actual space, not the manufacturer’s headline figure, which is routinely 20–30 mm optimistic once front fans are in place. This is the check people skip and then discover on a Saturday.
Slot spacing and width. Many capable cards are 2.7 or 3 slots thick. Confirm nothing below is blocked and that the case has the slots.
Physical support. A heavy modern card in an older case may need a support bracket. Cheap, and worth having.
The check most people worry about, which does not matter
An old PCIe generation is almost never a problem. Once a model is resident in VRAM, the card barely uses the bus — the dual-GPU machine makes the same point about layer-split inference passing only kilobytes per token.
A PCIe 3.0 ×16 slot on a machine from 2017 will run a modern card at close to full inference speed. It affects how long the model takes to load, measured in seconds, once per session. Do not replace a working motherboard over this.
The exception is a card with half the usual lanes — the RTX 4060 Ti 16GB runs at ×8, and if you intend to offload models larger than its memory, that link carries the traffic continuously. Its review works through when the slot generation starts to matter.
What is not worth upgrading
Money spent here is money that does not go toward VRAM, which is the only thing that changes what the machine can do.
- The CPU. Once the model is on the GPU it feeds and waits. Every other build on this site says the same, and it is most true here.
- More system RAM, if you have a discrete GPU and the model fits in it. 16 GB is fine; 32 GB is comfortable. Going further changes nothing until you are offloading, and if you are offloading, see below.
- A faster NVMe drive. Load time only. A few seconds per session.
- Cooling, unless you are actually throttling. Check temperatures under a long generation before assuming.
On CPU offloading
Tools will run a model larger than your VRAM by keeping some layers in system memory. It works, and it is much slower than people expect — dual-channel DDR5 delivers somewhere near 50–100 GB/s against a discrete card’s 900 GB/s or more, and generation speed tracks the bandwidth of wherever the weights live.
Treat it as a way to try a model once, not as a way to run one daily. The usual right answer is to run a smaller model that fits, which will be several times faster and often good enough.
When a new machine beats an upgrade
Be willing to reach this conclusion, because for some machines the upgrade is the more expensive route.
An older platform can cost more to upgrade than to replace once you add a card, a power supply that can feed it, and a case that fits it. Three purchases to reach a position a single coherent build would have reached for less.
Specific signals that the machine is not worth upgrading:
- The power supply is under 550 W or has no spare PCIe connectors
- The case cannot take a card longer than about 270 mm
- It is a small-form-factor or all-in-one machine with no full-height slot
- It is a laptop — the GPU is not replaceable, and that is the end of the analysis
If two or more of those apply, stop upgrading and read the staged-upgrade PC, which is the plan for buying a chassis once so that this situation does not recur. That article exists largely to prevent this one from being necessary next time.
If you have a Mac
Different arithmetic, and usually a pleasant surprise. Apple silicon shares one memory pool between CPU and GPU, so the relevant number is total unified memory, not a separate VRAM figure — and roughly 60–70% of it is available to the model.
A 16 GB machine runs 7–8B models comfortably. A 32 GB machine reaches 14B and beyond. Nothing is upgradeable after purchase, so the diagnosis is simply what you have — but many Mac owners are in a better position than they assume, and the Mac Studio comparison covers where that stops being true.
Before you spend: verification checklist
The unusual thing about this checklist is that most of it is measurement of what you already have, and the first three items are free.
- Find your exact VRAM figure, from the card’s specification rather than from memory. 8 GB and 12 GB variants of the same model name are common.
- Run a model that fits, today, before buying anything. Pick something at or below the tier in the table above. This single step resolves the question for a large share of readers.
- Time a generation and decide whether the speed is acceptable. If it is, you are finished, and the rest of this page does not apply to you.
- Count the spare PCIe power connectors physically, in the case, with a torch.
- Measure case clearance with the front fans in place.
- Check temperatures under a ten-minute generation before blaming performance on the card.
- Price the full upgrade — card, plus supply if needed, plus case if needed — against a coherent new build. Be willing to find that the new build wins.
- Confirm the machine is not a laptop or small-form-factor unit before planning any of this.
When not to follow this at all
- When you already know what you need. If the target is a 32B model and you have 8 GB, the diagnosis is over — go to the $2,000 machine or best GPUs under $1,000.
- When the machine is a laptop. Nothing here is actionable; the GPU is soldered.
- When you need capability rather than a start. Fine-tuning, serving a team, or 70B-class models are all different architectures, and no upgrade to an existing desktop reaches them.
- When the existing machine is doing a job you need it to keep doing. A machine that games well and now also runs models is a build with two workloads, and that article covers the contention this one does not.
Frequently asked questions
Can my existing PC run local AI?
Probably, and the deciding number is VRAM. With 8 GB you can run a 7–8B model at Q4 comfortably, which is a genuinely capable assistant; 12 GB reaches 13–14B. Find your exact figure — 8 GB and 12 GB versions of the same card name are common — and run something that fits before spending anything. A large share of people asking this question already have enough.
What is the single best upgrade for local AI?
More VRAM, and it is not close. Model capacity is what decides which models you can run at all, and no other component changes that — a faster CPU, more system memory or a quicker drive will not let a model fit that does not fit. If you can only change one thing, change the graphics card, and choose it on memory capacity rather than on gaming benchmarks.
Is my old PCIe 3.0 motherboard holding me back?
Almost certainly not. Once a model is resident in VRAM the card barely uses the bus, so a PCIe 3.0 x16 slot runs a modern card at close to full inference speed. What it affects is model load time, measured in seconds and paid once per session. Do not replace a working motherboard over this — put the money toward memory capacity instead.
Should I add system RAM instead of a new graphics card?
Not if you have a discrete GPU and the model fits in its memory, where system RAM does almost nothing. It only matters when you are offloading layers to the CPU, and offloading is far slower than people expect — system memory delivers roughly 50–100 GB/s against a card’s 900 GB/s or more, and generation speed tracks the bandwidth of wherever the weights live. Running a smaller model that fits is usually several times faster and often good enough.
Can I upgrade the GPU in my laptop?
No. Laptop GPUs are soldered to the mainboard, so the memory you bought is the memory you have permanently. External enclosures over Thunderbolt exist and work, with meaningful cost and complexity. For most people the honest answer is that a laptop sets a ceiling at purchase, and reaching past it means a desktop.
How do I know whether to upgrade or start fresh?
Price the whole upgrade rather than the card alone. If the machine also needs a power supply that can feed the card and a case that can fit it, that is three purchases to reach a position one coherent build would have reached for less. A supply under 550 W with no spare PCIe connectors, a case under about 270 mm of clearance, or a small-form-factor chassis are each a signal — two or more together, and a new build is the cheaper route.
What about my Mac?
Different arithmetic and often a pleasant surprise. Apple silicon shares one memory pool, so what matters is total unified memory rather than a separate VRAM number, and roughly 60–70% of it is available to the model. A 16 GB machine handles 7–8B comfortably and 32 GB reaches 14B and beyond. Nothing is upgradeable afterwards, so the diagnosis is simply what you bought — but many Mac owners are better placed than they assume.
Is a used card a sensible upgrade?
For this purpose, often yes, because capacity has aged more slowly than compute — a used 24 GB card can beat a new 16 GB one for language models despite being two generations older. Accept that there is no warranty, check the power connectors your specific model needs, ask the seller for photographs of the board, and budget for replacing thermal pads on anything from the GDDR6X generation.
Have you tested this on real machines?
No. Like every architecture here it is reasoned from published specifications and from how the workload uses hardware, not from a bench. That matters less than usual on this page, because almost every step is a measurement you take on your own machine — your VRAM figure, your generation speed, your connectors, your clearance. The most useful thing this page can do is get you to run a model before you spend anything.