NVIDIA GeForce RTX 4090 Review: Brilliant at the Work Language Models Do Not Do

The RTX 4090 is a genuinely excellent graphics card that is difficult to recommend for language models, and the reason is specific enough to be worth stating in one sentence: almost everything it does better than a used RTX 3090 is compute, and language model generation barely uses compute.

That is not a criticism of the card. It is a description of a workload, and it means the 4090’s case rests almost entirely on what else you intend to do with it.

The comparison that decides it

The 4090 has been out of production for some time, so buying one means the used market — which puts it directly against the other 24 GB card on that market.

Read the first two rows together, because they are what a language model actually cares about. Generation is memory-bandwidth-bound: the card reads its weights out of memory to produce each token, capacity decides which models fit, and bandwidth decides how quickly they run.

Both cards hold 24 GB. The 4090 reads it 8% faster. On a 32B model at Q4 that is the difference between roughly 50 and roughly 54 tokens per second — a gap you would struggle to notice in conversation, for a card that costs considerably more.

The +56% shader count is real and it is large. It simply does not appear in that number.

Where the extra silicon does show up

Three workloads, and if any of them is yours the card’s case changes completely.

Image generation. Diffusion is compute-bound, not bandwidth-bound — the Stable Diffusion guide works through why. Here the 56% shader advantage is close to the whole story, and the 4090 is far ahead of a 3090 rather than marginally. It also has FP8 tensor support, which Ampere lacks and which some tooling now uses.

Prompt processing. Prefill — reading everything you pasted before the first token appears — is compute-bound in a way that decode is not. Feed a long document to both cards and the 4090 starts answering appreciably sooner, even though it then streams at almost the same speed.

Fine-tuning. Forward and backward passes over batches are dense arithmetic. The fine-tuning workstation explains why compute stops being irrelevant the moment you are changing weights rather than reading them, and this is the card’s strongest showing against its predecessor.

If you do none of those — if the machine exists to run a chat model locally — you are paying a large premium for capability that will sit idle.

What 24 GB runs

The capacity is the reason to want this card, and it is the same reason to want a 3090.

That 32B row is the whole argument for 24 GB over 16. A 5080 or 5070 Ti stops at the 14B tier; this card reaches the one above it, and the jump in output quality between them is larger than any difference in tokens per second.

The 70B row is the ceiling, and no setting moves it. Reaching that tier means two cards, a 96 GB professional card, or unified memory.

Against the RTX 5090

The 5090 is 78% more memory bandwidth and eight more gigabytes, which are precisely the two specifications that govern language models. It is not a close contest on capability: 32 GB reaches 32B models with far more context headroom, and generation is meaningfully faster rather than marginally.

What the 4090 keeps is a 450 W board power against 575 W, which is the difference between a machine most people can build without thinking about it and one that needs the power and cooling planning the $2,000 architecture describes. It also asks less of the case and the room.

If budget were not a consideration the 5090 is the better card for this work, plainly. The 4090 exists in this conversation because used pricing sometimes says otherwise.

Buying one used

It is the only way to buy one now, so the practical caveats matter more than usual.

The 12VHPWR connector. This generation’s connector has a documented history of failures where it was not fully seated, with consequences beyond an unstable machine. Inspect the connector and the socket for discolouration or melted plastic before buying, seat it firmly until it clicks, and route the cable so it is not bent sharply immediately behind the plug. A native cable from an ATX 3.1 supply is preferable to an adapter.

Cards that have been apart. The 4090’s die has been in demand for conversion into other products, and blower-style rebuilds exist. A card that has been disassembled is not necessarily bad, but it is not the card the manufacturer shipped and it has no warranty path. Ask for photographs of the PCB and the shroud, and be wary of anything with a cooler that does not match the model name.

Thermal pads and paste. A card that has run AI workloads has run hot for long periods, which is harder on it than gaming. Budget for a repaste, and treat the seller’s willingness to answer questions about its history as part of the price.

No warranty, and no replacement path. Production has ended, so a failure is not an RMA — it is going back to the used market. That belongs in the price you are willing to pay.

Power and the rest of the machine

450 W board power, an 850 W supply recommended, and — as with every card of this era — size the supply for transient excursions rather than steady draw. Brief spikes well above rated power are what trip over-current protection on supplies that look adequate on paper.

Physically it is a large card. Three slots is common among third-party designs and lengths past 330 mm are normal, so confirm case clearance with the front fans fitted rather than against the manufacturer’s headline figure.

Displays are the one place the age shows in a small way: DisplayPort 1.4a rather than the 2.1b of the current generation. For most monitors this changes nothing; at very high resolution and refresh together it can.

Who should buy one

Buy it if your work is compute-bound — image generation, fine-tuning, long-context prompt processing — and you want 24 GB without paying 5090 money or building for 575 W. In that case it is a strong card at a sensible price, and the shader advantage over a 3090 is money well spent.

Buy a used 3090 instead if you are running language models and little else. You get the same 24 GB and about 92% of the generation speed, for considerably less. This is the recommendation most readers of this site will land on, and it is the one the arithmetic supports.

Buy a 5090 instead if language models are the point and the budget reaches. Eight more gigabytes and 78% more bandwidth are the two things that matter here, and no amount of shader count on the 4090 closes that.