I have $5k to spend on a local AI machine, what should I get?

Question

I would like to run local models as large and as fast as possible for around $5,000 USD. Is an Apple machine the best choice with their shared memory or is there a particular GPU that would be more cost effective? Thanks!

billconan · Accepted Answer

I guess I would buy nvidia digits https://www.nvidia.com/en-us/project-digits/

jotux · Answer

Not precisely what you were looking for, but this was going around yesterday: https://rasim.pro/blog/how-to-install-deepseek-r1-locally-fu...

cameron_b · Answer

Unpopular but highly promising way to go if training is on your mind- 4x 7900 xtx cards and some nuts and bolts to feed them could be a price per GPU memory high point. There are folks using ROCm with that to put up some interesting numbers in terms of wall clock and power required per training run.

monroewalker · Answer

This Reddit comment mentioned this site with used servers:https://pcserverandparts.com/ https://www.reddit.com/r/LocalLLaMA/comments/1i8rujw/comment...

vunderba · Answer

Be more specific - AI is a very broad field.
nVidia GPUs have the best inference speed (particularly around SDXL, Hunyuan, Flux, etc), but unless you're buying several used 3090s SLI style, you're going to have to split larger LLM GGUFs across main memory and GPU. I'm excluding the RTX 5090 since two of them (plus tax) would basically blow your budget out the water.
With Apple I think you can get up to 192GB of shared memory allowing for very large LLMs.
Another thing is your experience. Unless you want to shell out even more money, you'll likely have to build the PC. It's not hard but it's definitely more work than just grabbing a Mac Studio from the nearest Apple Store.

PaulHoule · Answer

I just got a Mac Mini with maximum specs (can't believe how small the box it came in was!) and that's not a bad choice. As you say it has the advantage of handling large models. I think the 5090 will outperform it in terms of FLOPS but it only comes with 32MB compared to the 64MB you can get on an M4 mini. The 5090 itself will be $2000 (if you can get it at that price) compared to the $2500 max mini M4. You'll probably spend at least $1k for the rest of the PC worthy of the 5090 card.

lulznews · Answer

More info needed but for 5k you can get a near maxed mbp that should be able to handle most models, perform pretty well, and also serve as your laptop so it&rsquo;s not a pure AI box. Then if you have heavier needs go to the cloud.

eatenbyagrue · Answer

Here's a nice looking DeepSeek build https://x.com/carrigmat/status/1884244369907278106?mx=2

giardini · Answer

https://www.amazon.com/Yassk-Fortune-Telling-Floating-Answer...AI in the palm of your hand! Best deal evarrr!

mikewarot · Answer

I used to think GPUs were the way to go, but now my goal is to get a used server with a Terabyte of RAM so I can run the full size Deepseek R1

throwaway519 · Answer

Why get an Apple? Even th3 keyboard lacks required keys for development. They're purely tech bro poser machines.