HACKER Q&A
📣 mempirate

Thoughts on AMD Ryzen AI MAX+ 395 for Local AI?


Wondering if anyone has played around with the Strix Halo based systems with 128GB of unified memory for local AI, and what their experience has been so far in running different models: - Practical limit on number of parameters x quantization - What you've been using it for - Which models and inference engines combine well

  👤 Iolaum Accepted Answer ✓
Own a framework desktop 128gb.

GPU bandwidth limits the amount of tokens/sec that you get.

I 've mostly been running QWEN-3.6-35B-A3B at Q8 and QWEN-3.5-122B-A10B at Q4 with Q8 kv-cache on llama.cpp (use vulkan version). Also use MTP GGUF's you get ~ 50% speed up when predicting 2 or 3 tokens.

Haven't tested the new laguna S 2.1 yet.

Another thing I liked was that I had enough ram to also run an embedding open model for LLM Wiki apps.

Overall I think it's cheaper and better (intelligence wise) to get a $20 codex sub and always run gpt-5.6-luna that get a desktop for local inference. Unless you have explicit needs for it (whether it is just experimentation or you have personal things you 'd rather keep on your computers). OpenCode go at $10 is also a great option.