How close are we to local LLMs being useful? What's the impact?

Question

Feels to me like local models are an under-covered aspect of this whole AI boom.If everything improves over time, at some point a good chunk of tasks won&rsquo;t need to be done in data centers or be subject to the whims of a few frontier AI labs.How close are we to that? Or is my thinking flawed?

david927 · Accepted Answer

I think we're past that point; they're absolutely useful already for a lot of tasks. I think it's about costs, convenience, and benefits of a frontier model for what you're doing.

PaulHoule · Answer

I do classification with SLMs and for my tasks when I have a few thousand samples the frontier models in zero-shot and few-shot modes are embarassingly bad in comparison.

buffer_overlord · Answer

until its cheaper to train and infer than 100k gpu data centers...i doubt it will ever compete.