Do you still make yourself believe some local LLM was helpful?

Question

Contrary to what I wrote before, I had a thorough look at my experience and I changed my mind. Now I see it like this:I tested every model available on huggingface.com and none of it made it into any kind of regular use for me. Useless, heavily biased replies even with so-called uncensored models and hallucinations are the main reasons. Nothing beats cloud llms when it comes to quality and unfortunately, if you have privacy sensitive data, you better not rely on AI to deal with it, because local LLMs won't make you happy anytime soon. You will just notice how much time you wasted hoping to achieve something with local LLMs that they won't deliver. I wrote this out of anger that nobody adresses this elephant in the room.

dtagames · Accepted Answer

I agree that it's niche but also typical of us programmers to declare "my way is the only way" even when we're into a niche thing. Look at the number of people who setup home movie streaming servers vs pay for Netflix, for example.To the platform victor go the spoils. I've had the greatest leaps recently with Cursor, which not only is a terrific RAG application but integrates several models. Does anyone want to write and maintain that who is not in that business? No. Hence, platforms.

meristohm · Answer

I've yet to consider any LLM helpful. I'd far rather invest in early-childhood family education to reduce trauma and set us up to enjoy lifelong learning and a sense of purpose rather than school as preparation for making money for bosses. The resource cost of machine learning is an opportunity cost; invest in people and culture and ecological diversity, not things.

caprock · Answer

It's very clear that more computation (and ram) on larger models will produce better results when talking about general model use. This will be true for a while.
Eventually the marginal benefits might plateau in combination with enough optimizations to make local use outweigh any cloud models.
More specific and narrow use cases are a different matter.