Local small LLM with retries vs. STOA Frontier, will it work?
I would just like to know, most machines don't have a ton of memory. If I put a super small LLM, and if I were to have unit test, do a loop until it passes this unit test. Is that a real possibility or is it just out of the question?
👤 kvisner Accepted Answer ✓
Depends on how hard the question is.
Simple functions in small code bases, will probably work.
Once you get large code bases and more complex work, you'd have issues with the small LLM having the context it needs to actually solve the issue.