Why LLM bad at memory/follow instruction?
why if we test different models to role play a character, some of them can do it really good with shorter system prompt but the rest are still bad even with longer prompt?
The same reason they hallucinate: ist in their design. When they are trained the data goes through it like a straw. Only a little or nothing stays.