Why is Generative AI non-deterministic

Question

Given a trained LLM model with fixed weights, why is it that the same prompt yields different responses? Or is it the case that some type of RL takes place?

bigyabai · Accepted Answer

Generative AI typically is deterministic, most inference software includes a random seed to yield different results on each repeated entry.

schoen · Answer

It's deliberately made nondeterministic, partly using something called softmaxhttps://en.wikipedia.org/wiki/Softmax_functionI'd say mainly in order to avoid boring its users.