entropy

“In some sense, it feels better to believe that there’s a secret society controlling everything than believing that entropy and chaos rule.”

The more he fumbled for answers, the more overwhelmed he became by entropy and uncertainty.

The amount of entropy was measured by a second LLM that focused on the meaning and nuance of the generated responses, rather than just the words used.

In this case, while the answers all used different vocabulary, their meanings are roughly similar—earning them a low semantic entropy score, which indicates the model’s response is likely to be reliable.

Responses to the same query that contained vastly different meanings earned high entropy scores, signaling possible confabulations.