The non-determinism in LLM inference at temperature zero, as described by Thinking Machines Lab, ..., Sonic AI
“The non-determinism in LLM inference at temperature zero, as described by Thinking Machines Lab, was formalized in Torch Lean to demonstrate how small floating-point arithmetic errors can alter the final argmax output.”