Nearly all evaluation steps for an LLM I produce some level of entropy. Yes you can evaluate an LLM model deterministically, but we don't. We even tune the levels to get good results. Model cards have sampling parameters supplied to give "the best" results. For example, Qwen 3.8 flash next gives

We recommend using the following sets of sampling parameters for generation:

Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Replaying the seed will only get you the same result using a particular engine and a particular random number generator. Run it on llama.cpp Vs Strata and you can get different results. Different libc, different results.

A compiler is systematically deterministic. No calls to rand()