AI is like starting from the cake and working back to confirm the eggs, the milk, and the flour.

that analogy does not make any sense.

I mentioned information acquisition and thinking are typically not separate processes, but if we act as if they are hypothetically: The cake is the final synthesis of your various information combined with the thought to understand them. LLMs give you output that may look like cake - it follows grammar, structure, and mimics thought. The output is also always a statistically correlative combination of various sources in petabytes of training data - Wikipedia, some random Reddit comment, a conspiracy website, a doctor's blog.

When you get that LLM's "cake," you can decide to eat it, and also decide if it tastes right. You can pick it apart and look at the structure a bit. But it takes a lot of effort to confirm what went into it after it's baked. You may be able to taste when it's made with rat poison, but some poisons taste like food. Was it made with sugar syrup or ethylene glycol? An LLM is just looking at probabilities and associations, and is limited by training data which you (and even the LLM company) cannot typically even reliably trace - those citations it gives you often do not even contain the propositions they're cited for.

Starting from reliable sources in the beginning is akin to starting with sugar, flour, milk. You've verified them, and now you know the cake is cake.