Wrote a longer reply to someone else, but briefly, yes, you are correct. Kinda.

Caches won’t help with bandwidth-bound compute (read: ”AI”) it the streamed dataset is significantly larger than the cache. A cache will only speed up repeated access to a limited set of data.