It's glorified autocorrect (/predictive text).

People fight me on this every time I say it but it's literally doing the same thing just with much further lookbehind.

In fact, there's probably a paper to be written about how LLMs are just lossily compressed Markov chains.