Nah, reality doesn't have a liberal bias. "Liberal" is something that humans invented, and not something that comes from reality or some intrinsic part of nature.

LLMs are trained on past written stuff by humans, and humans for a long time have not been ridiculously right wing as the current political climate of the US.

If you train a model on only right wing propaganda, it will not miraculously turn "liberal", it will be right wing. LLMs also argue not more logical than any propagandist, if they were fed by only propaganda.

I dislike it immensely when people argue that LLMs are truthful, unbiased, or somehow "know" or can create more that what was put into them. And connecting them with fundamental reality seems even more tech-bro-brained.

Arguing that "reality" is this or that is also very annoying, because reality doesn't have any intrisic morales or politics that can be measured by logic or science. So many people argue that their morales are better then someone else's, because they where given by god, or by science, this is bullshit. They are all derived by human society, and the same is true by whatever "liberal" means.

And lastly, assuming that some system somehow is "built into reality" shuts down any critique of the system. And critiquing any system in order to improve it is essential for any improvements, which should be part of any progressive thought.