Hacker News

Using the Political Compass is a bit of a strange way to conduct research. I do think it is important to identify biases of course, but at some point you have to look at the bigger picture and realise why the bias exists.

In order to swing ChatGPT more to the right (if you want to balance it at neutral in the end), you’d have to inject it with more racism, anti-science conspiracy and American Christian views - none of which are particularly pleasant.

Do we want a LLM that limits facts about COVID-19 so that those who view it as a conspiracy feel validated?

Do we want it to respond that homosexual people don’t exist? Or even to say “I can’t give a response to this that remains politically neutral”?

Or if someone asks how old the earth is, do we want it to reply with “about 3000 years old”?

Or to contest climate change?

Do we want to sacrifice accuracy in favour of neutrality just because one party has a denial stance on these topics?

From the study:

In a nutshell, we ask ChatGPT to answer ideological questions by proposing that, while responding to the questions, it impersonates someone from a given side of the political spectrum.

I'm not sure if I like this method. It's comparing the 'default' response to the response of it 'impersonating' the left and right of the political spectrum (reduction of politics to a spectrum an entirely different issue). You don't actually prove the default is biased doing this. It can just as easily be that the impersonations are more extreme than they should be.

If it impersonates Republicans as more extreme than they really are and the Democrat impersonation and default positions are as they should be, there would seem to be a Democrat bias.

If the impersonated Democrat position was less extreme than it should be and the Republican impersonation and default position are as they should be, you would still see a Democrat bias.