It happens. Remember the original meaning of the word hacker, versus its public perception? You can hang on to the original meaning and have an extra hurdle in communication, or you can accept it and move on by being more specific.
At any rate, I think public/private keys are already being used by Lemmy for authentication on the way you propose.