There are some interesting comments about how other languages do it in other threads.

Here's one more idea - is this person present or not during the conversation

Edit to expand on the idea:
We can have 3 pronouns

• Person participates in the conversation
• Person doesn't participate but is somehow involved with speakers and conversation (like talking about a friend, or a customer that bothered you at work)
• Talking about someone in an abstract sense. Like when writing about a celebrity, or a biography, or describing someone's life (even if a friend)

At the very least in the number, English and its almost non existent verbal conjugations is awful in that regard. When someone says you I need quite some context to know if it is talking to me or to a group of people where I'm included, it is even worse with they.

You can just as easily do without any gendered pronouns, like some languages do already