From the article.

Free software matters — that’s why you’re writing it, after all. Using Discord partitions your community on either side of a walled garden, with one side that’s willing to use the proprietary Discord client, and one side that isn’t. It sets up users who are passionate about free software — i.e. your most passionate contributors or potential contributors — as second-class citizens.

Maybe you'll take up more of your time answering lazy user's questions than speaking with those that are helpful with solving issues.

Your argument about time is more in favor of Matrix, and even more so in favor of just using your code hosting's issue tracker.