Maybe depends on which server implementation you use, I could be out of date. I heard about Synapse, the "flagship" implementation, being heavier though you of course have the option to use one of the lighter weight implementations. Personally, I don't self host and I am mostly going on second-hand information that I've read from people who do self host.