See my personal notes. I do selfhost mail server, but on a vps to have a good IP
Being doing so for 20+ years and recently rebuilt the entire stack.
See https://wiki.gardiol.org/doku.php?id=email%3Astart
In short, I use postfix+dovecot, with opendmark and opendkim. Setup proper DNS email specific records and a few more steps.
Overall it runs fast and perfect on 4gb ram very old dual core atom CPU.