It sounds like you just want email auth. Also known as passwordless login. Also known as magic link.
Fail2ban is a much more robust solution than automated up whitelisting. You are gonna have so many issues with that. What if someone opens your site from a coffee shop, or their isp changes their IP address with a router restart? You'd have to reauth that client. But then you'd also still be allowing the old ip.
I don't think automated whitelisting is "a thing". Mostly because an IP is not an identity so it's woefully insufficient to authenticate people.
Also brute forcing should be handled by whatever handles auth by rate limiting auth requests by ip.