The thing is, if you don't wish to port-forward on Legacy IP nor open firewall ports for IPv6, then the only other option is to reach out to some sort of relay server. But whether your own VPS fulfils that role or you use Tailscale's DERP/TURN relays, the same class of latency and bandwidth issues will still erupt. So you'd still have to debug them, although you presumably would have better visibility into diagnosing issues with your VPS relay.
Generally speaking, if Tailscale isn't working for you, then all other solutions will increase in complexity, not decrease.
Answering the ultimate question, I operate and secure my SSH server, with port 22 open only for IPv6 inbound. There's no NAT, so I don't have to deal with debugging odd port mangling. The SSH server only allows key auth, and all password attempts are logged and blocked, because there can't be any legitimate traffic like that. On this SSH server would be my mounts for backups.