EDITED TO ADD MY SOLUTION : FINALLY! I got it working. Leaving everything here for any future people.
I believe that a large part of the problem stemmed from not using a wildcard cert, for whatever reason? I switched my domain's DNS nameservers from Namecheap to Cloudflare (not overly happy about using Cloudflare but in the short term just happy this is working now). On Cloudflare, I made an API token with the permissions of zone, DNS, and Edit. I used that API token to create a *.domain.tld cert using a DNS challenge in Nginx Proxy Manager, and switched all my proxy hosts to use that wildcard cert. Once I did that, I achieved my goal!
Thank you to all the commenters who took the time to read and offer help!!
ORIGINAL MESSAGE BELOW
I am pulling my hair out and need help. I'm going to try to be as thorough as possible.
The Goal : To use sub.domain.tld to access a service hosted on my local network whether I am on the local network or not, with SSL certs either way.
The Current Situation : I have Unraid running on a home server on Unraid.IP.Address. On that server, I'm running a few services as well as a couple VMs which themselves are running services. I won't get into the details of all of them. I think the most relevant ones right now are DuckDNS, Nginx Proxy Manager, and Adguard Home - all of these run in docker containers on the Unraid host.
The Adguard home service has a static IP at AGH.IP.Address, and my router (an Actiontec T3200M) has been set to use AGH.IP.Address as both DNS Server 1 and DNS Server 2.
I own domain.tld through Namecheap and use their DNS records to point multiple sub.domain.tlds to sub.duckdns.org for dynamic DNS services. These successfully resolve through Nginx Proxy Manager when I'm outside my network to my various services, as well as those I host for some friends. Nginx Proxy Manager has a cert for each sub.domain.tld. I cannot gain access to Namecheap API for the purposes of a wildcard cert via DNS challenge, to my knowledge.
I also have Tailscale setup on the Unraid server. I currently use Tailscale to pretend to be on my local network when I am away to continue accessing my services from the same LAN.IP.Addresses whether I am home or away. This makes it seamless for me and my partner, but it wasn't my ultimate goal (as mentioned in The Goal).
What I've Tried : I have tried to use Adguard Home's DNS rewrites as well as custom query filters to catch local requests for sub.domain.tld and point them instead to Unraid.IP.Address, but this does not resolve. If I try to access sub.domain.tld from within the network with or without DNS rewrite entries, it does not resolve. I've tried using PiHole instead of Adguard Home, but was having difficulty determining if it was working at all as a DNS server, so I switched back to Adguard Home. I've also tried setting up a second Nginx Proxy Manager instance on my network at a different IP address, and tried to have Adguard Home rewrite DNS to that one still with no success.
This has been a thing I've worked on off and on for a few months with no real success so I may be forgetting a few things that I have tried. If they come up in the comments, I will edit this part with additional things I've tried.
I believe I want split DNS to achieve what I'm trying to achieve, but for the life of me I cannot figure out how to accomplish it. Any help would be super appreciated. Of all the things I've learnt on my self-hosting journey—switching to Linux full time, learning some docker and docker compose concepts, some light scripting, learning about VMs and passthrough, and more—networking as by far been the most difficult and head-bashingly difficult aspect of it all. For me, at least.
Does anyone have any suggestions for what my next steps should be to achieve my goal? I am open to any good or bad news. If I need to switch registrars, or change up my set-up radically, whatever it might take, I want to learn and I need direction because my research has hit its end.
Cheers!
Split DNS typically refers to splitting the DNS results of a single, existing DNS server depending on who asks it, which is not what you want here, because that same server would be serving both external clients and internal ones and would need to differentiate between them.
You want an internal DNS server JUST for your own LAN, and its full-time job is very simple: to have all your local machines pointed at it for DNS, then it will either pretend it's authoritative and return the proper local IPs for whatever name you ask it for that's supposed to be on the local network, OR it forwards any other requests it doesn't consider itself "authoritative" for onwards to your Adguard or other DNS provider to get a real authoritative external IP in response.
The very simplest option for a bare-bones, basic DNS server that will do what you need is dnsmasq. Here is the default sample config for reference. Simply leave all "dhcp" related settings in the config commented out and you'll probably also want to set:
no-hosts(won't use the /etc/hosts file)resolv-file(an /etc/resolv.conf style file that tells it what actual nameservers to use for all other queries)address=/sub.domain.tld/192.168.1.1(for the subdomain and everything under it)host-record=sub.domain.tld,192.168.1.1for only that specific subdomain exactlyThen change all your local DNS servers to point at dnsmasq's IP address (you typically would do this at whatever device is handing out IPs on your network with DHCP, for example the router)
I think that's pretty much it.