I am doing SSL termination at the handoff which is the caddy proxy. My internal servers have their SSL terminated at caddy, my traffic does not go to the internet.... It loops back from my router to my internal Network.

However DNS still needs to have subdomains in order to get those certificates, this cloudflair DNS. I do not want my IP to be associated with the subdomains, thus exposing it, therefore cloudflair proxy.

You're seeing the errors because the proxy backend is being told to speak HTTPS with Caddy, and it doesn't work like that.

You can have SSL termination at multiple points. Cloudflare can do SSL termination and Cloudflair can also connect to your proxy which also has SSL termination. This is allowed, this works, I have services that do this already. You can have SSL termination at every hop if you want, with different certificates.

That said, I have cloudflair SSL off, as stated in the OP. Cloudflare is not providing a cert, nor is it trying to communicate with my proxy via HTTPS.

Contrary to your statement about this not working that way, cloudflair has no issues proxying to my proxy where I already have valid certs. Or even self signed ones, or even no certs. The only thing that doesn't work is the ACME challenge...

Edit: I have now solved this by using Cloudflair DNS ACME challenge. Cloudflair SSL turned back on. Everything works as expected now, I can have external clients terminate SSL at cloudflair, cloudflair communicate with my proxy through HTTPS, and have internal clients terminate SSL at caddy.