I believe I've fixed it.
A recent docker update no longer supports overriding hosts with hostnames. (I was telling mlmym that 'reddthat.com' is 'lemmy-ui'. Which would make it request directly to the lemmy-ui container rather than going back out to CF and back again.
(For others/if you want to know the nitty gritty). I was doing this:
networks:
lemmy_internal:
extra_hosts:
- "reddthat.com:lemmy-ui"
Which did actually work, but now it doesn't since I updated/installed docker compose v5.0.0.
(Seems the correctly enforce the host:IP format now). So I had to do a work around.