Yeah, or just continue to have two separate config files.
My team is in a bit of a different situation as we are cloud hosted but I suspect it's a similar approach. We have secrets hosted in AWS and our config file just references the secret name. We then have a SecretReader utility in our code which we reference to pull the secret from AWS at runtime, that way it doesn't show up as plaintext in our code. Our CI/CD doesn't touch the secret at all in that use case -- we only use secrets in our CI/CD if there is a step such as connecting to a DB that is part of our pipeline.