there are tools in python to help

I haven't actually used pex but it doesn't look like it solves this - it's more of a way of distributing full programs. The .pex files aren't editable, which is something you need for this use case.

I imagine this strategy has its share of tradeoffs and gremlins.

As far as I know there are no downsides. It's basically win-win.