So, you could release the source but with the standard "all rights reserved" of copyright and let people compile it with a different version of the LGPL lib, but not let them modify or redistribute the proprietary Go code you've written to use it. It sounds counterintuitive because the source is "available" but this is how proprietary JavaScript code works in browsers to. It's there, you can read it, but it doesn't automatically mean you can "do" anything with it.

So yeah, distribute your Go binary with access to your Go code and instructio on how to compile it and you should be good, unless I'm missing something obvious.