I have Tyrian 2000 on GOG as well and was feeling nostalgic. I could replicate the issues and here's what I did to fix it.
- Installed the game with Heroic and added it (automatically) to Steam.
- Applied a community control layout named "Tyrian" to fix the mapping issue.
- Launched the game from Steam and used the onscreen keyboard to press 3 to go to "Game DOS settings". I had to run it twice before the controls worked for some reason. Change the music device to "FM Sound" (I think the default was "Midi 330h") and saved.
- Returned to the launch menu and pressed 1 to launch Tyrian 2000.
I usually like running DOS games in EmuDeck/Retroarch though.