the amdgpu-pro drivers are only available on LTS releases and on a few selected distros

Are you sure? I would expect AMD to have their own download & install instructions that could be used on any distro. That would be more work for you than just installing a package directly from your distro, of course.

Curiously, I just found a comment from last year claiming that amf-amdgpu-pro now works with Mesa's RADV. So maybe this approach could work without AMD's proprietary driver?

Mostly BluRay rips, so movies and TV-Shows

By rips, do you mean your source media is already in a container, like a .mkv or .mp4 file? Or are you encoding directly from optical discs? If it's the latter, then using a tool other than HandBrake for the encode might also require finding a disc ripping tool. (Not all encoding tools can decrypt and demux discs.)

I actually was not. Is it any good? But Handbrake does not support that either, does it?

I haven't used Vulkan Video. It's just an API, so I would expect the video quality to depend on your hardware's encoder, just as it would with VAAPI or any other API.

I don't think Handbrake supports it yet, so until they do, I think you would have to use some other encoding tool.

for anything involving multiple audio and subtitle streams and stream selection in general it is not an option for me…

You haven't said why, but if it's just that managing lots of streams using command line tools is more hassle than you want to deal with, you might take a look at MKVToolNix. It's pretty good at muxing, even if the source media is not a .mkv file.