The issue is that the keyboard focus changes with normal usage, and impairs normal usage. If you go to jump back with left arrow, and accidentally touch down arrow, the focus is unrecoverably moved and you can't even pause anymore.

This wouldn't be a problem if using the mouse or the up/down arrows didn't focus the volume slider. Tab should be perfectly capable of moving the focus if you want to focus volume specifically, especially as volume has dedicated shortcuts already.