I'm going to upgrade from my 6900 XT to a 9070 XT tomorrow. I've determined that there's an ~40% performance uplift. To me, that's worth it for 1440p gaming (170hz monitor). I'm also pretty stoked to see FSR 4 in action.
What VR headset are you using? In any case I'd say it's worth it for VR, since you really want the best possbile immersion and absolutely no stuttering/FPS drops.