Long term, I expect Vulkan to be the replacement to CUDA. ROCm isn't going anywhere...
We just need fundamental Vulkan libraries to be developed that can replace the CUDA equivalents.
cuFFT->vkFFT(this definitely exists)cuBLAS->vkBLAS(is anyone working on this?)cuDNN->vkDNN(this definitely doesn't exist)
At that point, adding Vulkan support to XLA (Jax and TensorFlow) or ATen (PyTorch) wouldn't be that difficult.