There are already chips being developed specifically for running LLMs, so it's almost certain this will become akin to having a GPU in the future.

• https://www.anuragk.com/blog/posts/Taalas.html
• https://wccftech.com/alibabas-tsmc-built-5nm-risc-v-chip-xuantie-c950-now-runs-qwen-3-8-27b-model-natively-unlocking-massive-vertical-integration-tailwinds/

And this tech is now being developed in China as well, so you might not have to rely on western hardware either going forward. Chinese companies are building stuff on top of RSIC-V like the Alibaba chip I linked above, and I fully expect there will be a lot of focus on Linux as China continues to decouple from the US tech stack.