There are a few different tracks here. One is software optimizations where models require less energy to use. That's been moving really fast over the past few years, and there are still a lot of papers that haven't been integrated into production systems that are really promising.

Another track is hardware architecture where the substrate stays the same, but chip design improves. A general example of this is SoC architecture like M series from Apple of Kirin 9000 from Huawei. The architecture eliminates the memory bus which is one of the main bottlenecks, and RISC instruction set facilitates parallelism much better than SISC. A more specific example would be ASIC chips like what Taalas is making which print the model directly on the chip.

And the last track is the one you mention with using a more efficient substrate. Notably this will directly benefit from the other two tracks as well. Whatever software and hardware architecture improvements people come up with, will directly apply to chips made out of graphene or other materials.

From a basic physics research perspective (as opposed to an engineering process development for production perspective), are we even sure graphene semiconductors have that much potential headroom for improvement beyond the best possible silicon ones? I'm not convinced it buys us more than a couple of process nodes. I mean, we're already making transistors so small you can damn near count the individual atoms in them today. Is making them out of atoms with one less valence level gonna be enough for a 10x, 100x, or 1000x improvement, even in the long run?