Without tooting our own horns too much, this already exists and has for the last five years: https://remocon.tv/

It's very customiseable, has the same extremely low latency that they promise without showing, and has all of the flexibility in hardware and interaction. People should hook their robots up to it.