It's already a market, look at all the android gaming headhelds that exist.

The biggest issue is that right now, the best phone processor doesn't even work as well as well as a steam deck. So you're paying $500+ to match a $300 device.

Maybe in the future the differences will narrow further as building games for ARM becomes more common which could make android support better.