There is an issue with your argument, fast charging is rarely used by EV owner.

EV can charge from a regular outlet overnight. So everyone with a private parking spot is charging their car overnight. Of cities can build outlets where there is public parking then only long distance travelers have to use the fast chargers.