What you're ignoring in this hypothetical situation however, is that self driving cars won't be 'self driving' for long long time.

Level 5, fully automated vehicles (where the driver can literally be asleep, and doesn't have to pay attention or intervene) are a long way from being implemented onto public roads. Ill be surprised if we see level 5 automated vehicles in 2040 like many manufacturers seem to believe.