I think one of the big problems is doctors are either super cool or completely awful. It's a profession of extremes.

So like is the doctor an hour late because they were actively saving someone's life or because they just fucking don't care and had a long lunch etc

I tend to take the day off for big things. I also usually try to get the first appointment of a morning or afternoon session where hopefully there's less delays/time to catch-up.