The issue is thermal mass. Buildings are designed to absorb heat in the winter. Obviously when you aren't using heat they'll absorb whatever the temperature is.
First 2 days of a heatwave the building holds a cooler temperature. After that the walls begin to heat and it is simply too hot at night to dissipate all the heat the building has absorbed