Ironically, the whole thing falls down on its own if you wait long enough, regardless of where you first click (not enough friction between the blocks)