Yeah, they can rip your siding or roof off if your house hasn't been designed for them (and if you live where they usually hit your house should be), and major hurricanes can beat that hurricane proofing. I've seen the dry remnants of a hurricane do some pretty nasty damage like that. But yeah it's mostly flooding that's the problem where hurricanes tend to be.

Tornadoes on the other hand, are a lot less likely to hit any building in its path, don't flood (meaning underground shelters are safer), and when they hit your home are a lot more likely to look like a bomb went off