They do, but it’s a very simple speaker that’s really more of a buzzer than what you might think of as a speaker.

Many motherboards use a combination of beeps to report hardware errors if you fail on power on.