You need to present it in certain cases (usually government stuff) as a part of ID verification. Other than that, I think everyone does just use the number since it is needed for tax forms every year. I'm old enough that my uni student ID used to be this number before they realized what a terrible ID that is (name, address, birthday, and SSN used to be enough to open bank accounts, credit cards, start/stop utilities, etc.)