36 states require identification to vote. A passport is ID.

Gonna stop everyone right here and mention that ID requirements were not the norm until recently. It is, and always was, possible to prove citizenship without issuance of an ID (let alone a photo ID). Studies have shown that it does very little to reduce voter fraud and only disenfranchises vulnerable populations.

Secondly, elections are the purview of the states. Any requirements that give feds control of the voting process are unconstitutional. A federal passport is US property issued at the discretion of the government and can be revoked at any time (as shown in this article), and is therefore unconstitutional.