Black people where a quarter of a person at one point.
It's worse than that. The fraction you're referring to is 1/5 and they weren't considered people at all, they were slaves. Slaves were not considered people in terms of rights, but the number of congressmen (and also EC electors) a state had added the slave population divided by five.
So slave states had more power in congress and more voting power to determine who would be president proportional to how many slaves they had. More slaves = more "democratic" power for the slave owners.
Slaves had no rights, but slave owners had more power from that evil 1/5 rule.