I think Goliath was a "giant" in the same sense that an NBA player is a giant. Not like a mythical species in the Bible, just a genetic anomaly family that was crazy big and strong. Meanwhile, the average height was back then was much shorter than it is now.

So think of a fairly average but smallish guy beating up Shaq and I think that's what we are supposed to envision.