Your view is an intellectually honest one from a modernist perspective. I would go further though and say that marriage should have no secular existence at all.

• polygamy is patriarchy, gay marriage isn't

First of all this is a self-refuting assumption. It assumes out of the gate that women aren't equally capable of leveraging polygamy to the subjugation of men. If women are not equally capable of abusing polygamy then patriarchy is naturalistic. If they ARE equally capable then this objection collapses.

Secondarily modernity leverages nothing but subjective feelings to make a moral claim about why something like patriarchy is wrong in the first place. "Patriarchy is bad" says who? And why should anyone care? Most of the world and history disagrees with that characterization. If cultural imperialism is "bad" isn't it culturally imperialist to wholly reject all surviving traditions that predate the last 150 years because they aren't compatible with an emergent value system? I could go on but hopefully you get my point about rootless modernism and it's lack of justification for ought claims. Not to mention the lack of logical consistency for their ever-changing framework.

• incest is bad for the gene pool, gay marriage isn't

Should people with genetic defects be able to reproduce? To what extent are we just acting as eugenicists?

Marriage and the rules around it are inherited from traditions that modernism rejects. The attempt to continue PARTS of these traditions by arbitrarily picking and choosing rules because of what makes us "comfortable" undermines the authority of marriage in general. Why even continue it?

Modernist takes on marriage are anathema to the entire point of marriage in the first place. Furthermore modernism offers no satisfactory reasons for why "modern marriages" should exist at all. "Taxes" is often cited but this could be managed in many other ways. (e.g. legal contractual relationships that enable many of the same benefits ala power of attorney)