A lot of golf courses used to be mens only, and the ones who did let women on the field had strict dress codes. Going as far back to Victorian times women had to have specific Golf dresses, corsets, crinolines, bustles, and multiple petticoats.

So a lot of it is based on social mores surrounding both class and sex. Even after dress codes loosened in the modern age, the culture of golf tends to be immersed in the social normative of the upper class.

Personally I actually think class has more to do with it than sexism in general. For example the first professional woman to play golf in slacks was in 1933, while the first black professional golfer was allowed in the PGA in 1967.

Then again the dynamics between social norms in sex and class are dynamic and inseparable.