that is a choice that is not determined by your material necessities.
Material necessities? Its the one thing we're hardwired to want to do in life. Procreation is one of the greatest material necessities we have. But thats beside the point here. The issue isn't buying a house. Its that people do choose to not participate in things based on their own moral choice. Some people are consumers some are not. A house isn't a material necessity. Having a place to live is but owning a home is not.
Following your logic, any job I take, by default, is a job potentially taken away from someone who needs it the most
You're on your own there because the whole point here is about men entering a job fair for women.
me taking any job will reinforce the current in balance. So what should I do?
Find job fairs for you. Taking a job here isn't the problem. Job fairs don't mean they get a job over you. It gives an opportunity to meet companies and apply. Something you can do on your own. There is a big difference here between applying for a job and entering a job fair marketed to women's and marginalized groups. You don't factor that in to your logic here and that's creating a big issue.