Sounds like you are doing a lot. I have found the phrase "the personal is political" became a very real thing for me years ago even when I just started cutting out certain meats (!) - when it came to the reactions of others as they found out, and also realizing in a very visceral way, that, with every single meal, there was a very concrete ethical, ecological, economical and health decision to be made.

I quickly found out that you have to "come out" at work (when food is being ordered out, restaurants are being decided on, etc) and for extended family, etc., even though you really don't want to necessarily answer all the questions, parry all the nonsense in that particular moment. Most people are fine, maybe a small subset groan and roll their eyes, but keep their opinions to themselves, but there is that small percentage that seem to keep harping on it.

I have seen similar reactions to early adopters of hybrid vehicles. Save for EVs later. Or, as a kid, when someone with a legit extreme dairy allergy refused all dairy. It's like there is a certain type of personality that really gets actually offended when someone decides to deviate from whatever system is handed to them as "the norm", whether it's ridiculously high meat and dairy consumption (no matter how harmful it is to themselves, even), or a standard internal combustion engine. Some people seem to really get worked up about it.

Anyway, I do what I can. I have not removed all dairy entirely, nor eggs. I view both as rather harmful to health, given the information we have (and the evidence seems to keep piling up on that), so I don't make them a central component in any meal. I never drank milk anyway even as an omnivore - it's been fairly clear that they are marketing that stuff as a "health drink" (lol) for a reason. Sometimes it's very difficult to assess whether a given food is vegan or even at least vegetarian, but labeling/awareness has grown with time, so that has gotten a bit easier.