What a dick move! You were told it was optional, you shouldn't be punished for believing what you were told in class by the professor. You should definitely appeal.

Lesson learned - follow the syllabus 100% of the time.

I wouldn’t appeal this personally (you do certainly have that right and you know the situation better than anyone). In the grand scheme of things a B instead of an A won’t make much of a difference if you aren’t competing for top-tier Law or Medical schools post-grad. It will be a lot of bureaucracy that will ultimately come down to he said/she said.

You might have the best luck just showing up to the prof’s office hours and asking (informed) questions about the missed material. Building some rapport with the professor might open them up to some leniency, especially if you’ve shown that you’re dedicated to making up the material.

Unless you have evidence that was said, eg an email, you're wasting your time appealing.