Except those weren't "holidays" in the same sense as modern holidays, as in you had the day to yourself and you could do pretty much whatever you wanted. Those were church days, you had to go to mass, fast for the day, and/or do other assorted religious rituals. You didn't get a break from work, you were forbidden from working even if you wanted (needed) to because you're supposed to be worshipping God on those days. Failing to do so can get you shunned from your community as a sinner, or worse, executed for being a blasphemer, especially if you're a woman. If you're falling behind on the quotas your lord gave you, those kinds of "holidays" would only make the situation worse because you're not allowed to use them to catch up.