they can say lots of things, reject framing and spit on fools
(it's akin to agreeing to questions of "would you say n-word to save your family", there are no comic book villains around)
P.S. What it does imply though, you can't take so and so is an anti-semite as being true judgement. Before corbyn, i would very unlikely to pursue the truth of somebody called that, and put them into the ignore box. Now that check of "anti-semitism" requires some verification (or more trustworthy than adl/government source)