There are definitely a lot of people who are interested in Roman history and are not fascists or racists.
Buuuuut it’s also true that Roman history sure does attract a disproportionate share of fascists and racists.
Source: I used to be a Roman history guy.