Baka and it's various forms are actually stupid, fool, idiot, and the like. Calling someone stupid is a pretty common way to insult them, so if you see that, it's probably pretty literal