He was talking about serious things and tan is an unserious color. It's never okay for a president to act unserious unless he's a white male Republican, in which case it's fine.
Racism is the context, the justification, and the motive. It's not really much more than that.