Gaius Baltar really fucked Jean Luc up as a little kid. He never wanted kids after that, but he did fall in love with Beverly Crusher, except she became the wife of his best friend.

When his best friend died and left behind a ready-to-go family, he started to have a very difficult mix of emotions, which culminated in all of his complex behaviors toward Wesley.