Given recent events in USA politics, I'd suggest not judging a (potential) candidate only based on his performance on an entertainment show, which is mostly scripted, with a writer team and a prompter to read from.

Colbert would certainly do better than Trump, but it would be good to set the bar higher, see how the candidates act under pressure. For instance with a challenging debate or interviewer. It's more telling than a friendly and scripted TV format, where the question are easy and obvious.