I think a lot of it is timing, too. Remember, the first Torchlight was 2009, we're talking pre-indie craze. There's been no Super Meat Boy or Fez yet, I think. ARPGs hadn't absolutely flooded the market yet and seeing a very competent and stylized, if simplistic Diablo-like back then could generate some interest. That carried on to the 2nd, which had a lot of improvements.

There's a bunch wrong with 3 and Infinite, but they were also competing in a saturated market and, you're right, the Torchlight "brand" didn't really have enough luster on its own to carry a series.

EDIT: Diablo II was 2000 and Diablo III would be 2012. We were fucking starving back then.