You're absolutely right.
Basically your credit score is some amalgam of how much credit you have available, vs how much you owe, and whether or not you pay your bills. If you have available credit and you have never used it, you suck. Giving you credit won't get the lender anything for their trouble. If you have a little debt, and you consistently pay it (read, pay them interest), then your score goes up because you're paying your creditors, so other creditors know that you're a good source of income for them.
The only thing that having a good credit score does, is give banks and institutions information on how much they could possibly wring out of you.
My score should be through the roof, but I'm too leveraged, I have something like 80% usage of my available credit. But I have a nontrivial amount of debt that I have carried for years and I've (sometimes painfully) always paid my bills on time.
My suggestion for anyone currently carrying significant debt trying to pay it off and increase their credit score: pay more than your minimum and take every offer they give you. If some institution says you're pre approved for some loan device, say yes. Even if you just throw it in a drawer. It will increase your total available credit driving down your occupied credit % and driving up your credit score.