It doesn't usually go down when you pay debt off. In fact, paying off all your credit card debt every single month is a great strategy that will get you a good credit score. And is ideal, because that way you avoid the high interest rates that credit cards have.

It also doesn't go down if you check it with sites like Credit Karma. I believe what you're thinking of is hard checks, which loan issuers use and they can slightly ding your score as they represent you about to get a new line of credit. Though honestly that part is pretty sketchy, since it applies even if you don't get a new loan.