You're definitely not the only person proposing your course of action. By the way, it would be silly for me to refrain from filing taxes since I'm likely to get a small refund. From your own reasoning, why would I want the government to keep my money even longer interest free? In that case, I absolutely should file my taxes to take the money away from them.
ETA: As far as paying taxes in your paycheck, just talk to your employer and change your W-4 to the lowest payout. Your employer is required to withhold some of it for taxes, so talk to your employer if you want to pay absolutely zero on payday (and they will likely say no). Obviously that all varies depending on your employment arrangement, but it generally holds up.
Fair enough. Thankfully like I said it's small and my livelihood doesn't depend on it.