Suppose some dude on the street hands out books for free and gives you a copy. Does it make you unethical for accepting one? Would it be different online?
Suppose your government charges a "blank media tax" on storage devices to "compensate" creators with the assumption you already "illegally" download their content, didn't you already pay for it anyway?
What if you're downloading stuff as a hobby but you'd never pay for it if that would be the only other option, did anyone lose anything of value?