I'm assuming the curse kicks in as soon as someone even tries to attune to it, or possibly even touches it. I suppose it's also possible that the curse has nothing to do with the crystal, and is just the DM commenting on the rogue being an idiot.