There's a HUGE difference between not allergic to unbroken leaves and not allergic to cutting up poison ivy leaves.
Take it from someone who isn't "allergic" to poison ivy and had a rude awakening after accidentally hitting some with a string trimmer.