Found some in the woods yesterday and looked them up. Apparently the fruit is very tasty, resembling Starburst. Have you tried it?
Some more info on season/edibility: https://www.youtube.com/watch?v=uovfSkeyfbA
Super super tasty, 10/10, would recommend! Starburst is pretty accurate. Also a lot of people debate over whether the seeds will actually hurt you or not, but plenty of people have eaten them and been fine. That's up to your level of risk aversion. I try to keep the seeds for planting even though they're rarely viable.
While there are plenty where I live, most natural areas here are so ridiculously overrun with invasives that I try to leave the native stuff alone. Plus the squirrels always beat you to the ripe Mayapples anyway lol. They're very easy to grow from root though, so I just get mine from the garden.