Don't get me wrong, if you have a good jam or preserves it's totally fine on a sandwich, and I wouldn't turn my nose up at it. However, peanut butter doesn't really need the help, and the extra sweetness from jelly is totally unnecessary. Toasted bread and peanut butter is all you need for a quality simple sandwich.
I like peanut butter in savory stuff much more than in sweet. So I would agree. I don't buy the kind with sugar in it. But I would never make a peanut butter sandwich either, unless the only food in the house was somehow peanut butter and bread.
Someone posted here that they liked a peanut butter and onion sandwich, I'd try that.