Don't get me wrong, if you have a good jam or preserves it's totally fine on a sandwich, and I wouldn't turn my nose up at it. However, peanut butter doesn't really need the help, and the extra sweetness from jelly is totally unnecessary. Toasted bread and peanut butter is all you need for a quality simple sandwich.
My unpopular opinion is that peanut butter does not belong on a sandwich. Don't get me wrong, I love peanut butter, on its own or as part of a snack or dessert, but spreading it between two slices of bread? It doesn't matter what else you put on it, it's way too sticky and messy for me to enjoy it.