Are you getting their specific electrons? No. (Electrons in AC systems don’t actually travel very far, you get the same ones jiggling back and forth!) But they make that much more power and your previous provider makes that much less power. The end result is you buy power from that provider, just as promised.

As you said, the grid must be balanced. Your old provider cannot generate the power, and your new provider must generate the amount if power you now buy. If either of those are not the case, the grid is not balanced.