The magic word here is "mulling spices". It's the same blend used for pumpkin pie, apple cider, and a lot of different mulled wines.

You can buy premade mulling spice mixes to give it a try, but I highly recommend finding a recipe online and tinkering with it on your own. Once you've got your mix dialed in you can whip up amazing apple cider all fall and winter.