the important part would be having similar interests, whether that is sports or music or politics or whatever. That is what gives you the chance to still enjoy hanging out with someone. Just talking about the past seems tough to do, there has to be other topics of conversation.
My two closest friends i've known for 2/3 of my life and im close to your age. One of them has a kid now so i see him less because of that, but we still laugh at the same stupid shit and will go to a show/event every now and then. I think these days everyone really wants to elevate every relationship i.e. friendship can't be JUST about the past or similar interests, or a relationship has to be like something from a movie. I sorta blame self-help books too. It's pretty much impossible to expect that from everyone, especially when many people are in committed relationships that take up their time. My dads friends were just guys he hung out with at a bar. Somewhere in Lebanon there are a bunch of dudes just sitting around watching soccer with each other.
Also you should be happy to have friends to reminisce with, because shared memory is a great thing. You definitely shouldnt let people go because theirs/your lives have changed.