I guess I haven't gotten those dialogues yet. So far the only ones with an explanation had explanations from what they were doing before they got infected.
spoiler
Wyll got his powers removed for breaking his pact, Gale lost some magic when he opened the evil book, Shadowheart could conceivably have had her powers erased along with her memories.
But that was tangential to my point, which is that all of the party members have backgrounds stating that they were operating on a higher level in a way that contrasts with the blank slate of the player character. This is not to mention that a high-level character in D&D should be swimming in magic items, even if they lost their actual character levels.