I deal with mostly a vegetable/herb garden and a native (to my area) flower bed, so my experience with gardening comes from that. I don’t recognize a lot of the plants in your photos so I can’t give a lot of specific advice. However here’s what I know/noticed:

• Dead stems: Yes you can cut off the dead stuff from last year’s growth. I cut them as close to the ground as I can. I wait until spring because it makes a good winter home for butterflies and other helpful insects. Plus if the flowers make any seeds birds and other animals can eat, leaving them until spring gives the animals some extra food during the winter.

• Weeds: What is a weed will greatly depend on the area of the world you live in. Several photos have creeping charlie in them. Creeping charlie is the stuff growing low to the ground with the purple flowers in image 4272. Creeping charlie isn’t native to my area, so I pull it because otherwise it creeps into wherever it can reach (which is everywhere). I would keep an eye on image 4261, that one the leaves look somewhat (but not exactly) raspberry/blackberry species which can be welcome or can easily take over. However I will say the stems aren’t what I’d expect for a raspberry species.

• Plant species: 4253: The leaves and stem are saying columbine to me. 4254: The tall leaves could be some kind of day lily or tulip like the flowering ones in 4275. 4259: Bleeding heart is in the upper left corner.

Hopefully someone else will recognize more of the plants so you can find out exactly what you have.