One problem is many devices don't have a proper BMS integrated into the charging circuit, that will cut power before the battery is below a critical threshold. Li-ion cells start to have problems below 3V, and often suffer permanent damage below 2.5V.

This is a problem with rechargeable AAs too, though less common maybe? Many kids toys and cheap decorative lights will drain a AA to dead in my experience. I've had to reintroduce disposable batteries with the rule "no microcontroller, no rechargeable"