I completely agree with all of that. But I'd also like to point out that the prison system in the US isn't being run with rehabilitation in mind. Our system is much more about punishment. They'd rather have people stuck in the system because they can be used as slave labor.
It's one of the biggest failures of the US IMO and not a lot of people talk about it. Slavery needs to be illegal in all capacities. And prisoners shouldn't be allowed to be used as labor. In theory I don't have a problem with it, but in practice it just encourages abuse, as you have pointed out.