One minor problem is that life as we know it is supported by the continuous input of energy from a nearby star. Without it, no photosynthesis, and nearly all primary energy production in Earth life comes from that.

The slightly bigger problem is that by the time there are only black holes, there are no planets. Because, you know, there's only black holes. So nothing outside of black holes for life to be on, and the vacuum of space isn't really the most conducive to life or interaction of any kind.

And dig plenty of holes