Mathematicians need to reinvent what they do: mountain climbing (proof creation) might be dead but theory crafting and increasing understanding (novel proofs) will never be.
This is a pretty easy mistake to make when you don't have firsthand experience, but the problem is that's not a reinvention of what mathematicians do at all, it's only a reinvention of the presentation. Big mathematical advances are almost invariably driven by curiosity, but the university writing your paycheck wants their name attached to yours in publications of breakthrough results. Proofs are just the way a mathematician justifies that they're working on math. No proofs means a risk admin will triple their teaching load and say "you're only getting paid to teach now" with lip service about encouraging them to continue math on their own time.
If you have an oracle that gives a true/false answer to a conjecture, it won't destroy a mathematician's curiosity to ask why, but it will absolutely affect his ability to convince the admin that can't even read the abstract that this pursuit is worth funding. That's the crisis. Math papers are unreadable without years of study training your brain to think that way. A headline of "AI proves big theorem" is likely to win over a public that's grown up disliking math because of its use as an educational gatekeeper, even when it's unrealistic for human mathematicians to verify the AI's claim. There is a very realistic risk of administrators who can't define what a topology is declaring that mathematicians are obsolete.
The final answers to whether conjectures are true/false have never been the fruit of mathematics research. Think about Grothendieck's work towards solving the Weil conjectures - the machinery he developed to solve them is modern algebraic geometry, world-changing stuff that's become the backbone of cryptography among other things. Learning the conjectures are true is a footnote. If everyone had already known they were true in advance though, no one would have paid him to start working on them in the first place.
It would be like someone telling you we can finally stop playing your favorite sport now that the games can be simulated, but you try convincing them otherwise when they've never understood one rule and never put a game on because they'll just be confused and bored.