All predictions are probabilistic.

Eh, not really. All math / model based predictions are probabilistic. There's other ways to make predictions, and not all of them are based on math, and they might be wrong more often than a probabilistic model, but they exist.

AI indeed isn’t great at modeling complex or difficult to quantify phenomenon, but neither are people.

Sure, fair enough, but there are times where a computer model is missing obvious context, and it's those times that I think we have to pay attention to.

The current industry adoption patterns seem to veering pretty close to "the computer did that auto-layoff thing" from Idiocracy in my opinion.