My analogy is a weather forecast. Regular forecasts give you the temperature and precipitation, and let you decide what to do based on this. If meteorologists instead black boxed the weather and just tried to tell people what kind of clothes to wear every day, I suspect people would start ignoring the forecast. But this is how so many ML use cases are set up.
I'm happy to see an application that respects ML for what it can do and not try and shoehorn it into the "prediction machines" paradigm.