I once worked at data science consultancy. They worked on a project that they ended up using random forest. But they also ran a simple single layer net to validate the results, scoring a similar accuracy (it was like 70%). They gave a presentation on a Friday to the company (it was a regular show and tell demo slot).
At the end, I asked them if they double checked whether the net weights corresponded to the random forest and almost everyone in the room just looked at me like I was an idiot.
I ended up causing a heated discussion in the pub later on because I’d actually done something similar for my masters thesis (it was for modelling XOR operations on binary inputs).
The company in question is notoriously short term focussed, so it shouldn’t really have been a surprise that they didn’t even try to think about it.
And that’s kind of the problem. Short term speculative projects don’t care about interpretability. A director at some clothing firm doesn’t care about the super nerdy maths, they like getting their numbers.
But, on the flip side, Explaining loss functions and backprop is also a no go for the directors. If you can’t help them understand an algorithm in 2 sentences then they often don’t buy it. So then explainability becomes a big thing (hence why random forest was implemented and not the net).