Oh yes. Numerical underflow is such a common problem when you're training machine learning algorithms. If you don't understand floating point numbers not only will you get inaccurate results, you won't be able to troubleshoot why.

Understanding the difference between time and space trade offs for floats and doubles (this is why even something as high level as numpy allows you to specify floats/doubles as a datatype) will not only allow you to trouble shoot things, but it will make your life a lot easier when debugging.

Of people who use these tools, only a very small percentage of them also build said tools. Which makes sense. In industry, companies want to build meaningful products and data sets that provide value which can be exchanged for money. That means delivering some majority of the value in the shortest amount of time possible. For that reason, engineering teams have little room for R&D, including the kind you're hinting at. Scikit, for example, has largely academic contributors (http://scikit-learn.org/stable/about.html). Not surprising. Talents of pedantry, proof-writing, and pushing the boundaries of theory probably lies in academia as opposed to industry. Industry rewards shipping code (useful oversimplification) and academia rewards novel theory (yet another oversimplification).

Solving problems requires understanding what solutions exist (and whether they can be used, must be built upon, could be used in ensemble, etc). Choosing among those solutions requires understanding (to some varying degree) why and how the solution solves the problem. Choosing the correct out-of-the-box solution is not trivial.

One very real danger here is unwittingly lying with statistics. Which is arguably worse than wittingly doing so.

It's important having at least a basic knowledge to avoid common, really stupid pitfalls that libraries can't handle out of the box but humans can (i.e. non-interpolable functions, small number division...)

Even if you aren't implementing things yourself (and past pretty basic usage, you'll have to do at least a little), you need to understand how these things work at so you can tell when or why the packages you are using are failing, and what to try after that. Beyond fairly basic modelling, almost none of this stuff "just works"...