Item — HN Reader

I despise the term Artificial Intelligence. This is all PROBABILISTIC MODELLING. Nothing to do with AI/AGI/whatever.

The computers aren’t thinking or learning. It’s just modelling fancy probability statistics.

E.g. classical neural networks are basically a load of linear regression equations with an activation function stuck on the end of each of them. No magic. Just lots of linear regression.

This stuff only works when:

1) you are trying to solve a specific problem that is suited to probabilistic models

2) you have a data set that is sufficiently large, varied and specific

3) the model is developed, trained, tested, implemented and updated in a rigorous and sensible manner

1 comments

Actually most modern neural networks are not probabilistic, they are deterministic function approximators.

Also your point 3) isn’t quite correct either, often a “standard” architecture and training procedure (e.g. ResNet50 with Adam) will work on a new task with sufficient training data and minimal modification of the model.

The only nnets I mentioned were “classical” as a purposefully over simplified example. Yeah, they can model any function, but historically they were used for probabilistic density functions (if I remember correctly).

Most of what the article talked about can be done with much simpler models, which is what I get peeved about.

Also, yes, you can transfer learn with resnet. But if I throw my bank statements at it, it’ll do bugger all.

Similarly, if I throw new images at resnet in a silly way, it won’t transfer properly.

You might be confusing the historical use of the sigmoid activation function with probabilistic modeling, neural networks in the 80s were used similarly to how they are today, albeit at a much smaller scale due to hardware limitations at the time.

The development of neural networks is a major contribution of the machine learning community, so even if you’d like to split hairs about whether the “computer is learning” (“learning” has a a precise technical definition by the way), NNs are not “just statistics.”

Ok, it seems like there are some crossed wires or missing context here. Also, widely off topic.

I never said anything about the term machine learning. Check my bio, see what I’m working on. Fully aware of neural network contributions.

I’m all for machine learning. Just not “AI”. “AI” is hype bullshit.

“Learning” when used by the people who spout this BS is not the technical definition version, and is what I was referring to.

Could probably have made that clearer, but I’m 1.5 days without sleep.

What does feeding test data into a network yield? Inference results. Inference seems vaguely familiar from probabilistic modelling?

Bayes rule applies to neural nets too. Two different models may give vastly different results. Whilst they can be very good approximators, they can also be very unreliable if care is not taken during training.

G(x) ~ f(w.f(w.x+b)+b) is literally a fancy weighted sum. A linear regression. It is some easy stats combined together with a few other things that aren’t explicitly necessary, eg activation function can be identity to cancel out f().

EDIT Both the parameters of a network and the training data are variables in the application of Bayes rule. Which inherently deals with likelihoods (probability). /EDIT

So at their fundamental, they are “just some stats” stuff. They may have a few more bells and whistles to make them complex (and better) systems, but they still output a classification/regression based on inference.

You can, of course, approximate many functions with them. I’ve built a network with only weights of +1/-1, for example.

But those examples have extremely specific use cases that are not applicable to anything the article discusses.