Item — HN Reader

As the article suggests, Skinner was a very rational person and often made the mistake of expecting the rest of the world to be just the same. He once said that if you come across something interesting, stop everything and study it. It's good advice.

Earning a Ph.D. in this area, I was always frustrated that - as utterly brilliant as Skinner was (e.g. he wrote a dozen books, including best-sellers, in the middle of the night after a short rest) - he always overlooked an obvious fact: people saw themselves as special from other animals, and would not accept any theory or technology that treated them as something less.

While 'operant principles' could explain everything from the emotional lives of people to the conditioning of neural networks, Skinner's approach was, once again, too rational to survive in the public sphere.

2 comments

But Skinner's ideas have been proven to be entirely limited to simplistic training. His most ambitious idea - that all complex behaviors are acquired through these simplistic reward mechanisms - is well known to be wrong now, a hopelessly limited understanding of the problem space.

As far as I know, he never engaged with the obvious arguments against his work - particularly the poverty of the stimulus argument. I don't believe he was a particularly rational individual - he was simply convinced of his simplistic model, and didn't want to see all the ways it didn't succeed in predicting all animal behavior.

It's not that humans are special, by the way. There are human behaviors that can be trained with behaviorist approaches. There are also animal behaviors that are obviously not trained in similar ways. For a good example of the latter, look at how long it takes quadruped animals to learn to walk - many do it within seconds or minutes after birth. Deer are leaping over obstacles hours after birth. That's not operant conditioning.

“Rational” in this context seems to mean “prone to reductive and simple explanations”.[1] But I have never heard of a scientific endeavor where people found out that things were more simple than they first hypothesized.

[1] And to be contrasted with those idiots who think that humans are “special” since they think that simple theories are not sufficient... or you know, maybe that organisms in general are complex machines or whatever you want to call it and that, yes, there are differences between a maggot, a bird and a human.

It's easy to dismiss it, and that's quite predictable. In fact there are a number of fields wholly or party based on Skinner, including Behavioral Pharmacology and Cognitive-Behavioral Therapy. See also the application to constructionist theories; https://en.wikipedia.org/wiki/Meaning-making

and see articles such as: https://www.sciencedirect.com/science/article/abs/pii/000579...

> It's easy to dismiss it, and that's quite predictable.

Your replies in this thread are quite predictable.

> "All models are wrong, but some are useful."... It is usually considered to be applicable to not only statistical models, *but to scientific models generally*. The aphorism recognizes that statistical or scientific models always fall short of the complexities of reality but can still be of use.

Emphasis mine. Human behavior is not physics. Even in that domain though, Newtonian physics is still useful - we just happen to know its limits.

https://en.wikipedia.org/wiki/All_models_are_wrong

For anyone really interested, operant conditioning describes being in the world. It was around long before humans existed. It's a selectionist system: neuronal pathways are the genotype, stimulus-response conditioning is the phenotype. In simple terms. You will think of a frog because I just wrote frog. That's historical conditioning. Nothing simple about it either, in animals or humans.

For example, a basic study allowed animals to self-administer cocaine. Other animals were administered the drug in the brain on the same schedule but without 'choosing' to take it. The findings showed that the areas of the brain affected differed depending on these two conditions. The role of behavior in this system of reinforcement is determinant. This is a study with rats, yet the effects of a seemingly small environmental/psychological difference changed everything about the reward experience.

Another study on electrical brain stimulation showed that while animals would self-administer it, when it was automatically administered, the same animals would respond always to turn it off. Again, this is agency and it is demonstrated via a Skinnerian model.

Yes, but this only accounts for certain limited behaviors, like consumption of substances like food. Other complex behaviors, like learning, are governed by other mechanisms, probably computational models that have evolved.

Again, as a basic example, operant conditioning can't explain how a newly born animal can operate extremely complex biological machinery like quadruped legs. Also, studying animal behaviors in the wild shows other mechanisms of learning, such as curiosity, self-directed play, or imitation. Imitation in particular can show one-shot learning of complex behaviors, without the need for reward mechanisms (for example, chimps and birds can learn how to operate certain mechanisms such as latches after observing someone operate it a single time).

Bringing human learning into all this is mostly a red herring. We can refute operant conditioning as the complete model of how behavior is determined without mentioning humans even once. That is not to say that operant conditioning is not a piece of the puzzle - it absolutely is. But it is far from the final word.

I only reply here because I know some readers will be open minded and interested.

There is a behavioral literature on what's called 'stimulus equivalence' - it shows how emergent, novel behavior evolves out of training the underlying relations (e.g., train a>b and b>c, then you get the emergent c>a). This happens with humans down to the level of mild retardation, but it's very hard to demonstrate it in animals.

Such emergent behavior is a kind of variability - the same genotypic variability that the environment 'selects' from to reinforce complex stimulus-governed learning.

If animals show observation learning, as you suggest, it's a mistake to conclude that it's not learned. As dozens of studies have shown, when the reinforcement is contingent, not of a fixed behavior, but on a conceptual behavior, that's what's learned (eg. an animal doesn't get food unless the sequence of three behaviors on a 3x3 grid are totally unique relative to the last 100 responses). Such contingencies are natural in all animal environments.

An animal or person can't execute complete motor performances according to some computational scheme, unless by this you just mean neural-networks (which don't compute, they just respond). If you look at primitive reflexive responses in insects, you see that the underlying neural pathways are, at the micro level, different for each individual. In other words, the brain develops according to environmental feedback (the first matrix movie has much to teach us about this). This is not learning, but the same notion applies to learning: a four legged animal has to learn to walk, then run, and gallop, and that neuronal-based learning is governed by feedback between the organism and the environment. You don't have to call this operant conditioning, but that's all that's involved: a complex biological system adapting from environmental feedback.

We overlook the power of conditioning because we think in simple terms of behavior being reinforced. As noted here -- https://psycnet.apa.org/record/2000-08385-002 -- behavior is not what's reinforced. What is selected is a stimulus-response relationship. Think of memory. Memories are not stored, if they were, the brain would have to remember where they were stored, ad infinitum. Memory is just a tendency to respond based on history: we see a movie, we remember a movie. It's all about remembering (a kind of behavior), and you won't think of what was the first car you owned if you don't read this sentence. Without stimuli, including interior/private thoughts, memories don't really exist. Sensory deprivation shows this impact of removing stimuli.

A problem with Skinner was he often simplified things too much. But not always. As seen here -- https://www.jstor.org/stable/27759371 -- his offering of a 'selectionist' model where the operant was the constructed product of cultural conditioning, anticipated a number of influential philosophers that few would think to tie to Skinner. As well as a number of post-modern philosphers

> If you look at primitive reflexive responses in insects, you see that the underlying neural pathways are, at the micro level, different for each individual. In other words, the brain develops according to environmental feedback

Let's take a look at this video [0] of a newborn foal learning to walk. It does so in approximately 30 minutes, without any explicit reinforcement from anyone (its mother is supportive, but doesn't explicitly encourage or discourage any particular behavior - it is simply caring for the animal). It also shows the ability to interpret visual input from its eyes, to recognize objects such as its mother, and the ability to produce and execute simple plans (get up, go to eat). If you look at a million million such foals, you will never see one behaving differently in any significant way - for example, not a single one will start grazing, or try to eat it's mothers teats instead of suckling on them, or try to suckle at thin air and so on.

The only possible explanation for this type of rapid discovery is that it was born with these abilities "built-in", trained by evolution over countless millions of years. There is no adaptation going on: the animal is just born with the "computing infrastructure" to perform these tasks. Of course, there is further learning to be performed later, and the algorithms for that are also part of the genetic endowment.

Note that "computing" is an extremely general notion, and artificial neural networks at least are absolutely computers. Whether biological neural networks are also computers is not settled science, to be fair, though it is hard to even imagine today what else they could be.

> Think of memory. Memories are not stored, if they were, the brain would have to remember where they were stored, ad infinitum.

By this argument, computers would be unable to store information as well. That's not to say that I imagine animal memory is equivalent to RAM or HDDs - they are vastly different. But RAM is definitive proof that it is possible to store and retrieve information without reaching an infinite regress of "who remembers where the information is stored".

> Without stimuli, including interior/private thoughts, memories don't really exist.

If internal states of the brain can be considered stimuli, than the entire concept of operant conditioning becomes a trivially obvious idea, a metaphor for almost anything. By this notion, CPUs can also be said to work based on operant conditioning: electrical pulses are "internal stimuli" that shape the behavior of the integrated circuits. Atoms themselves "learn" to decay when receiving stimuli from a neutron particle hitting the nucleus.

> An animal or person can't execute complete motor performances according to some computational scheme

Again, we have definitive counter-evidence to your statement from the world of computing. Modern robots that are able to walk decently in rough terrains are in fact doing so based on sophisticated (statistical) computational schemes. If you want to claim that ANNs are not computation per se (which is wrong), we can fall back to industrial robot "arms" which have existed for decades and are able to perform sophisticated environment-based movements while being manually pre-programmed using mathematical control theory (systems of differential equations).

[0] https://www.youtube.com/watch?v=7-kQJzd7muA

Well. Isn’t machine learning Skinner’s ideas applied to computers? This is what gave us the Deepl translator, and the Tesla FSD Beta. The other approaches, based on “cognitive psychology” and structuralism, on the other hand, we have, at least in technology, left behind in the 1990.

(And that influence likely is not just one of ideas, but also of people. “The Dream Machine”, the book on Licklider and the early days of computing mentions that some of the early computer pioneers shared the same office building with Skinner.)

Only reinforcement learning is based on Skinner's ideas - neural networks are instead based on purely mathematical and statistical methods - bakcpropagation and gradient descent. Other methods such as Support Vector Machines and Genetic Algorithms are also completely unrelated.

RL is useful for certain constrained problems, such as game playing, but it does not generalize well.

Additionally, all current ML methods are extremely slow to converge (they require enormous training sets compared to most biological learning), and they generalize pretty poorly outside their initial domain. So, their success is not a good argument in favor of Skinner's models being a good model for biological learning.

> As the article suggests, Skinner was a very rational person

In an audio recording he dismissed Chomsky’s critique of his work. He said that he hadn’t read it. He then went on to agree with people who had had read and disagreed with the critique... but how could he agree with them if he hadn’t read the critique.

Not sure. Skinner was at Harvard and Chomsky was at MIT. But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review. Skinner didn't believe Chomsky, or linguists generally, understood Verbal Behavior, which further encouraged him to ignore the review. At the end of the day, you can't have a real debate between two views unless both views are properly understood. Useful reference: https://www.researchgate.net/publication/223959896_Skinner_a...

> Not sure. […] But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review.

But that’s not the point.

Say you haven’t read X:

1. It is rational to say “I haven’t read X so I can’t comment on it”

2. It is less rational (or dishonest) to say “I haven’t read X but I agree with these other people who have read it and dislike it!”