It’s obviously real, at least as used in conversation, whether it meets some rigorous definition I’m sure there’s an out, but we’ve all known these people. With vibe coding they’re everywhere. Is this going to be a modern “begging the question” where everyone knows what you mean but someone pipes up that actually the technical meaning is different?
back
2 comments
The problem is that the original formulation was "most people" are unaware of being unskilled, but by now the name is used to mean "some group of people" is unaware of being unskilled. In that sense the article confirms that “a small number are", 5-6%, and therefore the effect exists in the colloquial sense but not in the scientific sense.
Mentioning the Dunning-Krueger effect incorrectly is wonderfully meta...
My experience with new CS grads was that most of them greatly overestimated what they knew, or alternatively, underestimated how much they did not know.
My experience with every person was that most of them overestimate what they know regardless of experience level. You just notice that more in new grads since its easier to tell when people are wrong about simple things than when they are wrong about more difficult things.
Software engineers tend to repeat the mantra "you cannot make accurate time estimates". That is true regardless of experience level, and everyone seems to be off by about the same amount. So there we have evidence that people overestimate their skills at every level, and its not that different.
gell mann effect leads me to pay closer attention and then its game over
It’s my experience that I should over-estimate what I know otherwise I miss out on assignments that take me about 1-2 focused days of studying to get to sufficient level.
Ie I was once perfect for a project except for point 7 out of 10 which was experience with Keycloak (if you’re higher via an HR dept it’s even worse, they just tick boxes, who cares if you are smart and have broad knowledge).
So I’m now proudly Dunning-Krugering around, and use LLMs for super-charged learning. If I underestimated something I take the cost/time myself. So far it hasn’t happened to a significant degree.
This then boils down to people being generally bad at estimating their own level of ability.
Which is exactly what the Dunner-Kruger effect is. Knowledgeable people statistically underestimating their knowledge and non-knowledgeable people overestimating their knowledge.
It that’s the whole point that it’s just a mirage and is the same with random data. The people on the low end can’t underestimate their results as much and the people on the upper end can’t overestimate their results as much. That will come out of any correlation that is not perfectly correlated, which is why the article talks about it being replicable with random data. An interesting graph that would demonstrate a novel effect would be something nonlinear.
There are specific distributions that can be explained by random noise, and other distributions that cannot be explained by random noise.
But even if it is entirely caused by normal distribution variance, that doesn't make it a mirage, it gives you a reason.
I've seen at least 3 different ideas on this topic:
- people not skilled in a thing are bad at estimating their skills
- people are generally bad at estimating their skills
- people skilled in a particular areas often feel they are intellectually fit in other areas
Out of these three I feel like there's some truth in it, at least anecdotally.
RationalWiki calls the third one 'Ultracrepidarianism', and says it's rather common with engineers in general:
https://rationalwiki.org/wiki/Ultracrepidarianism
(the Wikipedia article has that as a subheading on the Ne supra crepidam page instead: https://en.wikipedia.org/wiki/Ne_supra_crepidam#Ultracrepida...)
But I'd say it's a subtype of the Dunning-Krueger effect rather than part of the general definition.
Personally I've always thought of Dunning-Krueger as an extension of the first definition here. Newbies/amateurs overestimating their skills and experts underestimating them, though I feel like the second definition is probably more true of human behaviour.
>RationalWiki calls the third one 'Ultracrepidarianism'
LOL because RationalWiki was written by engineers. I've often seen it referred to as "Engineer's Disease" for how it's commonly encountered.
for the last one, that's the whole story of Socrates being the wisest man because unlike others he knew the bounds of his knowledge:
"I seem, then, in just this little thing to be wiser than this man at any rate, that what I do not know I do not think I know either."
For those interested, it’s not just in Apology this comes up. In Theaetetus he does a pretty good job disproving that it is even possible to know what knowledge is.
I think the third one is a separate thing. I don’t know of a name for it, but I’d call it the engineer’s (or physicist’s) blindspot or something like that.
I can’t remember of DK suggests some sort of effect where the expert has undue self-doubt, though…
> I can’t remember of DK suggests some sort of effect where the expert has undue self-doubt, though…
There is a brief bit where they attribute that observation to other causes:
In making these predictions, we felt that we could account for an anomaly that appeared in all three previous studies: Despite the fact that top-quartile participants were far more calibrated than were their less skilled counterparts, they tended to underestimate their performance relative to their peers. We felt that this miscalibration had a different source then the miscalibration evidenced by bottom-quartile participants. That is, top-quartile participants did not underestimate themselves because they were wrong about their own performances, but rather because they were wrong about the performances of their peers. In essence, we believe they fell prey to the false-consensus effect (Ross, Greene, & House, 1977). In the absence of data to the contrary, they mistakenly assumed that their peers would tend provide the same (correct) answers as they themselves—an impression that could be immediately corrected by showing them the performances of their peers. By examining the extent to which competent individuals revised their ability estimates after grading the tests of their less competent peers, we could put this false-consensus interpretation to a test.
Link if you need it:
https://www.researchgate.net/publication/12688660_Unskilled_...
> people skilled in a particular areas often feel they are intellectually fit in other areas
This I think is a separate phenomenon, maybe Nobel Disease but there might be a more general term, for example that includes celebrities.
It's called "engineer's disease" which is pretty appropriate for this community:
My experience has been that this isn't limited to engineers. I'm sure most people here have had people from other departments tell them how they need to do their jobs in one way or another.
For example, there are always at least a few people who you have to consult regularly because their tickets are always written how they perceive a problem should be solved rather than explaining the underlying problem to be solved. Then there are the sales and marketing types who like to dictate how the UX of the front-end should be because they like to retell anecdotes to prospective clients about how they solved a problem. Then there's the guy who hands you a spreadsheet he's maintained for years and tells you he wants you to put something exactly like it in the flagship product as a new feature.
>My experience has been that this isn't limited to engineers.
To some extent, but the most severe form does seem fairly unique to engineers, specifically programmers. That is, I've met many arrogant lawyers and doctors, but never a lawyer or doctor who assumes they could program a computer due to their legal or medical expertise. But I've seen many a programmer assume that because they are proficient in programming they can confidently and authoritatively expound on all other areas of human endeavor (those being obviously inferior and trivial in the face of their own).
I'd say celebrities and creatives in general are often big offenders here too. Lots of actors/athletes/musicians acting like being good at what they do means they're also qualified to talk about science and engineering. And a lot of reviewers, documentary creators and specialist writers that decide to wade into discussions about topics they have no knowledge of/cover something they're new to, only to look ridiculously out of touch as a result.
See the Nostalgia Critic review of the Wall. Or the Alex Meyers review of Obsession.
Or all the celebrities who write children's' books.
Medical doctors and physicists also regularly suffer from this. It’s more of a disease of expertise in general, it seems.
Well, if I'm systems engineer and everything that exists can be modelled as a system...
There's also the idea on the other side of the coin - whereby experts on a topic underestimate how much they know about it compared to the average person.
And of course there's a relevant xkcd: https://xkcd.com/2501/
Imo it would be more accurate to say that topic experts overestimate how much the average person knows about it.
Sorry, I'm confused. My formulation "experts on a topic underestimate how much they know about it compared to the average person" and your "topic experts overestimate how much the average person knows about it" seem semantically identical to me, just from a different side of the relation. What nuance did I miss?
What is “it”, of those 3? The last one?
The four levels of competence:
[ Unconscious Incompetence ] --> "You don't know what you don't know"
↓
[ Conscious Incompetence ] --> "You know what you don't know"
↓
[ Conscious Competence ] --> "You know, but you have to think about it"
↓
[ Unconscious Competence ] --> "You know it so well, it's second nature"It can be true tho. You might be genuinely skilled in one domain and realize the extent of your skill, while being over-confident in one you have little experience.
I also noticed people really successful at one thing tend to underestimate other domains, either thinking they are easy or that they can have meaningful input while being essentially novice (see any time Musk opens his mouth for example)
I think it's beautiful irony that many people have the dunning-kruger effect specifically concerning the dunning-kruger effect itself
With all due respect, being cocksure enough to declare that the Dunning-Kruger effect is definitely a thing, even when more careful analysis shows that the study conclusions were in error due to faulty analysis in the original study, kinda sounds like the perfect example of the Dunning-Kruger effect...
The one that doesn't exist?
Yes, that one.