back
75 comments
You should ask the following about every interview question you are considering asking:

1. What answers would lead to a hire and what answers would lead to a reject

2. Does this align with someone able/not able to do the job.

Without these the interview question doesn't provide you the information you need as an interviewer.

3. Have I tested this question on current employees?

4. Is this a puzzle question with only one answer, or are there subproblems they can still solve and I will get actionable data from?

I would like to politely disagree. Unless you are hiring robots or people working in a robot like role, there is - extreme outliers notwithstanding - almost never a single kind of answer that would make or break the interview.

When interviewing, you should be looking for evidence that the candidate might be able to deal with the problems that they are confronted with in their new role. Questions should be directed to finding that evidence, and answers can basically provide you with evidence, provide you with evidence to the contrary or be neutral. At the end of the interview you hopefully have collected enough evidence so that you are convinced the candidate is able to successfully work in the problem space their new role brings.

This is great advice.
I ask this question - always - but it goes like this.

Q1: Rate yourself 1-10 on X language? Their answer is Y (and almost always 7).

Q2: What's the difference between a Y and a Y+1?

Q2 is real question with Q1 being there just for framing. Some people simply state that the difference between a 7 and an 8 is "just more experience". TBC, that's not a wrong answer.

However what gets me really excited is when this line of questioning shines a light on what the candidate is curious about, or interesting in learning next. It goes a step beyond just self-awareness that there is always more to learn. To me it's a signal that this person has a plan and a drive to grow oneself.

While I do not make binary hiring decisions on this question or any question; I can vividly remember the answers I was given to this line of questions by some of the best engineers I've worked with.

Couldn’t you just ask „what are you curious to learn?“ directly, instead of beating around the bush with awkward questions like „what would you rate yourself?“ that people have to „game“? It’s just another way of asking „what’s your biggest weakness?“ with the obligatory „my perfectionism“ response. I just skip the fluff and communicate openly.
I like this reasoning but would rather ask this question about a technology rather than a language. That may just be my particular field, though.
I do ask this question in a initial questionnaire. I list bunch of technologies and just require this input. I do agree with all the points in the article, but my goal is to have relative scale. I am not able to verify all the needed technologies, but I can do verify few. Whatever the scale candidate choose, I assume its consistent. If someone self-scores 9/10 out of Postgres and doesn't understand basic things, I can safely assume his other 9/10 are worthless.
Agreed, it's a bozo filter. It's not intended to find good candidates, it's intended to weed out bad ones.
Hmm, if they ask you to rate yourself out of 10. Should you then ask them to rate various aspects of their company on same scale? Like work-life balance, leadership, team's technical skills, competence of managers and so on?
Do you not ask interviewers questions during an interview?

Like yes, during an interview you should attempt to learn what it would be like to work there.

i'm surprised people bother asking questions this inane. (or at least surprised people think they get value out of this.)
If someone responds with “10” and fails to answer basic questions about that specific topic you know that the person is either lying or is overly confident.
It's bullshit psychological gaming. Obviously 1–6 are right out. Can't pick any of those. 10 is also out. (You're not that arrogant, are you?)

So the question becomes: "7, 8, or 9?"

Still some strategy to think about here. If you answer "7", then you're saying, "I'm not that good but even I know not to answer lower than 7". Answering "9" is similar, except you're saying you know better than to go full self-aggrandizement.

Yup. Nothing of value is gleaned from this question.

When I was asked this (early in an application process) the question came with an anchored scale. That is, each number had a corresponding description. I thought it worked fine.
This is entirely different, and does avoid the central problem discussed in the Fine Article: is 1...10 to mean ±5σ, or are they deciles? Either way, what's the sample group? It also sucks because there's not a whole-number median answer: 5 is further from 10 than it is from 1, 5 + 5 is 10, 5 - 5 is 0, not 1.

However, ten levels is too many. Five is better. With ten levels, someone is always unsure whether they belong one higher or one lower than the answer they give, so you have a test which combines a real self-assessment with an unknown variable: the personality of the person you're asking.

For a skill of interest, five levels will give most candidates a clear image of where they actually belong, and the differences are large enough that a dishonest answer can be cross-checked with straightforward "I expect a 4 to answer this correctly" questions.

I was once asked: "Where do you see yourself in 5 years?"

I answered: "CEO of this company"

Didn't get an offer :-)

"probably somewhere else" is the correct answer in the US.
Have been freelancing for ages and in my experience talking about recent books you have read that are relevant to the employment absolutely seals the deal. Implicitly it shows you are 10 out of 10 on a topic.
I think the correct answer should involve a calibration, some reference to what you interpret the value to mean.

I wouldn't ask this question to a job candidate, but in any context where it might come up, if the response is "10 because I am at least 90%tile on a speed or error rate compared to other candidates" then that's acceptable to me.

It should make sense, though. If the response is "where 10 is having written a frequently referred to text on the entire subject I would score an 8," what does it mean.. Wrote 80% of a book? Wrote a book but it's referred to 4/5 as often as the best book? Neither of those are really practical interpretations.

As usual, it's the way an answer is given that can make more difference than the response per se. If ever asked this kind of "rank on 10 your skill" question then giving a reasonable benchmark for your choice can improve the clarity of your communication, at the very least.

I see candidates putting skill level bars, e.g.

    ------- 1 ------- 10
    C++:    xxxxxxx
    Python: xxxxxxxx
    SQL:    xxxxx
I'm not seeing any upside. Are you someone that did this, why?
I don't do this, but I considered it because it allows you to list a wide range of skills (critical for getting past filters) but also tempers expectations in an interview. E.g. I can and have programmed in Java, but if you ask me about less common Java features I'm probably going to give pretty weak answers.
I have seen this used two ways:

1) As a recruiter-level screening question. A recruiter can't evaluate your answers to a coding problem, but they can reject all candidates who select less than "7" on a slider. It's basically a cheap way to go from 1,000 resumes to a more manageable number that's actually shown to the hiring manager.

2) As a "gotcha" on technical interviews: oh, you selected "9" on C++? Well, here's my favorite IOCCC entry, figure it out! You can't? "No hire, the candidate lied." It's counterproductive, but many engineering organizations approach interviewing this way (also see: "brain teaser" questions).

Between #1 and #2, the safest answer is always "8".

There was a ratings system like this in early days of Google but it was accompanied by a very explicit guideline for what each number means. IIRC 6 was demonstrated mastery, 10 was "literally wrote the book on the topic."

It wasn't for candidates, though, it was initially for interviewer stats and possibly maybe referred to for team transfers.

(the current version is in a completely reworked system and on a four-point scale that includes "interested" and "currently learning" so really only a two point scale of claimed knowledge)

I think at some point this started spreading around on blogs as "a thing you can do to separate yourself."

A lot of resume advice floating around is not good.

I did this. I was a freshman in college, didn't know better, and needed to pad out my resume somehow.
It can too easily go this way: https://www.youtube.com/watch?v=IkYUDQCYGHA
I used to have this type of rating on my resume when I started my career. Over 10 years later, I can tell that today it wouldn't communicate anything.

At most you could convey that you're more proficient in language A than language B.

This article misses that the asker knows about these limitations as well, and it also misses that the answer in itself alone is devoid of meaning for the interviewer.

It’s the context that matters - for example a comparison of skill A which the candidate rates at x out of 10 to skill b where they rate themselves a y out of 10. Both x and y are meaningless, but the relationship between these two is not.

Self awareness is also important, both in terms of strengths and development areas. People need to be aware of their blind spots because we expect them to take a step back and let themselves be led where appropriate. And people need to be aware of their strengths because we expect them to step up and take the lead if necessary.

A quantitative scale for qualitative things really helps having the right discussions with candidates.

I've been asked this (on a scale from 1 - 5) and have had some success referencing previous performance reviews (i.e. meeting expectations or exceeding expectations). YMMV obviously.
As usual, you can do a thing relatively well or poorly. The post is describing the dumbest version of it and attacking it.

iirc, Google recruiters used to (still do? dunno) ask people to rate themselves in a bunch of areas.

* Why are you asking? So we can pick interviewers knowledgeable in the same areas you are.

* How should I rate myself? Here's a reference. 10 means you are the world expert, maybe invented the thing I just described. 9 means you literally wrote the authoritative book or are on the standardization committee. 8 means you wrote a book on it or the like...and so on.

In general, you should expect that you will discuss the things you rate yourself the most highly on with someone at Google who knows them better than you do, or at least (if you really are the world expert) one of Google's most confident people on that subject.

I think this was a useful practice, at least in the context of Google where they have a very wide interview pool and actually sometimes do interview people who literally wrote the book on something.

> * How should I rate myself? Here's a reference. 10 means you are the world expert, maybe invented the thing I just described. 9 means you literally wrote the book. 8 means...and so on.

This is exactly what I'd do, ask what constitutes a "10", and if the interviewer can't answer that question why are they even asking?

I do the same thing at AWS, except use a 100, 200, 300, 400 level scale. I am just looking to see what the candidate thinks they are strongest on and weakest. If the candidate says they have never heard of AI/ML topics, there's no point in quizzing them on it. Not everyone can know everything, it wouldn't be fair to judge someone based on not being able to answer questions for a domain they don't claim to have any experience in. Or worse yet, waste valuable / limited interview minutes on fruitless questions.
But it's still a useless question isn't it? If someone literally wrote the book on the subject then you should probably know that if you're interviewing them. And anything below 8 feels meaningless. How do you differentiate a 5 vs 6 vs 7? Why can't the answer just be "what do you know about X" instead of having to use an arbitrary number?
chuckle back when I worked at Intel, a question we'd ask candidates for a C++ programming position was, "On a scale of 1 to 10, how would you rate your understanding of C++?"

Anybody who gave an answer of 5 or above was an immediate no-hire :-) Because nobody is more than a 4 out of 10 on knowledge of C++.

> Anybody who gave an answer of 5 or above was an immediate no-hire :-) Because nobody is more than a 4 out of 10 on knowledge of C++.

You'd definitely hire programmers with the same sense of humor I suppose.

I do agree that asking to rate one's knowledge of C++ is an obvious trap question. Any reasonably seasoned person would probably laugh and respond with another question, "... which part?" But yeah they'd have to share your sense of humor.

I wouldn't be so absolute about it, but I certainly empathize. Any time a candidate told me they were an expert in C++, it didn't go well. I think that suggests they are either wildly overestimating their abilities or are too interested in being "clever" and using weird corners of the language that aren't necessary/beneficial for the problem and that no one else will understand. Probably both.
When someone asks you if you're a god, you say, "yes!"
The only way I'd ever answer “How would you rate your knowledge of techonolgy[sic - but c'mon, is this supposed to be a professional blog?] X out of 10?” with a 10 is if I built the technology myself, and then probably not even then as I know there's someone out there who is going to make it do things I never even imagined.
I think this question is very revealing when paired with another related question. The rating is unimportant and not to be used as a requirement, anyone can lie and say 10.

I feel that it shows insight into how strong a candidate really is and how much they are bluffing, for example “rate yourself in JavaScript” and then asking a JS question. If their answer lines up with their self-assessment (somewhere in the ballpark) then it seems more likely their other claims are true.

However, I would never ask this question and rate a candidate on whatever number they decide to come up with. Lots of times the most skilled engineers rate themselves lower than you would expect.

So yeah, the number is meaningless, but I think the question can be revealing in an interview setting.

We used to ask people to rank themselves on the page-jones scale of expertise (detailed here https://medium.com/@helen.taner.nc/how-to-estimate-your-soft...)

Generally, people who ranked themselves as Masters were subject to the dunning Kruger effect (and/or arrogance)

Eh. If you are an experienced professional, you should be able to triage. Making semi-accurate, off-the-cuff judgements is exactly what you are being hired for. I've never minded these questions.

> It’s entirely ambiguous, and depending on which interpretation you give, there’s really no wrong answer.

This is a feature, not a bug. There are going to be other parts of the interview process, knowing where you are confident and where you are not is an invitation for further discussion.

I would not ask such a question. If someone asked me such a question, then I might say: Due to the Dunning-Kruger effect, I don't know.
Meh.

From 1 to 5, rate yourself and explain why.

The value itself isn’t what matters, but the reasoning of the candidate and let both of you discuss where the candidate needs some ramp up and where they want to grow.

At some point, a question is like a metric: once you track it and try to improve it, it ceases to be a good metric.

That’s why multiple interviewers cross check themselves and the probation period helps confirm or affirm the fit in the position and the team.

a really talented developer i worked with told me that the absolute rock stars in the industry would call themselves 2-4 while the worst would say 6-8. dunning kruger in effect
"I'm an 11 on the Tufnel scale."
Sort of tangential, when I was interviewing for a job, one of the hiring manager asked me list my four (!) biggest weaknesses.

I can think of hundreds of weaknesses on my end, but of course it's an interview so you don't necessarily want to tell them everything, just stuff that either doesn't make you look too bad. It was extremely difficult, and I thought it was kind of a sociopathic question.

Granted, I did get an offer for that job (and took it), so I guess it all worked out, but it was a very strange part of the entire interview process.

as inane as asking people their biggest weakness

"me? my biggest weakness is that I work TOO HARD!"