The Last Technical Interview

Signals I've got from this post:

- Steve's company got acquihired by Amazon, granting him a free ticket in without all the torment of the multi-stage interview pipeline.

- It's a well known fact that it's easier to jump from one FAANG to another, so while interviewing at Google he had significant advantage, plus the blog gaining popularity.

- All of this has caused a deep down imposter syndrome, which resulted in an attempt to "improve" an interview process from the inside - but the wings were clipped pretty quickly by the corporate politics. It turns out that lawyers are not planning to reinvent anything there and hence are somewhat more important than engineers.

- The post itself is an self-applause over essentially a failed effort. "I've tried"

The "provisional employment" idea sounds good at first, until you think about how it would actually work in practice. You have 100 applicants for 1 position. Which one do you provisionally hire?

Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment.

You haven't replaced anything, you've just added another level of hassle for everyone.

The gold standard in hiring qualification is work-sample testing. It works fine. You do not need to "make hiring a profit center" or "provisionally hire" or do internships. Work samples done correctly demand less time from candidates than interviews and scale better than interviews. They are standardizable and iterable.

What I feel like I'm reading here is someone who has been poisoned by FAANG hiring practices --- and they are terrible --- and has missed most of the work that's been done (outside of Google's admirable work in debunking their own processes).

I appreciate the "kitchen confidential" here, but with respect to Yegge, I think he's been working at the Olive Garden this whole time. Go stage at Gramercy Tavern! They're working at a different scale, yes, but you'll at least get a different perspective on the "gold standard".

This kind of thinking is why a lot of market place startups fail.

You have two parties engaging with each other in a subpar way, and your solution is to make things better for one party (hiring side) and significantly worse for the other (candidates). Trying to convince candidates that this is good for them, won't make it so. Eg:

> Every stamp that you hand out, pass or fail, leaves a candidate richer than they showed up. This attracts strong candidates to you, because even your rejections are worth something to them.

Candidates don't want stamps. They want stable work.

I gave the feedback at one Google interview that they should send Google employees through to see how many get hired. Good to see they basically tried that.

The conclusion at the end that bringing someone on board is the ideal method is true I'm sure, but even that runs into the issue that employee evaluation is an even worse situation than the interview process.

You can openly see some managers panic when they realize they have no idea what their employees have been doing for the last 6-12 months when they're asked to provide feedback.

I couldn't find the text of this joke, attributed to Dirac. I'll paraphrase.

A man walks into a pet store. There's a parrot for $100, it says this parrot speaks perfect English . The one next to it is $1,000, and says, This parrot speaks 12 languages fluently.

Then there's a bedraggled looking, droopy, parrot, and its label simply says One Million Dollars.

Does it sing opera and has successfully run for President? the man asks with a sneer.

This parrot, says the store owner, _thinks_.

That's what this entire post is about - how to evaluate people with a series of attributes, score, correlate, blah blah blah.

Hire them, see if they think. If they don't, fire them. It's cheaper than this credential/signal rigmarole, most of which is about CYA legal b+llsh1t. Yes, it's a simplistic strategy and it doesn't work for Shoogle, Banthropic, Goober, whatever. You know what, boo f*cking hoo. You're a trillion dollar company, suck it up. You have a zombie horde at your doors and you're just upset the "true gems" are hard for you to spot amongst the slavering masses. You're going to heartlessly lay them off anyway in a few years. You SHOULD feel this pain and anguish of having to sort through them, constantly regretting all your choices. That's the only way to have balance in the Universe.

It's funny that the post states: "None of these band-aids really help — we still all hire tons of false positives (unqualified) and turn away false negatives (actually qualified), despite every attempt to make the process perfect, or even good"

The post touches on it (regulation about firing causing legal issues) but the unfortunate truth is that turning away false negatives has little cost to a business as long as you make sure to turn away false positives. Bad hires are extremely expensive, where as the lost revenue of turning down a good hire isn't nearly as bad (it's easy to try to hire them again later).

The stamp idea is good, reputation based hiring is important, as shown by the use of referrals in hiring. I've always wondered why we don't keep track of referrals inside companies in order to figure out who is recommending their peers who legitametly are good fits, vs those who are just trying to help out friends (who might not be the best hires).

Further I think there is also the problem that anti-corruption/anti-nepotism often harms hiring. It's much easier when working in a start up to hire the most talented people you know without other people sitting in, compared to at a big company where I know people who actively know good people they want to hire, but who can't be arsed with the long interview process and high chance of rejection.

If it wasn't for the regulation around firing some sort of staking your job on the line would be effective. You can refer someone and have them skip most of the process but your job is on the line if that was a bad call.

I don't understand how a failing stamp for a campfire is good for the interviewee. It signals that they weren't good enough to get hired. Why would they want to parade that around?

How about instead of a contrived take-home project or doing unpaid "real work" for the company, we ask candidates to do n hours of work on an open source project?

That's real useful work, if we don't hire them they can show it off to a different hiring company, and it makes the world a better place.

The most talent dense place I've worked had a dead simple process - two one hour chats, one with your potential manager/team and one with the CTO or CEO. If things didn't work out, well you got sent away, probably happened twice per month. There was a particular meeting the CTO/CEO used and if you saw someone meeting with them there on a Friday afternoon, you would not see that person on Monday.

The place was not big - never got much beyond 100 engineers, but produced dozens of founders, VPs, GMs etc at well known companies, as well as engineers with very notable OS projects and lots of high-placed engineering in FAANG companies.

I'm skeptical of the idea that work can be consistently decomposed in such a way that an outsider can be effective immediately on anything other than toy problems. Saying "they can point their agent at it" is handwavy.. it helps but any mature codebase is still going to take weeks to get up to speed on, and there's going to be gaps that an agent can't fill because the knowledge isn't in the codebase. This seems like it would only work for mostly green-field projects that don't require a lot of expertise. Even things like "how do I use the product" can take a couple days to get up to speed on.

I also think this is pretty bad for candidates. The last thing I want to do is a bunch of unstable gig work for fickle startups in one of the most expensive places to live in the world. Also, I really think this only works if you decouple things like health insurance from employment, otherwise the instability can be too much for a lot of people.

I see this kind of “let’s make candidates work for free” proposal from time to time.

It doesn’t work for software engineers because nobody wants to do free work. Also, free work isn’t consistently, and fairly testable because project requirements can evolve or completely change over a hiring period. Not to mention IP issues.

What has worked in my experience is a synthetic take home exercise that isn’t easily LLM solvable, the same one given to all candidates with the same constraints, and offering to be available to answer their questions by email. Sometimes we can tell a lot about the candidate just from the questions they ask, or how they package the solution, without even looking at the code.

In my experience (FANG hiring) less than 10% of candidates refuse to do this exercise, so it’s worked well for us.

The paradox that strikes me is that "hiring is broken" yet these companies are beyond successful. So there's still yet another layer of something in between observing that employees are capable / incapable, and the company successful / unsuccessful.

Seems a bit detached from the real world.

If I am doing a 6 month contract which is what he is proposing then yeah I want a great day rate and to earn 12 months salary in those 6.

It basically means I invest in a company that doesn't believe in me and probably wont try to help me succeed.

Also mortgages, car loans, life and health insurance etc. are harder to get on a short contract.

Hiring is broken (maybe) but 6 month paid interviews are not the solution.

I don't know if it's the same in the US, but every job in Australia generally comes with a 3-6 month probationary period where employment can be terminated for basically any (non-protected) reason with little notice. I've observed that the option to do so is seldom used - probably because there is not usually much incentives for managers to decrease their number of reports unless there's serious problems.

Most places still interview. Because hiring someone for 3-6 months is still an expense. For large companies, onboarding can take many weeks before you're even thinking about being productive. Interviews don't need to be 100% accurate. They need to be time-efficient and an ok filter to prevent hiring the worst people.

I don't see how a bad job market is going to make skilled people be willing to intern in order to get employment. Employment is a market, if demand goes down, then the reaction will be that price goes down.

The reason students are willing to intern is because the supply is so high and demand so low that you can effectively hire them for nothing (or next to nothing). The interns know that on completion the internship will upgrade them to a new category with new supply/demand. It's the same reason they are willing to pay large amounts for a university degree.

So interviewing will stay the same. If demand collapses, we will see wages drop, and I imagine the supply of software people will react through early retirement, career switching and reduction of people choosing it as a career before we see an upheaval of hiring methods.

The thing is the internship or code campfire or probation or whatever all need the hiree to stop doing what they were doing. It’s incredibly intrusive. Lots of the best candidates have jobs and families and aren’t going to give up their current job to try out for a company that’s not committing. The whole post is so biased to the hirer.

From my POV, the main thing that's really broken with interviewing right now is the filtering process, even before candidates do a take-home test.

In the last few years I was the main tech interviewer for a 300-employee fintech.

For a specific position, one recruiter got around 150 applicants, selected 5 great ones, who did take-home tests and mild-tech interviews. Offers were made to most.

For the same role/salary, but from another queue, a second recruiter got around 900 applicants, cherry-picked about around 70 of them. Out of those, only 40 completed the 1h take-home test. Only 20 delivered it, only 10 implemented the requirements. Of the 10, all were unable to answer even basic questions.

This was concurrently, so it wasn't "affected by AI".

I didn't changed my methods and in fact I didn't even got close to asking hardball questions to the second group.

The second recruiter didn't get their contract renewed and left.

> There is a material difference between the signal from an internship (~7 weeks of usable work time after ramp-up) vs a co-op (5 months actually working)

And that's why the trades require you to perform years of work as an apprentice before you're ever qualified to pass an exam to become a journeyman. Not only is it like a years long internship, but you're paired with someone who has already passed a high bar. You're learning directly from vetted people, and those people can vouch for you.

> University of Waterloo famously sends their Computer Science students through a total of six internships, giving them roughly 2 years of real-world work experience before they graduate.

Still less time than an apprentice is required to work, but it's better than nothing. Most trades require you to be an apprentice for 4 to 5 years.

> The reason is, hiring engineers has historically been so competitive that you couldn’t convince a senior engineer to do an internship [...] So the industry converged on not requiring it.

Which is one reason the trades made it the law that you have to follow the apprenticeship in order to become a journeyman. "The contracting companies don't feel like hiring licensed workers" isn't an option. Companies don't do the right thing unless they're forced to.

This is why we need professional engineer certification for software engineers. We need a rigorous, time-tested, reliable process to ensure engineers have actually done the job in the right way. Otherwise it's a bad guessing game like Steve is explaining.

This is also why we need a software building code. We need to explicitly define exactly how you're supposed to engineer, so that we can create a certification that people can pass. Otherwise designing and building software is completely subjective. Engineering should not be subjective.

This is not some mysterious experimental idea that nobody knows if it'll work. Trades have been doing this for decades. It's not perfect, but it's much better than the alternative.

Steve Yegge is one of my very favorite authors I love his work. But lots of things to say here.

FIRST - is that before you get to campfire anyone, you have to have done some sort of interview process to boil it down to the one person to do the campfire - so how does that work eh?

SECOND - campfire is deeply invasive to the candidate's life and time.

THIRD - you have to pretty damn sure someone is a hire before doing a campfire. You CANNOT do campfire as an evaluation step, after which there are more interviews.

FOURTH - this is effectively just a really really long version of the take home work test which is absolute bullshit.

FIFTH - there's STILL no science to the campfire. Don't give anyone a fucking test if you don't actually know how to scientifically evaluate the results. And campfire does NOT result in a scientific outcome, it still results in an arbitrary opinion.

SIXTH - any company that wants me to do a campfire - to commit days or even weeks of work as part of them trying to decide to offer me a job - can fuck off. Sorry, the party got spoiled by all the other companies who asked me to do something as part of the interview process and then either ghosted me or gave me some bullshit outcome like "they didn't like your work".

I can tell you how to recruit people and it does not require campfire.

You TALK to people about software development - you engage them in extended conversation about what they have done, what they know, what their interests are, what they have built, what projects they worked on, what went right, what went wrong. You look for people who have BUILT STUFF - this was true before AI and is 100X more true now - anyone who has not built anything today is not worth employing and anyone who has built something must be able to talk about it in depth. This interview processes worked before AI and it works after AI. And finally, you accept the limitations of recruiting which is that people are people and you won't find out how well someone performs until they have been on the job six months - live with it.

Sorry Steve - I love your work but I'd never work for any company that wanted me to do a stupid campfire because they don't know how to actually work out if I can do the job or not.

here's the uncomfortable truth. most software engineers are good enough to hire.

i've seen a few interview types in my time:

1. technical interviews run by nontechnical or junior engineers who can't judge technical talent. this typically produces sub-par hires. imagine you are made the director of UX design and you have to hire someone but know little about the industry. what type of hires are you going to get?

2. technical interviews that focus on CS skills. can you write a red-black tree in Java? can you write a bubble sort in haskell? The problems here is that this has bias towards new CS grads that dont have much industry experience but just took tests on stuff like this in their degree. google pioneered this style of interview and it's taken off as leetcode. the problem is that experienced devs dont typically write bubblesort algorithms. this type of interview is biased toward younger out of college hires and codecamp hires that have little experience but gring on leetcode sites.

3. technical interviews that have too many opinions for a hire recommendation. this seems to be a new trend where you need 3 or 5 or 7 thumbs up to get hired and any negatives sink the candidate. this does raise the bar, but typically what i see is someone on this panel has a high opinion of his/her abilities and gives everyone they seem a thumbs down.

What i've found is that soft skills and team dynamics are > the technical accumen. I can teach you how to write go(lang) but its much harder to teach you how to influence your peers or to communicate effectively.

Experienced devs probably dont know how to ace your leetcode interview, but they do know how to influence and communicate and when to stay away from bad ideas. You dont get tested on these things. It's analogous to a being combat veteran. You've been there, you've done that you know how to survive. You might not know the new tech, but that can be taught. I have never seen an dev not be able to pickup a new platform/language/skill within 6 months of hire - anywhere.

Here's hoping this will get better one day but i'm not holding by breath.

A lot in here about Google, which hilariously has done a bunch of studies that concluded it's hiring process is awful (is sure is!) and the takeaway is that candidate evaluation is impossible, not that Google in particular does it badly.

That discussion triggered a memory from a comment I made a long time ago here (2020, found it: https://news.ycombinator.com/item?id=24841235).

There is a way to make the campfire approach a little bit less utopian: California could pass a law that makes it legal to have a second temporary job. So that FAANG engineers for instance, would be allowed to campfire/interview at another company with legal protections. Put a bit of grease in the interview system is good for candidates.

Internship etc. proposals miss that status quo job seeking is a parallel process for applicants. An internship model means each candidate can only “consider” one employer at a time. I think this is what Steve’s campfire proposal is trying to counter by making samples/internship outputs public?

I think the real solution is something like the Bar Exam. Apply the hard filter once, publicly, administered by a trusted neutral party. Anyone that gets through is assumed to be good enough, and interviews can assess only firm/team specifics

>> None of these band-aids really help — we still all hire tons of false positives (unqualified) and turn away false negatives (actually qualified), despite every attempt to make the process perfect, or even good.

Yeah and you know why? Because like everyone in the industry you make the mistake of trying to hire engineers at the level of expertise and with the knowledge and skills you think they need to do the job. Except, any engineer worth their salary (i.e. their salt, I guess) is going to grow into any job and learn new skills and new technologies that they need to do the job, so the person you really want to hire is someone who is not yet ready to do the job you want them to do but has the potential to learn it really, really well. And the engineers you really hire, if they already have the knowledge and skills that you want them to learn, won't learn them because they already know them, will be bored to hell with the role, and as a result do a sloppy job and move on quickly to another role that is more challenging. That is, if they like to be challenged. If they don't like to be challenged- congratulations: you hired yourself a boring engineer.

That's what's wrong with your hiring practice. And you have no way to fix it because you keep telling yourself you want to hire "talent" but what you really try to hire is "experience". If you hired for talent, you'd get both talent and experience, but if you hire for experience, you often end up getting neither.

Incidentally, my pet theory is that this explains why 90% of modern software is crud.

Lots of replies here pointing out that that any kind of "provisional employment" wouldn't work for the majority of candidates who are currently working.

From the candidate's POV what is mostly broken about the application process is all of the ATS gaming, resume tweaking, etc, just to avoid getting filtered out before an interview, as well as the inane leetcode screening that some companies are doing.

A better process might be to replace all initial profile/resume-based screening with task-based evaluation (evaluation could be at least semi-automated - if a computer/AI is going to reject me, I'd prefer it to be based on task evaluation job skills rather than ATS-filter avoidance skills!).

Lengthier on-site interviews/evaluations could also be task-based - for a developer role perhaps a 2-4hr peer-programming or problem solving task. Far less signal that a 3 month provisional hire of course, but maybe a better use of everyone's time than a traditional talk and brief whiteboard challenge process which is clearly failing as a useful filter.

I wouldn't expect companies to forgo the traditional touchy-feely team/culture fit type of screening as well, but better if this came after they'd already determined, as best they can, whether you've got the chops to actually do the job well.

Maybe I'm an oddball, but I've always thought this: if a cool company wants to hire me but isn't sure, give me a 3-6 month fixed contract and LFG. There's zero doubt in my mind it would work out. And if not, so what, I'll be okay.

Today I'm 45 with family and have a fancy VP title, but I would have no problem to do this for an interesting role at a cool company.

The industry should get down from its egocentric ivory tower and start hiring like other industries. You are not special, tech. Just have a sort of bar exam you have to take every x amount of years. The actual interview should just be an in-person behavioral one, with your future boss. Period. Don't get enough signal from that? Sorry, that's life, taking risks. Imagine trying to open a business without taking risks, and only kickstart it once you are a 100% sure it will work. No other industry is as delusional. Here is a reality pill: what you are asking, a 100% safe investment, is impossible, as any economist will tell you. "Campfire", give me a break, grown up adults with families don't have time for that Silicon Valley wankery, and you have no evidence that the results will be as good and unbiased as you think they'll be...thanks for bringing attention to the subject though, and in passsing also confirming how garbage the Google (and FAANG, in general) hiring process really is.

Just one more thing: the industry should really put a bit more weight on measuring people's potential and the concept of long-term growing and learning on the job. Just saying. You know, like every other industry on this planet.

Seconded, stop the theatrics and gatekeeping and let's keep a growth mindset while training / retraining those who 'pass the buck' overtime. At least everyone can get skills and talented outliers will find themselves with more structure and collectively we'll produce better outcomes for more engineers at multiple levels of experience.

His idea has some merit but will require the old system to completely crash out before anything new will be considered and I'm not sure if it will crash or just keep limping along. If it really does crash out hopefully we will see multiple new strategies emerge as there are many possible options once the current one is off the table.

> Another reason is that on the supply side, nobody wants to sign up to do a bunch of free work just to be rejected. If you just put up work, the candidates incur all the risk, meaning they walk away with nothing if you don’t hire them.

It's true, but prepping for a typical senior+ onsite loop in big tech still requires weeks of grinding leetcode, re-learning the latest system interview questions and the system interview answer framework, refreshing and rehearsing STAR stories, studying the company and its unique quirks that you're expected to know to pass the culture filter, remembering how to do all of this speedrun-style since you only get 40ish minutes per session, etc.

While that knowledge is more reusable across onsites, it's likely even more work than doing real or pretend-work for the company for a couple of days.

> When candidates get to walk away with something of lasting value that they can keep forever

I'm curious why them getting rejected from the position, even with the work sample they can carry away with them, wouldn't be still interpreted as a negative from future employers. "The other co passed on them, am I the fool for thinking they're good?" type of herd mentality which is often unavoidable.

Won't that "work sample guest book" be treated as the list of all companies that rejected you, a net negative for your personal brand you're projecting?

> (Me paraphrasing what Steve was implying) Take-homes are impacted by AI one-shotting them for candidates

I've been pleasantly surprised by how much you can glean from having the candidate upload their conversation log with the coding agent for whatever take-home you give them.

The solution would seem obvious - decide what you want, then hire the first applicant that qualifies.

Of course, the real issue is that a prospective employer doesn’t know what they want. Our “engineering” industry runs on vibes, heroic effort, fashion and hype-cycles. Someone very smart, enthusiastic, quick learning, flexible and affable is the only real job description.

Combine this with the “supply” side entirely overwhelming demand, having to compete with AI, and experience having no value, and for most software developers the future is bleak until a new more equitable equilibrium is found.

Nevertheless in the meantime, at least one can have solace in collecting the authors “failure stamps”.

Well, campfire method sounds easy in the blog post only.

You need an NDA from a candidate (many would not like to risk - what if someone gets job on the competing company and is sued for revealing some secrets from the campfire interview process).

In many cases interviewee would need to get a laptop from the company, as there are specific requirements about data security (disc encryption, usage of solutions like Fortinet or Zscaler), be added to company SAP to get access to the resources (Office, Teams, etc.), company need to purchase licenses for Office, Github, Jira, etc.

Surely hiring is hard, surely there are false positives and false negatives, but fixing this requires hell a lot of resources and organizational changes and costs.

We have been working on https://talentpulsar.ai for exactly this. Hope to find some good collaborations with the hn community with some little self promotion, hope this is okay.

I like take home projects the candidates then need to present and answer questions about. LLMs just mean you can be more ambitious here (though you should pay for their tokens).

The problem with provisional employment is that it can take quite awhile for a new hire to be productive. In a complex FAANG environment I'd wager it's about six months before they're not a net drag on the team and maybe 1-2 years until they're close to fully productive. These are complex not just technologically but organizationally and there are tons of hidden rules and micro-decisions that are the difference between a project stalling vs moving forward.

This just generally over-estimates whether you can "measure" good work on scale, and make your org produce more of it. Google didn't manage to measure and predict its own employees' work [0]. Even with 6 months probation – the default in Germany – it's just guess work.

If you want good employees, poach those that others are paying millions already.

[0] https://www.nytimes.com/2016/02/28/magazine/what-google-lear...

It is really not that difficult.

1. For juniors, any sort of proof you have passion is enough. 2. Treat mid-levels as seniors. 3. Seniors have to show proof of passion (with longevity and intensity, AI and consultants exist for expertise and menial work) and competence (Take home problems with a long time scale with whatever tools they need. Use AI. I don't care, but be expected to be scrutinized for your design decisions, depth of exploration, and architectural write-ups.)

I don't think this works, and other comments have also pointed out the same. You open a position, you receive 200 CVs. Ok, then what, get them all to come in for a couple days work? A company that needs only one position can't possible handle that.

Now you're amazon/google. You have 200 open positions in a certain site/country. You receive 10000 CVs. Same problem, different scale.

So ok, you need to filter CVs, mmm, which sucks, right, so perhaps we do a screen interview? mmm, low signal, maybe....

The guy looks like to be a kind of cancer of all that is going back with hiring in FAANG.

He is indirectly responsible or at least part of everything that was terrible in most of the big techs he was part off. By his own admission, each time things were evaluated, the scientific conclusion was that it was "horse shit" and totally randomly hiring results in the end despite having candidates go through an awful process. And now he is full of ideas about what kind of new nightmare we can create to reach the same stupid result...

What I see in his post, and that I find despicable is that it looks like that he has this "superiority" syndrom making him consider candidates and employees as disposable resources.

Instead of considering the process as a "mutual" process, where it costs and the investment on joining the company is also from the candidate side, he is considering that just the employee is a liability for the corp and that the candidate should submit fully to it. Even wasting weeks, month or years without guarantee of stability or not wasting his time.

I’ve seen this before. The result is a revolving door of temporary work. Because soon companies will be “you already put 6 months in, we don’t have enough signal work 6 more months” and at the year mark they swap you for a new candidate.

And before you say that this is inefficient consider that despite being terrible for morale and efficiency (proven in un’etica studies) companies still maintain the bottom 10% out or up or out policies.

Companies always love their power on their employee over efficiency.

Most interviews are just referrals then confirming the bias, even if you fail some questions, you are likely in. If you came from no where, they are not gonna like you unless you get all the questions right.

If you want to know if someone is good at your company in 3-4 interviews, it’s tough, the best they can do is ask these technical questions. Talk to you about your past work, ask you technical what ifs. Most dumb ass companies will ask you to do trick coding leet code crap.