Item — HN Reader

I've had interviews start very positively with such an approach of questions, but then "just to be thorough" I have them code something simple on the whiteboard and they completely fail.

Some of them couldn't even get nested for loops to work correctly.

If you're hiring someone to write code for you, you need to have them write code in the interview. There's no substitute for it.

Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem.

7 comments

Years ago, I definitely was one of those people who completely bombed a code-on-a-whiteboard problem. It was exactly that - a nested for loop problem. It was one of the first real interviews as I was in a new geographic area and previously I had moved into jobs by being recruited by my professional network. This was before I started specifically studying the patterns used for engineering interviews.

Thing is, I still had a good 3-4 years of professional experience with stellar reviews, some hackathon wins, on top of having coded throughout my entire life prior. But when I was up on that whiteboard, I pretty much froze for that entire segment. Whiteboards are just a different skill and if you don't regularly code on them at work, you better buy one and practice at home because using an editor does _not_ translate to coding on a whiteboard.

I'm glad you mention this: "Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem", because I did better in interviews that followed which allowed for coding in my usual working environment. Heck, I even passed a technical screen before this company brought me on-site where I live-coded some elevator problem they asked me to implement. After bombing the whiteboard, I wouldn't blame them if they thought I had paid someone else to solve the tech screen.

This disproportionally filters for those who can think/code and perfom while under pressure/observation... i.e. the "porn stars" of software development, if you will.

Similar to how not everyone is able to perform well while having sex-on-demand in front of a camera -- these same people are more than likely able to have sex when not under pressure and observation, and probably are quite adequate while doing it.

My personal experience in software development spans decades, I'm quite competent and I have definitely "choked" at the whiteboard, just like you're describing -- for the same reasons I don't do well speaking in public, I'm not comfortable doing it, and I'm just not cut-out for it... I don't think it's a good measure of actual problem solving and/or software development competency.

An analogy that every male acctually can relate to is peeing under pressure from the que behind you.

By the same logic you might say that we shouldn't interview at all because it selects for people that are good at being interviewed. Obviously this is true but I think it would be pretty crazy to not interview people.

Frankly I think most people suck at being interviewed initially, but you can definitely improve with practice. The same as public speaking and interviewing candidates.

Some people are even good on a take-home test, good at chatting in an interview, good at coding at their desk, but completely unable to code in an interview. And you can fail an interview at some places if you are one of them.

> "porn stars"

That's.... a much more accurate term than the "rock star" that gets tossed around.

To be fair, the reddit post was about a DevOps applicant being asked a single dynamic programming questions for the interview.

It's as if you asked an auto repair mechanic about designing involute tooth profiles on fishbone pattern gears in a transmission. Sure, if they could do that, then yes they'll be a great technician, but it has nothing to do with brake jobs and oil changes.

EDIT: Per the responses in the reddit thread, the question was: "So basically it was a variation of the exact change problem (I realized afterwards). You had to validate a string of size n. 1 chunk of 8 characters could have 1 format, 1 chunk of 16 characters another, and 1 chunk of 24 characters another. You essentially had to determine if the string was valid. So in other words, find a combination of these sizes in which the string could be validated. I had 30 minutes."

We don’t know what expectations were set in the interview for the solution space, but anyone with (Unix) ops experience should have the thought “grep can do that” pop into their head. And people with a formal CS education should be able to come up with the dynamic programming solution.

So, whichever side of dev-ops you lean to, there’s a reasonable answer that should be within your field of expertise. And given Palantir’s line of business (the company involved), I wouldn’t be surprised if dissecting a byte string with limited information about it’s structure comes up from time to time in their actual work.

Nested loops are difficult to code up on a whiteboard, much easier to do on a computer with a code editor. It’s just difficult to manage the real estate, while fixing edit errors on a whiteboard is super messy (I’m left handed and constantly smudge also), easily throwing me out of my code think.

Who the heck writes code on a whiteboard for production anyways?

This doesn't Test if they can develop software; this tests if they can diffuse a bomb.

Also there's no point in bringing someone in for an interview before they've shown any coding experience. It's a colossal waste of time for everyone, including the candidate

This is something I've found as well.

We give an invite for a phone screen. It's just a chat so both candidate and interviewer can make sure that the job on offer and experience of the candidate match up. It's also a nice time for candidates to get a decent idea of what our company does so they can bow out if it's not really what they're looking for, which is good for everyone.

Then we send a tech test (take home). It's a simple challenge, with a lot of freedom. We give a guide time of ~90 minutes, but also let the candidates know that they can do more or less if they want. We ask the candidates to submit a written response as well, so they can explain if they're happy/unhappy with the test and their submission. Didn't have time to refactor as you wanted? Didn't write as many tests as you would usually do? That's fine, just let us know generally what you'd have done so we can take it into account when looking at it.

Then it's onto the face-to-face interviews etc.

The tech test segment is really enlightening. A lot of the time we can figure out a lot about the candidates ability to create software systems and communication skills just from the test. Most of the time, we spend a non-inconsequential amount of time looking at the test submissions. If a candidate was good enough to write it, we should take the time to really see what they did and review it. Sometimes we get really poor tests back, even from candidates we were enthusiastic about. Other times we get pleasantly surprised, with candidates who phone screened poorly smashing the test and giving a wonderful submission. I think it's immensely important to see what a candidate does with actually writing code. I also think it's important to make sure we see that prior to inviting them in, as they need to take time off of work, get to our offices and prepare for it. And even when we really want to progress with a candidate, the tech test gives us something to talk about to get to know the candidate better, and hopefully have a conversation that allows them to ask us questions as well.

Yes but I’m not writing code everyday on a whiteboard, I’m writing it in an IDE with access to Google.

When I am at a whiteboard I am drawing very crude architectural diagrams.

It's a great point we hear often, so I incorporated it into our last round of hiring. Some still couldn't do it with an IDE, their language of choice and access to Google.

(sorting an array, with any algorithm you want)

I must have written a couple dozen sorts, but only in high-school or when preparing for interviews. It's the one thing that is very well supported in most programming languages. C++ has stable/unstable sorts, partial sorts, whatever you want.

Accepting things like bubble sort also makes the exercise doubly pointless.

Since people still fail it, even with all might of Google at their command, even with bubblesort allowed, it's anything but pointless.

And if you are unable to sort an array it's a strong signal that command in all other areas of CS is lacking. Does C++ has a standard library for RSTP? Do you think a person who can't sort an array after 5 years of school would be able to fix a bug in a switch' spanning tree implementation?

You're looking at this the wrong way IMO. One should not give exercises and try to eliminate candidates, instead they should prepare a set of exercises so that at the end, if the candidate solves them correctly they can confidently say "this candidate knows X, Y and Z", so they should do a good job in our project.

In your case, at the end you've only figured out that your candidate knows how to sort an array, which even high-school students can do. You can confidently say that your candidate has a level somewhere between student and Donald Knuth, a pretty useless piece of information.

Is there any connection between sorting arrays and spanning trees? If I'm not mistaken, the latter involves graphs.

Problem is however that a good crop of candidates doesn't know how to sort an array, am not sure why am finding myself explaining it for the 2nd time. So all I can say is they are below high school student in your neck of the woods, even if they have Masters in relevant discipline. And if you can't sort an array with all help of Wikipedia and Stackoverflow, you would not be able to walk a graph, let alone do anything useful with it.

So I'm not quite sure what is your beef with it exactly. Do you think I should have used a more complicated task? Like, Dijkstras' on a graph? Are you sure that you wouldn't find that objectionable too, given that sorting an array is apparently asking too much?

I was saying all along that your exercise:

a) is too easy

b) doesn't have enough meat to differentiate between junior / regular / senior

c) covers very little if anything at all from what a SWE does on their day to day. OTOH I guess/hope you had more things, not just this.

Let’s say someone didn’t know any sort algorithms since he has been using one provided by the language. Would you have been willing to explain on a high level how the algorithm works? I wouldn’t be opposed to any algorithmic type question if you explain the requirements on a high level and let them do it within an IDE.

But, you are hiring for a software developer, who should be able to translate business problems that you face everyday into a working system. Don’t you have any real world scenarios that you could use to test on?

Seeing that the Bubble Sort was something I did in 6th grade in the 80s as part of an AppleSoft BASIC programming class - congratulations, the interviewer has now proven he can hire a programmer with the skillset of a middle schooler....

I would totally hire a middle schooler who can program over a PhD who can't. That said the guy we hired did an insertion sort.

It depends on the position. If I’m looking for a developer for yet another software as a service CRUD app, I may not need anyone that understands “CS fundamentals”. Heck it’s been 20+ years since I graduated from college with my CS degree and over a decade since I’ve had to do any low level programming.

Does C++ has a standard library for RSTP?

Maybe not, but I know that there are very few new problems under the sun that most people face and someone has solved the problem before. I also know how to use Google.

https://stackoverflow.com/questions/1460010/best-c-rtp-rtsp-...

You mixed up RSTP with RTSP, I would revisit my Googling skills if I were you.. making a beautiful case though for this kind of interview process.

(before you go on googling again, the algorithm operates over the switch port forwarding fabric, you can't really use a library)

So are you really saying that you could not look at an open source implementation and reimplement?

https://github.com/shemminger/RSTP

I partially agree, but a bubble sort is slightly harder than FizzBuzz (?) and even if you give them the algorithm, it might be a little more fair since knowing how to do a nested for loop is less esoteric than the mod operator.

Now if someone did a BogoSort, I would hire them just for their ingenuity and sense of humor.

10/10 I would go for randomize the array and check if it's sorted. Repeat if it's not.

I like the people I work with to have a good sense of humor.

Would using a built in library be acceptable? Actually in C knowing how to use qsort() does require having a little knowledge of how function pointers work. If they did, I would ask them what could be some drawbacks of using the built in qsort function (function call overhead).

On the other hand, would you accept a bubble sort as a solution? Or the most degenerate case- a bogosort ;) ?

Bubble sort, bogosort, sleepsort, anything that solves the assignment (gets an array sorted) of course. And no, a library call would not count towards implementation, but it's not too much to ask from candidates with masters degrees.

I went through a similar interview once - bring your own laptop, and code-up a bowling scorer which spanned two sets of interviewees. Was not the worst interview experience I've had.