Unfortunately, those kinds of interviews also select for some other things that they shouldn't.
* Youth. People who have very recently studied these things in school, and use the same languages as the interviewers, have an advantage.
* Free time. People who have families (for example) might have less free time to study "Cracking the Code Interview" and such.
* Absence of anxiety. This disadvantages women, minorities, and people with psychological conditions that should be covered by ADA. Also, people whose financial situation is precarious will be more anxious than those who don't need the job, independent of which is actually a better candidate.
* Conformity. People who can recognize the flaws in a measurement technique, and who have the strength of character to push back against its application - both good qualities for a candidate - will self select out.
There's a lot of overlap among these, of course. There are better ways to measure "general IQ" and "grit" (which are both questionable concepts anyway). I've passed every such interview I've ever taken, but I refuse to administer them (despite the fact that my refusal has carried a quite tangible cost) because whatever benefit they provide is outweighed by their many flaws.
> Resistance to anxiety. This disadvantages women, minorities, and people with psychological conditions that should be covered by ADA.
I resemble some of those categories, and I don't know if I would feel comfortable making the leap to correlate them to a some inherent reduced level of resistance to anxiety. That seems like a generalization which I feel that, on an aggregate level, seems unsupportable by data.
I think that determination should be on a case-by-case basis, as is currently done at universities.
Allow me to elaborate, then. There's an inherent power dynamic in interviews, which creates stress in the interviewees. That effect is magnified for anyone who is unlike their interviewers, who still tend to be white and male. It's magnified still further when the power dynamic within the interview reflect the one that - very unfortunately - still persists in society at large. Lastly, the funny thing about stress/anxiety is that it tends to be additive. Having experienced high levels recently (including in other interviews) makes one more susceptible to new triggers. Thus, anyone who is the least bit "outside" in any way will feel more anxious.
Is any of that even controversial enough to require citation? How many Psychology or Sociology 101 textbooks should I cite? It's easy enough for those who don't feel this kind of anxiety themselves to brush it off, but for those who are less fortunate all those magnifiers can create an anxiety level that's quite debilitating.
I can see what you're saying (in good faith). I do think interview anxiety is a collocation of multiple factors. I don't think the correlation of anxiety to the listed categories are easily deconvolved from other factors -- it might seem like a common-sense correlation, especially if you start from certain priors, but it's a big assumption to make on average.
(Only this past week I had experiences that challenged my assumptions about certain demographics (seniors) and how I would expect they would behave, and how they actually behaved. The lesson I learned was -- don't assume, always collect real data)
The white-male interviewer power dynamic has some basis in reality (I've experienced it occasionally, not all the time), but its effect on my interview performance may be less than 10%? (to throw out a number). I find I'm much more affected -- maybe 90% -- about (1) my competence in the subject matter and (2) how well practiced I am (for instance, I know the theory for a great many subjects but am unpracticed at some of them, so I tend to stumble and lack ease when it comes to demonstrating my subject matter knowledge in real time).
For different people, those percentages shift, and I believe in a way that is not obviously or necessarily correlated with their demographic (psychological conditions, yes, but also depends on which ones -- some don't affect anxiety). But all I have is anecdata so I'm not able to provide strong evidence one way or another so this is just my two cents worth -- and I do mean this in good faith.
I appreciate your willingness to continue this conversation in a constructive way. So let me ask you a question. You say the white-male interviewer power dynamic might affect your interview performance by 10% or less. (Personally I'd pick an even lower number, but then I'm a white male myself and also old enough that I've been senior to most of my interviewers for a while now.) Would a 10% difference in interview performance be enough, on average, to affect who gets offers or what those offers are?
My basic point here is that even 1% would be too much. If an interview process creates any inherent disadvantage for some groups, I'd say it deserves serious scrutiny. BTW, by "inherent" I mean beyond what can be addressed by bias training and such. That can enable interviewers to conduct any type of interview in as fair and kind a way as possible, but not to change the interview structure itself. If the structure is the problem, training isn't the solution. I think the in-person white-board algorithm interview is unavoidably weighted toward factors that have nothing to do with likely on-the-job performance, and thus should be avoided.
I'm probably not the best person to gauge that in general, but I think the "inherent" part is where we differ.
Going back to whiteboarding: I don't like whiteboarding myself, but to be fair it does measure certain dimensions quickness-on-feet, memorization abilities, ability to exude presence, fluency in language, etc. While these are laudable abilities on their own, I agree they might not correlate with overall on-the-job performance (but it depends on the job).
I guess "job performance" is this amorphous latent variable y that is correlated with a bunch of direct predictors x which we can measure, u which we can't measure, so we use proxy variables z to stand-in for them: y = f(x, u, z).
The worry is that some candidates, who may be bad for the job, but just happen to be good at these proxy dimensions (or train for them) might get the job; on the flip side, we may exclude certain candidates who are potentially good for the job but perform badly at the proxy dimensions. Whiteboarding measures the proxy dimension z.
Edit: oh look, an article on HN's front page on this very issue:
>Is any of that even controversial enough to require citation?
I'm sorry, excuse me? Are you saying that non-minority, non-women don't suffer anxiety? Your parent comment certainly seems to suggest that. Which, at a minimum, is flat out wrong. Educate yourself[0]. And then zoom out and ask yourself why it's not only permissible, but often lauded, to so flippantly say what you just said.
> I'm sorry, excuse me? Are you saying that non-minority, non-women don't suffer anxiety?
To be fair, OP did say "That effect is magnified for anyone who is unlike their interviewers"
But, I do agree some evidence from OP would help their claims.
I wouldn't be surprised if minorities experience more anxiety, on average, in situations like an interview, though. I think it's established that imposter syndrome is more frequently encountered for example but it's too late here to go digging for evidence :)
Stop dehumanizing an entire group of people. All humans feel anxiety and that's a fact. Excluding one group of people that is equally likely to experience it because it's currently in vogue is still discriminatory.
Yeah I’ve never heard of women and minorities having less resistance to anxiety as a group. I’m curious where that’s coming from. Certain psychological conditions I could see, but that’s such a broad category that I think that’s also an over generalization.
I don’t know about “resistance to anxiety”, but it can certainly be anxiety-inducing to be the only member of a given minority in the interviewing room (or entire floor, as is sometimes the case in tech).
What if qualities you might judge in an interview other than “general IQ” and “grit” are even more bias-prone, and using those other qualities is actually worse than measuring “general IQ” and “grit” and applying a corrective factor?
In other words, I’d rather work with a disadvantaged person who may appear rough around the edges but made it this far and has the “general IQ” and “grit” to do well on the programming problem, than the preppy white kid who has a lifetime of experience preparing for the task of exuding status and competency when answering behavioral questions or engineering case studies, but lacks the “general IQ” and “grit” to solve the programming problem.
Just curious, what are some examples of better ways to measure general IQ and grit? You said they are questionable concepts, so how do _you_ interview people?
I don't, any more. I'm at a company that enforces a very rigid structure that I don't agree with, so I simply opt out. I pay for it every review cycle. Ironically, the rigidity of that structure is explicitly intended to reduce personal bias, which is a laudable goal, and I believe it succeeds. Unfortunately, I think it just replaces personal bias with systemic bias.
When I did interview, which was a lot at times when I was in a leadership role at a couple of startups, my favorite interview technique was to let the candidate lead and I'd follow. If I wanted to ask about algorithms, I'd ask about one they'd used in a project they'd worked on. How did it work? What were its strengths and pitfalls? What others were considered? What bugs were found in its implementation, or caused by its use? Besides flipping the control dynamic of the interview, it often led to more interesting conversations. Highly recommend.
> what are some examples of better ways to measure general IQ and grit
IQ has been under a shadow since _The Bell Curve_ and I'm not keen on letting it back out. ;) If one must measure it, I'd say measure it directly with simple challenges (e.g. memory or pattern completion) or puzzles ... but even those are apparently fraught with cultural baggage and of questionable relevance to a knowledge-heavy domain like programming.
As for grit, it's often readily apparent from someone's resume. Were they self-taught, worked through college, or took a free ride? Did they stay with companies and projects through hard times and get promoted "in the field" or were they always the first rat to abandon ship? It usually only takes a few questions to figure out whether someone's a coaster or a fighter. Funnily enough, the people with the most actual evidence of grit are the ones least likely to have spent their time studying specifically for the interview. They were busy actually doing stuff.
* Youth. People who have very recently studied these things in school, and use the same languages as the interviewers, have an advantage.
* Free time. People who have families (for example) might have less free time to study "Cracking the Code Interview" and such.
* Absence of anxiety. This disadvantages women, minorities, and people with psychological conditions that should be covered by ADA. Also, people whose financial situation is precarious will be more anxious than those who don't need the job, independent of which is actually a better candidate.
* Conformity. People who can recognize the flaws in a measurement technique, and who have the strength of character to push back against its application - both good qualities for a candidate - will self select out.
There's a lot of overlap among these, of course. There are better ways to measure "general IQ" and "grit" (which are both questionable concepts anyway). I've passed every such interview I've ever taken, but I refuse to administer them (despite the fact that my refusal has carried a quite tangible cost) because whatever benefit they provide is outweighed by their many flaws.