> How could the candidate have started coding in one language, then decided actually they don’t know that language and should start over from scratch in a different one? How could someone with 10 years experience on their CV be unable to start a new project in their own editor? How could the candidate spend thirty minutes trying to generate a random number and still fail? This is crazy!
They panicked.
I would like to see a large company trying a replication study on their interview practice with their current employees. You'd need to make sure that the test was significantly different from the one used to hire them, and that they were put under sufficient economic and social psychological pressure - perhaps tell them that if they fail they will be fired and immediately escorted from the building (+), then make them do it with the CEO sitting behind them. I would expect over a 10% failure rate, including some people who are normally regarded as business-critical important team players.
You then have to hope that people lean the lesson "interviews are a poor predictor of performance" and not "some of our people are really imposters"..
(+) obviously don't actually do this, but like fire drills you need to convince people that it might be a real fire sometimes. I am also aware that there are ethics problems with performing stress experiments on employees without their consent, although nobody seems to care about this for interviewees
I would expect a far higher failure rate, maybe closer to 80-90%, especially at a successful company with happy employees.
The engineers will be focused on building actual systems and be totally unprepared for technical interviewing.
Engineers great in one domain will be interviewed by those in different areas with different views on what constitutes “basic” knowledge.
None of their work history within the company can be used as part of the evaluation—you must simulate the problem where nearly all your work is locked up behind an NDA.
> I would like to see a large company trying a replication study on their interview practice with their current employees.
IIRC Google did that and the results showed that more than half of employees wouldn't pass the interview. I can't really find the links for that bc I heard that from close friends.
They panicked.
I would like to see a large company trying a replication study on their interview practice with their current employees. You'd need to make sure that the test was significantly different from the one used to hire them, and that they were put under sufficient economic and social psychological pressure - perhaps tell them that if they fail they will be fired and immediately escorted from the building (+), then make them do it with the CEO sitting behind them. I would expect over a 10% failure rate, including some people who are normally regarded as business-critical important team players.
You then have to hope that people lean the lesson "interviews are a poor predictor of performance" and not "some of our people are really imposters"..
(+) obviously don't actually do this, but like fire drills you need to convince people that it might be a real fire sometimes. I am also aware that there are ethics problems with performing stress experiments on employees without their consent, although nobody seems to care about this for interviewees