The Ultimate Guide on Technical Recruiting — Part 2
In case you’re joining us late, here are the links to the other two sections.
Part 1 — Philosophy & Planning
Part 3 — Sourcing, Closing, Calibrating
SECTION 2: PROCESS & STRUCTURE
In this section, we’ll break down the core pieces that makes up an interview structure. This is the system that will need continual calibration as the company scales. If optimized correctly, it will save financial, engineering and recruiting resources and lead to hiring a great team.
Buckle your seat belts, this is a pretty thorough section.
INTAKE MEETING
What: 30min ‘interview’ between recruiter and manager to understand the role, help create the pitch and flesh out the process.
Topics to cover:
Pitch:
Highlights of manager’s background — how did you get here, why are you here, what did you do before?
Investors — Who are they? What successful exists have they had? What do they bring to the table?
Team Growth — team now, team in 6 months, team in a year, team long term
Responsibilities — What are some exciting projects candidates can look forward to? Why would someone want to leave something comfortable to do this?
Product — What makes it so attractive? Why would someone want to build this out?
Technology — Stack? Any new framework, language, tool that would be attractive?
Team Charter/Mission — What’s the objective of this team? How does it relate to the overall company vision?
Reservations — What are some reasons why someone would choose a different opportunity over this one? What risks, what weak points should we call out upfront so it’s not a surprise later on?
Qualifications:
Background — Target companies? Github repo’s to dig into? Years of experience?
Keywords? What would separate a good candidate from a great one? Things to avoid?
Examples of profiles? People on team, ‘dream’ past colleague/engineer you know that you’d want?
What job titles are used in the industry?
Screening questions — Are there any particular questions we should be asking? How should we be asking?
Process: (More on this below)
Profile/Resume Calibration: (If not the first intake meeting). What could we do better? Any common points of failure?
Set expectations:
We will need 3+ weeks to build a pipeline.
This is partnership and I will do my best to be open to feedback, responsive and accountable and I expect the same on your end
This is a candidate driven market so we will need to work together to provide a white-glove, hyper personalized closing experience.
At times, we may need your assistance to do exploratory calls, sell calls. It takes a village!
Helpful Tips:
Get this templated as it will be re-used in the future
Bring a couple sample resumes for the hiring manager to review
Emphasize the partnership piece. First impressions do count and this will be the ‘kickoff’ to a hopefully successful partnership.
JOB DESCRIPTION
What: Living document to keep both the hiring manager and recruiter accountable for what a ‘qualified’ candidate means.
Success traits, scorecard. What does a successful engineer/marketer/etc look like? What does a A-player look like? What would we need to see for us to have a 90% confidence interval that they will succeed at each stage of the interview and after, if hired?
This is the hiring manager’s responsibility, not the recruiters. Make sure that is clear from the get-go. Hiring managers should take this seriously as the job description weeds out subjective opinions that will arise subconsciously and consciously during hiring discussions.
Start with a summary about the company mission, the team and the responsibilities
The standard format for most companies these days are responsibilities, the minimum and preferred qualifications
Legally, the minimum qualification really means just that. What is the absolute minimum qualifications a candidate needs to have? (someone with only a Masters legally cannot apply and be considered for a job description that says a Ph.D is a minimum).
Stay objective. Leave out any subjective words that could be loosely translated (strong in C++, excellent coding skills, etc)
Make sure finance is in loop and understand how many hires you plan to make with this posting
Helpful tips:
Great artists copy but don’t steal — don’t reinvent the wheel
As soon as it’s public, make sure the team is aware. Have the team at the very least post it on their Linkedin network. Extra points if they change their Linkedin job title to say, ‘We’re hiring!.”
Leave the fun and fluffy language out. This isn’t the place to show off your culture.
Consider lowering the minimum qualifications if you expect this to be a pipeline req (pipeline meaning you’re hiring multiple positions against this one re)
INTERVIEW STRUCTURE
What is the most cost effective and optimal way to evaluate talent?
The general structure consists of a funnel like shape: Recruiter Chat> Technical Assessment > Onsite > Offer.
Let’s first examine what takes place with the initial Recruiter/Prescreen calls before diving deeper into some trickier areas with the Technical Assessment and Onsite pieces.
RECRUITER CHAT
Typically a 15–20min chat between recruiter and candidate to assess fit in technical background, career expectations, potential leveling and timeline. If there are potential concerns around salary expectations, recruiters may typically also ask for that.
Every candidate’s situation will be different and the conversation should be treated as so. For example, a passive candidate with a qualified background with x company, x school may require more selling up front than an active candidate who is actively looking.
Example template geared towards active candidate:
2min overview on the work you’re doing now
Tell me about some of the most challenging projects you did
What part of your job do you enjoy doing the most?
How do you decide what to work on?
What motivated you to have this conversation with me?
What do you want to do next in your career?
What’s missing at your current work that you’d ideally want to see at your next job?
At this point, I would ask a few technical questions that were suggested by the hiring manager to ask. We would have a provided answer bank as well. The questions, though basic should be pertinent to the signals we’d expect in the latter two stages. The expectation from candidates should be swift and well-articulated answers.
Tell me about your timeline if things were to move forward.
Are you authorized to work here in the US? If so, what would you potentially need visa wise in order to legally work here?
More importantly, sell the vision!
TECHNICAL ASSESSMENT
What: Get the highest quality technical signals on potential onsite performance with as little time/energy from team as possible.
Methods: Two main methods the industry has narrowed down to with some new interesting models that are up and coming.
PHONE INTERVIEW
Typically 45min to 1 hour. Low investment of engineering time that can weed out outliers and can be tweaked to align with onsite bar. This is standard format most companies have adhered to.
Format:
1–3min: Greetings and Resume Overview
3–5min: Explain problem set
5–10min: First question
10–20min: Second question
20–40min: *Third question
40–45min: Wrap up, Q&A.
*You can also swap out coding questions for project deep dive (from resume or other open source project), concept explanations, domain deep dive for more niche roles or other open ended problem solving questions.
Tips on phone interview questions:
Ask easy to medium questions. Save tougher ones for in-person onsite
Focus on coding but pay attention to communication skills as well
Ask questions that have multiple iterations vs binary pass/fail questions
Develop a rubric for what would warrant a greenlight for onsite (I.e, 1 point for each correct answer, 0 for incorrect. Tally up, divide by average and move only those 80% forward)
Focus less on syntax, more on thought process and problem solving abilities
Stray away from a second, follow-up interview unless there were technical signals that were clearly missing or if there was some systematic, coordination error.
TAKE HOME ASSIGNMENT
Take home assignment candidates can work on in their own time. Typically mimics a real world project that the startup is working on. After it’s submitted, interviewer will review and decide if it meets the bar for candidate to come onsite and explain in further detail.
Other methods:
Give interviewee a choice between the above two
Combine methods
Tried 3rd party platforms like Triplebyte, Hired that does initial screens for you
Skip this step entirely
Helpful tips:
Industry benchmark for phone to onsite ratios is roughly 50–60%. Higher than that, bar is too low. Significantly lower than that, bar is too high. Always cross reference however with onsite:offer ratios to see if there’s any correlation.
Experiment but commit. See the results through and measure everything.
Remember: this screen stage is meant to weed out candidates who are unlikely to make it pass the onsite and thus, saving precious time for your interviewing team who would have otherwise dedicated a whole day interviewing, writing feedback, debriefing and helping sell.
Resources:
ONSITE
What: It’s not perfect but it’s the industry standard format for a reason. It’s scalable, provides the most signal with the lowest amount of engineering time investment, easy to track, easy-ish to fix.
DETERMINE FORMAT
Start here — develop a criteria for what’s expected for this role. Refer back to the leveling chart, job description and other helpful documents that will help determine the format.
Signals we want may include…
Technical competency
Domain competency if relevant
Ability to work with others
Ability to problem solve and break down big picture ideas
What level (what interviews will give you signal here?)
Potential for growth (especially if it’s a more jr-mid level candidate)
Common ‘Loops’ that engineering uses:
Coding — An imperfect and sometimes controversial science. Understanding technical and communication signal, less on syntax and other things candidates can just google.
Whiteboard — typically to squeeze at least 2 or 3 questions. Emphasis on data structures and common algorithms. Rubric created on speed, problem solving ability, communication and cleanliness.
Laptop — Continuation of take home exercise or new assignment that’s related to some current project at Company. Candidate works alone for 45min with follow-up debrief to understand the why’s, how’s.
Design — Understanding candidate’s ability to take big picture concept idea and implement it into smaller components. Measuring depth and breadth of the candidate’s experience as well as ability to explain concepts via whiteboard / block diagrams and communicate thought process well.
Behavioral — Understanding ability to work with others. Times of conflict, technical disagreement, examples on how they receive and give feedback, ability to meet tight deadlines and other project management pieces.
Examples loops:
Facebook typical Software Engineering loop
-2 Coding (whiteboard, heavily focused on algorithms, careercup/leetcode type questions)
-1 Design (design instagram, focused on scalability, systems, etc)
-1 Behavioral
-1 domain for IC5+ (can be used for additional coding or design)
-Lunch (non-voting)
Facebook Silicon (Verification role)
-1 coding (SV/UVM, whiteboard)
-1 design (Testbench architecture design, block diagram)
-1 problem solving (open ended question focused on methodologies and solving real world situations)
-1 behavioral
-1 partnership
Uber typical software engineering loop
-1 coding (no whiteboard, candidate has short exercise to complete by themselves)
-1 resume deep dive
-1 ‘bar raiser’ (interviewer not from team focusing on behavioral)
-1 design (design feature for real-life project)
-*1 domain (optional, will depend on role. Domain deepdive)
Helpful tips:
There is no need to reinvent the wheel. As you can see above, it’s all fairly similar with some minor changes here and there. Ask around the industry to see what’s worked best. This is a win for the company to make decisions quicker and test as well as candidates who are already used to a certain format. If you want to go outside the norm, be sure to test slowly before making any tweaks.
INTERVIEW TEAM
What: Your interview team is the core of how the team grows. Be open to feedback, build scalable and efficient processes and reward generously.
Common ‘Best Practices’:
In the beginning, you (hiring manager/founder) may be the only person on the team. Leverage other teams if there are any until your team is hired, trained and ready to interview
Ideally, new hires start with more senior engineers. These will be your core interviewers until the team grows to 5+ and jr-mid level engineers come onboard
Once the team scales to 5+, your interviewers should ideally be interviewing at or below their level. I.e, a mid level engineer should not be interviewing a senior / tech lead, etc.
For any female/diverse candidates, it is encouraged to have at least one female/diverse interviewer on the loop to help eliminate any bias that come up in an otherwise all male panel
To reduce interviewer burnout, create a robust, scalable interviewer training program for new hires (more on this below), ask and give feedback and reward (recognition, sharing the good news with someone they interviewed was hired, etc).
Partner regularly with the interviewing team (especially in the early days) to calibrate on questions (is it meeting the bar? Is it reasonable? Is there cohesiveness in the phone and onsite questions, etc).
INTERVIEW TRAINING
What: 30min-1hr training for new interviewers. Critical for scaling and hiring teams, building a culture that prioritizes hiring great talent and helps to eliminate bias in the hiring process.
Typical topics covered:
Managing bias (why it’s important, how we avoid introducing bias, etc)
Best practices on how to conduct technical interviews (more on that below)
Interviewer etiquette (how to greet, enforcing empathy, providing hints if needed)
Interviewer expectations (not discussing beforehand, timely feedback submittal, accepting + committing to invites, attending debriefs)
How to write feedback (more on that later)
Process moving forward (training rounds? shadows?)
Helpful tips:
Consider asking a 3rd party company to lead a class on managing bias. Introducing bias can interfere in gathering objective, evidence-based feedback.
Depending on bandwidth, it’s recommended new hires at least 3 months before jumping into interviewing. This will give them time to get acclimated and focused on their actual role before stepping in.
Consider using a shadow, reverse shadow methodology. Shadow = trainee in the back taking notes and observing. Reverse shadow = trainee is now leading the interview with the interviewer watching. Bare minimum before passing forward would to do 1 of each.
Work with finance/HR to understand how rapidly the team expects to grow over the course of a half or year so that you can plan for trainings accordingly.
DETERMINE RUBRIC
What deems a hire or no hire? What will the rating systems look like? What evidence will help determine that decision? Reference this post.
Helpful Tips:
Binary decision work best. Hire or no hire. You can assign magnitude to each but do your best to stray away from maybe’s, neutrals, etc. Facebook example — Hire (absolutely, high, average, low) and vice versa with No Hire.
Keep careful notes in the interview. Stay as objective as possible! State the facts of what was observed and avoid offering personal opinions as much as possible.
Create some kind of ledger that will keep you accountable throughout the interview to assign micro grades. Plus/minuses, then you can make a final decision at the end looking at everything conclusively.
INTERVIEW QUESTIONS
What: Interview questions can make the difference in how a team hires so take the time to research, discuss, select and calibrate as the team grows.
Every team (Software, Hardware, PM, Design, Sales/Ops/Marketing, Leadership, etc) will have their own customized version so take the time to think through this through and put processes in place to be able to calibrate and improve.
Given my background has mostly been around software, I will share examples from those areas.
CODING:
Two objectives in this round is to get signal and provide a good interview experience. Getting signal is hard but this in-person 45min-1hr format has continued to prove to be the more efficient and effective way to measure current ability.
Quick notes:
This should be a continuation of the phone interview signals gained. Make sure to address any concerns that were brought up there.
Two ways to ask these coding questions. Either choose 2 individual questions or 1 that is iterative and builds deeper.
For newer interviewers, it’s encouraged to start with the standardized/approved questions before branching off on their own. This gives them a chance to calibrate and practice properly.
Format:
2–3 minutes of intro. Outline the interview structure, talk about yourself, ask for a bathroom/snack/beverage break.
10–15 minutes: Warmup question.
20–25 minutes: Followup question.
5 minutes: Wrap up.
Signals to look for:
Communication
Could the candidate understand your question and get clarity when necessary?
Do they incorporate hints or advice that you provide to them?
Can they describe their ideas to a problem in such a way that you can understand them?
Problem solving
Raw capability — can they come up with solution(s) to the problem? Can they compare alternatives?
Can they characterize and use appropriate data structures?
Can they talk about the space and time complexity?
Coding
Can they convert their solutions into executable code?
Is the code well organized and does it capture the right logical structures?
Verification
This is usually done through testing. Does the candidate consider a reasonable set of test cases or otherwise come up with a good argument for correctness?
If they have bugs, can they step through their own logic to find them and explain what their code is doing?
Helpful tips:
Skip background, other ‘softer’ skill interviews can cover. Also sounds repetitive if they are being asked the same question 4 times. (i.e, tell me about yourself/background/resume)
Make sure they understand problem (write example on board if needed)
Let them write inefficient signal first and course correct. This in itself could be valuable signal.
Avoid giving hints too early (doing so could give away the answer)
Ask questions / clarification after they finish coding (don’t disrupt their thought process)
Use examples to exposure bugs
Set expectations at the beginning that you’ll be taking notes
Good places to find new Coding questions.
Algorithms For Interviews by Adnan Aziz and Amit Prakash:
http://www.algorithmsforinterviews.com/
http://www.amazon.com/Algorithms-Interviews-problem-solving-approach/dp/1453792996/
Elements for Programming Interviews by Aziz and Prakash:
https://www.amazon.com/Elements-Programming-Interviews-Insiders-Guide/dp/1479274836/
Programming Pearls by Bentley:
DESIGN
Assessing candidate’s ability to design software systems. Usually done on the whiteboard with diagrams and in-depth discussions to explore deeper.
Quick notes:
Strive to cater towards candidate’s background / domain (more frontend vs backend? ML Ph.D vs backend storage expert?)
Question should be just slightly outside their normal domain and contain challenges they haven’t experienced or aren’t familiar with. The goal there is really understand how they’re able to think outside the box, problem solve and adapt.
If the area is too different and unrelated to prior experiences then your expectations must be lowered. If the area is too similar to the area of expertise then your expectations may have to be raised
Signals to look for: (below is an example more catered to a software engineer, please adjust accordingly to your positions need).
Problem Exploration: Interviewers should be treated as a customer or potential client who is approaching them with a new features or idea that they’d like to discuss with.
How well can candidates dig into the motivation?
Understanding basic requirements and clarifying any ambiguities with the question (which there will be intentionally)
How do they define a successful system? What does the output look like? Senior engineers should be able to articulate this quicker then junior candidates (some jr candidates just haven’t had exposure at this level but they should still asked)
Handling data: Entities and logical data model — what are the entities? What are the things that need to be stored?
Physical data model — How will it actually be stored and manipulated inside a computer?
Data storage — which storage is used? How effectively is it leveraged?
Data transport — What protocols? How does the flow throughout the system?
Completeness of Solution: Were they able to solve the problem? Encourage candidates to start simple before narrowing scope and digging into specific components
Trade-offs: What positives and negatives are they considering in their design choices? How are they explaining the ‘why’s in their decisions? Why is it the right one for that specific situation?
Deep Dive: Review candidate’s resume and see and areas they’ve worked on before. Allow the candidates to pick areas that they’re familiar with and ask open ended questions for them to explain, (security> vs product design, storage systems, etc)
Leveling: Design interviews provides insights into the depth and breadth of a candidate’s potential level. What were they able to show in terms of ownership? Communication? Evaluations of positives and negatives of their decisions? How articulate were they in explaining and defending concepts and decisions?
A more in-depth guide should be created that breaks down each level and the expectations!
Format:
Part 1 Problem selection -
Should not be in the candidate’s area of expertise.
Should be fresh to you or else you’re too close to the subject and will introduce cognitive bias
Part 2 — Setting the Scene
Reiterate that this is not a coding interview
You need to do X, Y, and Z, in a way that I understand, to be successful.
Explain problem and under-specify
Part 3 — Getting to a solution
Can the candidate find any solution at all? Do they appreciate that there is a problem here, that requires ingenuity to solve?
Once some solution has occurred to them, does the candidate’s mind remain open to other solutions? Or do they cling to the first potentially workable solution as if to a life preserver?
How structured is the candidate’s thinking about the set of solutions? Are they able to identify the key decisions that chop up the design space?
What questions are the candidate asking? Do they understand the consequences of the decision here, or do all designs just seem like undifferentiated matters of taste?
Part 4 — Twist
Your goal here is to see how flexible the candidate is. In real world settings, architectural discussions are typically fluid and can change as new ideas come to mind.
Ask candidates what they would do if XYZ came up as a constraint. I.e, what if the system required more memory or if the users expanded from 100 to 1,000,000? etc
Ask them to explain and defend their decisions. Probe deeper into the Why’s and Why Not’s.
Ask them to rethink their solution but in a different context. I.e, instead of designing for self-driving cars, what would this look like in a lighter weight, less data consuming device? etc
They will need to backtrack on their initial solution and reconsider their approaches, often in deep ways. The goal again is less about the end solution but the process of getting there. Were they able to think creatively and quickly? Were they able to confidently defend their decisions? Were they able to communicate their thoughts effectively?
Last notes:
Focus on problem solving and thought process, less about recall and fluency of their communication.
Take notes, only 45min-1hr to get signal
Additional resources:
What are system design questions
Preparing for a System Architecture Interview
The Complete System Design Interviewer Guide
Top 10 System Design Interview Questions
PROJECT APPROACH
Is this person able to talk in detail about the work they’ve done? Can they demonstrate a deep understanding of the contributing factors to issues they’ve worked with and solved?
Signals we’re looking for:
Ability to draw from experience
Develops thorough understanding of issues
Ability to convey what they’ve learned
Evidence that they can look past surface issues to solve complex problems
Perseverance, works past obstacles, finishes, seeks help when appropriate
If they take/assign credit, is it done appropriately?
Communication skills
Format:
Pick a project of substantial value to the candidate, can be the most challenging technically, highest impact to the company, biggest failure, etc
Focus on both approach and knowledge — how did they tackle it and what did they learn?
Seek specifics rather than generalizations
Dig — Ask follow up questions, respectfully push for more info until you feel you’ve reached the extent of the problem or their knowledge of it
BEHAVIORAL:
How well can a candidate thrive in your company’s organization? How will they be able to work with others? How strong are their project planning skills?
Quick notes:
What this is NOT: Will I want to hang out with this person? Do they ‘fit in’ with my peers today? Independent of their ability to code, architect. This is translatable across all teams and roles.
Real value is in the follow-up questions. Listen and ask open-ended questions to explore and understand deeper. Initial questions are usually generic
Create a leveling document on expectations for jr,mid,sr candidates for behavioral interviews
Exploration:
The Framing. The focus on this part is to understand the context for the situation being described.
When did the situation that they are describing occur?
Who were all of the people involved?
What was the candidate’s role?
What was the primary problem or goal?
What was the general time frame? How long did everything take?
2. The Action. Here you are seeking to understand what happened.
How was the solution space explored? What options were considered?
A solution (or potentially multiple solutions) were decided to be pursued. How was the decision reached that that was the right course of action?
What action was taken?
3. The Result. Here you explore the results.
What were the direct results?
Was the project a success? How did you measure success?
Were there any mitigations put into place to handle potential failures? What were they? Were any of them triggered?
What lessons did you learn?
How have you applied the lessons learned since this happened?
Was the project — independent of whether or not it is considered a success — the right thing for the company to do?
General Signals to look for:
Motivation
Empathy
Ability to be Proactive; Get started
Perseverance; works past obstacles and finishes
Able to work in an unstructured environment; ambiguity tolerance
Conflict Resolution
Growth
Communication
Format:
Explain the format of this interview — 45min to get your 5–6 questions with 5min at the end to ask questions.
Warm up — introduce yourself (watch for signals on their ability to listen, see if anything perks up as your explaining).
Ask questions, reference above framing on how to explore and gain clear objective signals
Helpful tips:
Create a shared ‘Question Bank’ that compiles questions from each interviewer across each loop.
A QB is useful as it’s reusable as the team grow and it allows full transparency across all interviewers and hiring managers to keep, discard or improve the questions.
Be sure to include how to ask question, how to provide hints, variations of answers, code samples, what edge/corner cases to look for and any other useful information that’ll quickly calibrate new interviewers.
If you’re trying out a new question, it’s encouraged to ask experienced interviewers first.
Practice how you would set up the question, how you would guide them through and ask for feedback on the difficulty level as well as it’s effectiveness.
After its polished, stick to it. Try it out for a 3+ interviews and decide if it’s worth keeping, improving and discarding.
PRE-ONSITE PROCESS
We’re a little less than halfway through the interview process. It’s time to start closing! A great close means seeking information early, setting expectations and providing an incredible candidate experience. An incredible experience starts with aligning and preparing the candidate and the interviewer team for the onsite interview.
CANDIDATE PREP
Preparing candidates is a win-win situation for both parties as it brings parity to the process and helps to eliminate bias. Same prep, same format, same questions equals the playing field for every candidate that makes it to this round.
What to cover:
How many interviews and how long each one are
Frameworks that organizes thought processes and answers (i.e, for coding, asking questions, solve problems and optimize later. Behavioral — utilize the STAR method, etc)
Other general tips on communication (concise, specific, results-oriented answers)
A general sense of what each interviewer will (i.e, Bob will cover coding, Sarh behavioral)
High level idea of types of questions but not the exact ones (obviously).
Resources that would help (coding websites, specific chapter from a book, coding practice websites)
Confirming where they are with other interviews (any burning offers?), their decision factors and setting expectations around timelines through the rest of the process
INTERVIEWER PRE-BRIEF
What: Aligning interviewers with expectations for role and level. Helps prevent overlap in signals and any other missing signals approvers will want to see.
Setting expectations on ‘target level’ will also help interviewers calibrate on the questions they’re asking and the performance expected.
Content of pre-brief:
Recap of candidate’s background and potential leveling. Call out any flags from previous interviews or things to dig more into with background
Hiring manager will then assign interview lanes. What will each interviewer cover? This is particularly important for non-pipeline roles. Is there a specific part of their past project you want to cover? Deep dive into Ph.D thesis? Diving into specific concepts needed for the role?
Last reminders to take fill in feedback within 24 hours, take pictures, stage presence
Helpful tips:
Remember — this is the responsibility of the hiring manager, not recruiter to manage.
Put a system in place (reminders, calendar invites, etc.) to make sure pre-briefs are always sent
POST-ONSITE PROCESS
It’s time to guide the candidate and interviewing team to the finish line. Often times, a lack of urgency and indecision leads to a negative return of investment with the team’s time, energy and financial resources. Take the time to set up a process and have clear expectations communicated from the top-down to prevent delays!
WRITING + SUBMITTING FEEDBACK
Quick tips:
Keep feedback evidence-based, objective and unbiased
Time kills deals. One of the most common bottlenecks is in slow feedback submittal.
General format of feedback:
Summary: 3–4 sentence synopsis that outlines interview, the final decision and why that decision was made.
Questions asked: Utilize a template if the same questions are typically used
Feedback: More on this below but make it easy for others to read. (I.e, no big blocks of text, utilize lists and bullet points)
Code/Design: Take pictures
Writing and taking feedback:
Explain to the interviewer that you’ll be taking notes but that you’re 100% focused
Notes = evidence. There’s no need to write everything down but it’s important to capture the main points.
Well-detailed notes are objective and can be shared if other interviewers, approvers or future interested parties want to review.
Keep it concise and relevant to the role and the assigned loop
Create a system, +/- for each part of the interview. Refer back to rubric for what dictates a hire/no-hire.
Make a holistic decision after that’s defensible and most importantly, have confidence in the signal you got.
Decide if you want interviewers to provide leveling suggestion.
Negatives: Interviewers might be uneducated about leveling differences. May also be biased based on their own interview experience and current/desired level
Positives: Creates uniformity in signal that decision makers can utilize.
Submitting feedback:
Set a 24 hour SLA. Have org leaders communicate this downstream and bake it into the interviewing culture. Remember, time kills deals!
Consider scheduling 15min after interview finishes so they have time to write feedback while it’s fresh and not be in a time crunch to run to the next meeting
Interviewers should not discuss their feedback with others, especially ones who haven’t submitted feedback. This is one of the easiest ways to introduce bias.
Depending on the system, there should be a feature that allows submitted feedback to be hidden. It should not be released to other interviewers until EVERYONE has submitted.
DEBRIEF
What: In-person meeting (can also be done online) with the goal of making a hire, no-hire decision.
General format:
Recruiter introduces candidate, role, team then starts with the positive results first before moving to any that were negative.
Recruiters and/or hiring manager’s role is to facilitate the discussion, point out any trends/patterns that are arising and clarify any feedback that wasn’t clear
Hiring Manager should ultimately be in charge of summarizing the thoughts in the room and making a decision to move forward, reject or if necessary do follow-up interviews
Hiring Manager will usually determine leveling at the debrief stage as well
This can be done during the debrief where HM will ask interviewers for their opinions and come to an agreement together.
Alternatively, in most cases I’ve seen, the HM will wait till interviewers leave and work with recruiter to determine suggested leveling.
Recruiters should be taking notes (evidence!) that approvers can read up or follow-up on later
Keep debriefs to 30min (or less). Time limit keeps everyone accountable in sharing concise feedback, debating and making a decision
Helpful tips:
In the beginning stages, it may be worth meeting in-person no matter what the outcome of the decision is. This helps everyone calibrate between resumes, phone interviews, onsites and expected outcome vs actual results. After the interviewing team feels calibrated, you can keep debriefs only for those that are borderline.
For jr-mid level engineers, stress the importance of making decisions based on potential/growth. What objective evidence was shown that gives us the confidence that they can grow?
For senior and above candidates, the bar should be held high. The expectations should be greater as they’ll have a larger set of responsibilities and bigger cross-functional impact/scope that could have bigger consequences if the wrong person is hired.
Follow-up interviews should only be done if the interviewers feel like there was missing or additional signal needed to make a decision. I.e, strong coding signal shown in phone interview, very strongly in first coding round, and somewhat weak in second. It’s not definitely clear so a follow-up coding round can be scheduled to give everyone (especially the approvers) the confidence needed to move forward.
False negatives / false positives — it’s almost inevitable that great engineers will be missed and mediocre engineers will be hired. The goal for any sized companies is to avoid outliers by detailing out your ‘scorecard’ for what makes a hire and reflecting periodically on the performance of engineers who are hired.
For startups who don’t have the same brand power as FAANG companies, it can be both devastating for the team to never hire (burnout, low morale, product doesn’t get shipped) as well as hiring weak engineers (brilliant jerk that poisons culture, can’t keep up with engineering expectations, slows down processes).
Common Objections For Not Moving Forward
Not enough headcount You have a candidate that the team is keen on moving forward with but no headcount.
What to do: Remind hiring manager and team that the goal of the debrief is to make a decision if they meet our requirements. Assure them that we can get creative and find ways to pull forward and/or borrow headcount
Other qualified candidates in pipeline
What to do: Pull up any available data on conversion rates from phone to onsite. Give them a rough idea of how many candidates might actually convert to an offer. Provide any info you have on other offers candidate have in hand. Remind them that it’s a competitive market and the risks at hand if they chose to not move forward and waited (might miss out entirely on a good candidate, especially if no one else converts, further delay filling gaps in team, chance of getting false negative signal by being too stringent on requirements
Wasn’t as good compared to past candidates
What to do: Raise concerns on outcome bias. Remind them that we have evidence at hand and that we should do our best to make a judgement based on that.
Doesn’t raise the bar with current team
What to do: Call out the subjectivity of the statement. Encourage interviewers on the thoroughness and alignment of the interview process. It was the same process that they themselves were most likely on!
OFFER APPROVAL
What: Final review of interview ‘packet’ by main decision maker before moving to a verbal offer.
Format: Depending on the size of company, the overall format can be as informal as an email or as formal as a 3 person hiring committee (more on that below).
Pre-work: Recruiter will put together the following:
Written interview feedback
Debrief meeting notes
Call out any green, yellow flags (there shouldn’t be any red ones)
Supporting statement from the hiring manager on why they want to move forward for the role, level and project
Supporting statements from any ‘weaker’ interviews from interviewers
Goal of approver (whether it’s a committee, director, founder, etc):
Look at the packet purely from an objective perspective. ‘Fresh’ eyes to call out any concerns that the hiring manager may have missed regarding decision to move forward, level that was determined and the role / team they’ll be. Also an opportunity to calibrate on the interview process (are the right questions being asked, are we using the right interviewers, are we being too soft in our decision making, etc?)
Hiring committee — min 3 person committee formed by directors from across different orgs to eliminate any bias. Typically seen in bigger organizations that have less control in maintaining their bar. (Linkedin, Google, Facebook). Similar goal as above but more rigorous process as there are now 3+ eyes scraping through every part of the packet. Typically done in-person with the committee, recruiter and hiring manager.
Common reasons why ‘packets’ don’t get approved:
Not confident enough in certain signal (coding, wants more problem solving, design wasn’t up to snuff)
Concern on growth trajectory (has already been at X level for X amount of years. Interview didn’t show great promise that anything will change, etc)
Flag on behavioral or very basic technical concepts that they should know
Interviewers didn’t ask right questions or had overlapping signal
COMMON BOTTLENECKS
Time kills deals.
Here are some of the bottlenecks that create delays and gives competitors an edge in closing.
Common bottlenecks:
Candidate not providing availability — give yourself reinders to follow-up
Scheduling not prioritized accordingly — when there’s an influx of potential candidates, it’s important to stack rank by priority. Who has a burning offer? Who has a higher likelihood of making it through?
Last minute changes to format or interviewer — This can be mitigated by double checking initial panel formation, especially with new roles.
Interviewers asking low signal, uncalibrated questions — Interviewers should be getting feedback from the team / experienced interviewers. A Question Bank could help too.
Interviewers delay feedback submittal — One of the most common headaches. Speed and urgency needs to be communicated top down. Leave 15min at the end, use data for repeat offenders
Hiring Managers can’t find a final decision. Wants to check references, take time to think, ask around the team for second opinions, etc.
Almost to the end! With the foundation all set in place, we can move on to the fun part. Sourcing, closing and calibrating!
Check out Part 3 here.
