Introduction
I would like to state that after reading this article as a job seeker, you will be better prepared to tackle product management interviews.
The truth is that you won't, because I'm the only person that I know of that uses this interviewing plan.
Unless you're interviewing for a role in my team, this article will be more of interest to people that need to design an interview panel and plan for product management positions. This plan actually works beyond product management since I have used it to interview people for roles spanning software engineers, engineering managers, finance directors and other positions.
Why do you need an interview plan
For hiring to be successful you need to know what you are looking for and you need to design an interviewing plan for it. There are different philosophies from ultra-detailed job descriptions to broadly hiring for talent. The philosophy itself doesn't matter. You need to know what you're looking for. And if what you're looking for is just generally smart people, you can put a plan together to filter for that.
If you don't know what you're looking for, then usually you will not be able to put a plan together other than picking a few people for the interview panel. And what ends up happening is that everybody asks whatever they want, usually overlapping questions, and in a debrief you can't really tell if the candidate is good or not. I've been in some interviewing debriefs like that and the outcome is that the more vocal opinion wins. If the person expressing a strong opinion is a natural talent discoverer and a high performer with good soft skills (which are important for interviewing) then that can actually work pretty well. Given that throughout my career I have met a lot more people that are very vocal while lacking the talent discovery characteristic, I wouldn’t count on this approach working.
In general, to put together an interview plan, you need to:
- Have a job description for the role which actually describes the role (or what you’re looking for in people, if the role is undefined)
- Define the number (and sequence) of interviews and what each interview is for
- Identify the people that will be part of the panel
- Define how you will collect feedback and make the final decision
Let's go through each of these in detail.
1 - Job description
The first important thing here is that the job description is not just for the candidate; it is also for the other people in the interview panel. The interviewers need to know what they're interviewing for. In small companies, this is less of a problem since people tend to know what an open position is for. In bigger companies instead, where people might be pulled in because of scheduling availability or because they are broadly in the same job category, it is very important to frame the team and the job duties expected of the candidate so that the interviewer can ask appropriate questions.
Many job descriptions out there are completely generic and bland.
The general structure of a section explaining the responsibilities of the job and a following section explaining what experiences and skills will make a person successful in the role is usually a good start. I like to frame the responsibilities sections like acceptance criteria: after a few months on the job, it has to be clear whether the person that was hired is on track to fulfill them or not. For the experiences and skills, I also like to focus on key hard skills like coding in specific languages or key experiences that the person needs to have achieved (for example, having shipped a product to 1 million users). Skills such as “good communication” and “attention to detail” are so subjective, it’s hard for a candidate to tell whether they have them or not. I like to point out specifically the task, cadences, stakeholders and outcomes which the role will be responsible for. I'll probably write another article specifically on job descriptions at some point.
Note that if you define the correct skills and experiences, you won’t need the number of years of experience as a requirement! I never list desired years of experience in my job descriptions. People move at different speeds and will accumulate learnings and experience at a different pace. Certain industries also move at a different pace. Both working on a plane from end to end (for example, the new version of an A320) and working on a car end to end (for example, the latest Toyota Prius) count as “work on a product from inception to delivery which is used by millions of people for transportation”, though the first one requires significant more time compared to the second one. If you are in the car industry, that requirement would automatically filter out people with less than 6 months to 1 year of work experience. If you are in the airline industry, with that requirement you would filter out people with less than 6 - 10 years of experience.
So focus on what is actually needed on the job and leave years of experience out of it.
2 - Number and objectives of interviews
There is also not a hard rule here, though in general I've seen three to four interviews being the right number. Hardly any company does less than three; five or beyond start becoming repetitive and taxing for both the candidate and the people at the company involved in them. Each interview should have a specific objective, whether that is testing a specific skill, asking about specific types of experiences or evaluating knowledge of a topic. The objective has to be clearly communicated to the interviewer ahead of time so that they know where to focus their questions. This is a big help to the interviewer. I've seen some interviewers head into an interview without actually knowing what they have to ask and coming up with terrible questions.
This happens especially for data scientists or engineers that interview product managers. They are not used to that interviewing cycle, so they don't know what to ask.
My preference is to go for 4 interviews.
- The hiring manager interview (the person that is going to be the manager of a hiree)
- We'll get into the detail of this later
- Core focus is evaluating the all-roundness of a person, the fit of a team and the fit of their past experiences and expertise with the demands of the role
- Interview by another product manager
- This can either be someone in the same product management team or in a team that would work closely with the hiree. Ideally, it is the same type of product manager as the candidate.
- Core focus is evaluating product management skills, frameworks and cadences specific to the company. Given product management is so different at different companies, this interview is to make sure that the hiree is the right kind of product manager for your company
- Interview by an engineering manager
- This would be the engineering manager that the product manager works closest with
- Core focus is evaluating the technical understanding of the candidate
- This is not a coding interview. The focus is on software architecture, software concepts, software best practices and, especially, how the product manager would like to interface with the technical team
- This last part especially varies a lot from company to company. So this interview is to make sure that once the candidate joins, they will be able to fit into (or appropriately modify in the case of an eng manager who is struggling to change a team) the cadences of the team
- Interview by a designer or data scientist or marketing (or generally an important stakeholder to the PM - non technical preferred)
- The role of the interviewer depends on the type of product management role for the position
- This should also be someone the product manager is going to work closely with
- The focus is on stakeholder management (especially past experience with similar stakeholders) and on the competences of the product manager in the specific field of the interviewer
The people chosen should be crafted to the role, though in general one other product manager, one technical stakeholder interviewer, one not technical stakeholder interviewer is a very well rounded panel.
3 - Identifying the people on the interview panel
Beyond the role, you also have to look at the personality and the traits of the interviewer. You have to look for someone who is good at holding a conversation, who holds opinions on products, and who will also portray the company in a good light. Remember that an interviewer also needs to be selling (especially if the candidate is looking promising)! This means that sometimes you have to choose people who are not directly stakeholders of the future PM, though that fit these characteristics better than the actual stakeholders.
From my experience, especially for PMs who are going to interface a lot with engineering, you might want candidates that complement the existing engineering manager, and that potentially will conflict with the engineering manager (in particular if you’re not happy with how the engineering team is being managed). In that case it's better to keep the engineering manager in question out of the panel.
Good traits / experience for interviewers:
- Extrovert and outgoing
- Not shy of giving direct feedback (it’s important to test that during the interview)
- Hold opinions and voice them in meetings
- Are enthusiastic (in a realistic way - not excessively) about the company and the opportunities that the PM role that is being hired for would be able to drive
- Have some expertise in the field that the PM is going to cover
Bad traits for interviewers:
- Not happy at the company and would criticize the company during the interview
- In general would be against the role that is being hired for existing in the first place (usually due to office politics)
- Not knowledgeable of the field that the PM is going to cover (and hence would not be able to judge expertise there)
- Critical of the existence of product managers in general. There are people like that, usually engineers which have a lot of product sense and hence don’t think that someone who is just a product manager and not also an engineer should exist (which is an opinion I respect). There is one exception to this rule and it's the case of senior PM hirees who you want to put in difficulty in an interview setting to see how they react. If they can convince a PM-skeptic then it’s a really good sign
4 - Pre-briefing the interviewing team
Even with a good job description, a good process laid out and great interviewers, I found it helpful to still have a pre-briefing meeting before the interviews kick off to align everyone on the interview panel, see if there are questions, and make sure that everyone understood their role in the interview panel. This is also a chance for the hiring manager to explain the type of person that they are looking for (for traits that can’t be well described in a job description). These are things like:
- How they expect this person to fit into the rest of the team (challenge the rest of the team, mentor the other PMs, get coached by other senior PMs, have strong opinions, be more of a coordinator etc)
- Whether they’re looking for someone to grow inside the company or for someone to take care of a specific team / product periodically
- Whether they’re looking for someone who is a wild card (high risk / high reward) or a person who could perform stable job duties and be easy to manage
5 - Making the final decision
Running an interview panel is like running an experiment. If you don't establish the criteria for passing upfront then by the time you get to the end of it you'll find yourself questioning some assumptions you had. Many companies have an “everyone needs to say yes” policy, which undeniably fails once you get to the final decision debriefs.
What I found is that in final decision debriefs, people who are more vocal with their opinion and/or who are higher up usually get their way, even if the score of others is different. This especially happens in panels where the hiring manager really wants to hire the candidate, but the rest of the panel doesn't think the same.
Regardless of whether you're the hiring manager or not, you should establish very clear policies which are not interpretable and hold the rest of the interview panel to it.
What I believe in and have heard many other senior executives tell me is that it's better to keep a higher bar always and have false negatives (not hiring someone that should have been hired) rather than false positives (hiring someone who should not have been hired). As soon as you start making exceptions, like accepting more “nos” on the panel or putting the candidate through an extra interview to decide tiebreakers, they start to become the norm and taint your interviewing process.
How I like to go about making the final decision:
- Everyone needs to fill in a scorecard with a single score and a description of Pros, Cons and the reasoning for the final score. The greenhouse default of “Strong No”, “No”, “Yes”, “Strong Yes” are good options for scoring. Avoid “unsure / neutral” scores. If your company does have neutral scores, count them as a “No”
- The hiring manager reads the scorecards offline
- Criteria:
- Majority of “No” or “Strong No” -> reject
- There is a “Strong No” and average is neutral or slightly towards yes -> reject
- There is a “Strong No” and a “Strong Yes” with average of neutral or slightly yes -> go to in person debrief with the starting point of a reject
- Majority of “Yes” and no “Strong No” -> go to in person debrief with a starting point of an accept, mostly to check on any red flags
- Majority of “Yes”, no “Strong No” and at least one “Strong Yes” -> accept. No debrief needed
Bonus mentions
- Time: scorecards and debriefs should be done as soon as possible after all interviews are done. As a hiring manager, you're responsible to follow up with people to ensure they fill in their scorecards
- Scorecards: Many companies ask you to fill in more than one score per card. That's useless. As a hiring manager you don't have time to review different types of scores, plus, people don't follow the same scoring criteria. Only the final suggestion matters
- Knowing your interviewers: As you start getting the same people in interview panels, you will start identifying the propensity to score high or low of different interviewers. In an ideal world, you will get the past averages of all scores of all interviewers to understand how they tend to score. After you've been in a few debriefs with people, you will start seeing those who tend to score higher and those who tend to score lower. It's important to take that into account when reading scores. A “No” from someone who always says “Strong No” might actually be a good sign! In the US West coast instead I see a lot of people who always tend to score “yes”. Make sure to “discount” the value of those yes. A top notch HR organizations would benchmark interviewers to understand their averages and biases, though I have never seen that happen.
A brief overview of product management interviewing standards
It's helpful to start knowing how companies right now are interviewing for product managers. There are two major approaches that I’ve seen out there:
- The big company “bias free” approach
- The inexistent approach
1 - The big company “bias free” approach
Spoiler alert, this method is not only bias-free, it is also signal-free. It is broadly not helpful in distinguishing good candidates from bad candidates and disproportionately privileges entry or low experience candidates from top schools and consulting.
I don't know exactly where this method originated but it probably lies somewhere between Google and Facebook. The name “bias-free” comes from the idea that the candidate should mostly be interviewed on standardized questions where anyone, regardless of their background, can excel.
This usually involves two to three types of standardized interviews (I’ll go through them). Usually they are all administered by other product managers at the company, who have been specifically briefed on how to conduct the interview and what to look for, even though they may not know nor work with the team that is doing the hire.
- Product sense = this covers the ideation of a product, from objective setting through user need identification to MVP. The product usually has to do with the company that the interview is for
- Execution = this is usually focused on identifying metrics, designing dashboards, root cause identification (through metrics) and what if scenario planning
- Leadership = this is the only one in which questions about the candidate experience are answered. The candidate is expected to provide specific examples from their past that showcased a specific value or skill. The questions are the usual “tell me of a time when…”.
I'm not going to spend too much time describing these interviews in detail given there's many resources online that can do that.
Pros:
- Very scalable, since people can be trained to administer this and need little context on the role to be able to do this correctly
- Very standardized, makes it easy to compare candidates
- Generic, which means that someone excelling at these can usually adapt to different PM roles (provided that no specific knowledge is needed)
Cons:
- Rewards candidates who have more time to practice and who are used to applying to jobs with standardized interviews (new college grads, consulting, people who are studying to get into consulting). These are usually a very homogeneous set of people
- Don’t ask about situations or skills which the candidate would use in the day to day job, resulting in poor signal for the specific job
- Reduce the interviewer’s freedom in what to ask, which can dampen the effect of very good interviewers
- When questions are leaked (they will for sure leak), it becomes even more a game of who prepares more, not know is more knowledgeable or skilled, which leads to bad hiring outcomes
Caveat:
- The question bank from which the questions are chosen is critical for the success of this type of interview. In general, as I said, I don't think this is an effective style, but with very tailored questions to a role, this can actually become a fairly good signal for specific skills. However, most companies have very old question banks and no one likes writing these, so they quickly get outdated and leaked. Similarly, given no company wants to maintain a huge question bank, the questions to choose from usually tend to be very generic since they need to work for product roles all across the company. At Lyft this is the method used and I solved for the question bank problem by using LLMs to help me draft role specific questions for each interviewee.
2 - The inexistent approach
Not a lot to say on this one. Especially at startups, the interview process is usually random and doesn't really have a structure. The interviewers are chosen, they look at the job description, if they’re lucky they are briefed by the hiring manager on what to probe into and they have the total freedom to ask what they want. Usually the interviewers will be stakeholders of the open role and so they are aware of what the role entails.
As a result, the success of the interview panel really hinges on the ability of the interviewers.
The experience on the candidate's side can be disorienting sometimes, which can hurt the image of the company and reduce likelihood of acceptance of the offer (if the candidate does get offered one).
How these interviews usually go is the following:
- Candidate introduces themselves and gives a 5 - 10 min summary of their career
- The interviewer introduces themselves and gives a 5 min overview of the team and the role they have at the company
- Interviewer asks about what the candidate did in one of the companies the candidate has been at (usually one that the interviewer knows about or a known company)
- Candidate goes into a little more detail on their experience at that company
- The interviewer asks why the candidate is interested in the company
- The candidate answers with usually a fairly standardized answer
- Bonus mention here: you have no idea how often a recruiter reaches out to me, I say I’m not too interested in the company though I’ll take an interview and then (later in an interview) the interviewer asks me why I’m interested in the company. The real answer is that I’m not interested though I'm there to be convinced (which I don’t say obviously)! Knowing how the candidate was sourced is a key piece of info all interviewers should know to avoid this situation
- Interviewer then latches on some generic concept that the candidate mentioned and asks more about that
- Candidate does that and, in the process, lays out a few of the experiences that they had and have rehearsed specifically for the interview
- The interviewer asks if the candidate has questions and spend the rest of the interview talking about the company
I have interviewed for roles at startups which didn't have a process and it usually feels like different interviewers are asking the same thing without a purpose or an end goal in mind. I have left cycles of interviews knowing that there is no way the company could have understood my abilities (or even my shortcomings) given the terrible questions or follow-ups that they were asking. It made me wonder whether the company even knew what role they were hiring for (see my other article on how to properly define a product manager role).
While being in interview panels at Mythical, I also sometimes noticed that some interviewers were asking the wrong questions or asking questions on a section or job responsibility that someone else had already asked.
Pros:
- Can work well if you have a great interviewer team that knows the expectation of the role being hired for well (and a great hiring manager that briefs the team about it)
- Can work for executives or strategic role hires where the job description is not clear but you want to understand the general mindset and ideas of the candidate.
- Little process required usually means less overhead to handle the whole interview process inside the company
Cons
- Due to the randomness, different candidates could be asked very different things resulting in difficult decisions when comparing different candidates who look promising
- Interviewers usually end up asking the same questions multiple times, which is not a good use of anyone's time
- Higher likelihood of completely missing testing specific skills or knowledge of the candidate (happens especially when interviewers don’t read the job description, which is often the case)
- Smart candidates have the ability to completely steer the interview in a direction they want, not allowing the interviewer to unearth the shortcomings of the candidate
For anyone beyond a very early stage company, I would highly suggest moving away from this style. This usually works well when the founders or core leadership team are the ones interviewing, since they also start to know each other’s style of interviewing and compensate for it. However, it doesn't scale well to many interviewers inside the company.
Check out the next section for a great way to interview people!
My go to interview plan
This is my go-to interview plan for product managers, but I adopt very similar adaptations for other roles (I'll mention changes on this for other roles later).
It is based on three sections of the interview, plus a short introduction.
Pre interview
As an interviewer, before you interview a candidate, you should have taken a very good look at their resume (necessary for the section below on the CV deep dive) and also on what public information there is on the candidate (especially their Linkedin profile) which can help you understand their personality (and especially their humbleness).
This should take at least 5 to 10 minutes of preparation before the interview, sometimes up to 15 minutes, and result in written down notes of what you want to ask the candidate. Write down the questions or topics you want to ask! You won’t have time to recall them during the interview and risk missing some of them.
1 - Candidate introduction (5 min)
First thing I ask the candidate to introduce themselves and their career. The key thing is that the candidate has to keep this under five minutes, ideally three. Objective is to:
- Evaluate whether the candidate can be succinct - no matter how long the career, they should pick out the most important parts in 3 min
- Evaluate storytelling - regardless of the career path, the candidate should be able to explain a compelling story on it
- Understand why they're looking to change their job (if they applied for the role) or what they’d be looking into their next role (if a recruiter sourced them actively). This can change where in the interview the interviewer will ask more questions and how to interpret certain answers from the candidate
2 - Interviewer introduction (5 min)
Here I will give a brief overview of myself and the team. Objective on this is twofold:
- Selling why the position is a good one. At this stage, you don't know if the candidate is a good one or not, so you should anyway put a foot forward and try to sell them on it
- Making sure the candidate knows what they're getting into. Even before getting in the weeds of the interview, you want the candidate to really understand what the position is about and make sure that they will be up for the challenge. This is not that important for the interview, but if you get to the offer stage, this is the part that the candidate will remember, especially if they are evaluating different positions
3 - CV deep dive (15 - 20 min)
This first section focuses on the past of the candidate.
My baseline when reading a resume is that people tend to play up all their achievements. Unfortunately, this is something that is necessary nowadays and that I also do, since you would most probably not pass initial CV screens compared to candidates that do inflate their experiences if you don’t.
I don't consider exaggerating achievements in the CV an unnecessary disqualifier, though this section of the interview is to really understand what the candidate has achieved and was doing in their previous team.
While reading the CV (in the pre interview stage), I try to identify interesting experiences or achievements which I feel are important for me to know in deciding whether the candidate is a good one or not. I don't focus on the whole resume; I pick literally two to three specific things that the candidate has said in the resume and ask them cascading questions on them.
The parts I choose to deep dive in are usually a mix of:
- Product launches (or product the candidate claims to have launched) - I want to understand practically what the candidate did as part of these flaunted launches.
- “Leading teams” - I want to understand what type of leader the person was in these teams
- Skills - Candidates like to put a never ending list of skills. I want to understand what level they have in those skills (do they use them daily? Did they do a class online but never used them? Did they last write an SQL query 2 years ago?)
- Numbers - This is where candidates go the most wild. I do want to make sure that the numbers are real (they usually are) and that the candidate actually had something to do with them (which usually they did not)
Objective of this section:
- Evaluate fit based on past experiences
Questions the interviewer should be trying to answer during this part of the interview:
- Does the candidate have past experiences which fit with the job description?
- How hands-on is the candidate?
- How hard working is a candidate?
- Do they have more broad expertise or deep expertise?
- How independent is the candidate?
- With which functions does the candidate usually interface with?
- Understand their approach to metrics (if they mentioned any in the CV)
4 - Current use case scenario walk through (15 - 20 min)
This section focuses on the potential of the candidate (for the specific role).
Not all candidates may come from careers in which they have the opportunity to have the experiences that would be required for the role. This doesn't mean that they would be bad candidates. However, it might mean that they don't fare particularly well versus other candidates in the previous section. This section is really to evaluate how they would behave in a real-world use case that someone in that job would need to tackle.
For this section I take a problem that I am facing on the job at that time, or something that I have recently faced (very recently - in the past few days ideally). This would be something I am very familiar with and that, if the candidate had been hired in the role earlier, they would have had to tackle. The best types of problems / use cases are those that are still ongoing and for which the company has not found a solution yet. Being the hiring manager for the role helps in identifying these kinds of problems / use cases.
I take five minutes to walk the candidate through the problem, usually by showing artifacts, UIs or current experiences and explaining what we're trying to do.
Examples of challenges from when I was at Mythical (and that I asked candidates) would include:
- How would you design an auction system (I walked them through the current website and experience that didn’t have it yet)
- How would you solve our catalog filtering latency issues from a product perspective (I walked them through the UI and the characteristics of some of the digital goods that were on the marketplace)
After the explanation I turned it over to a candidate and asked them how they would approach the problem and what they would do. This creates a nice setup given the solution hasn't been found yet, so it is clear to the candidate that there isn't a specific answer that the interviewer is looking for. The fact that the interviewer is actively working on the problem also creates a teamwork dynamic and it’s ok for the interviewer to answer questions from the candidate about proposals that have already been considered.
I do highlight to the candidate that we only have 15 minutes to cover their solution and push them to make assumptions to move forward. I would expect the candidate to take five minutes to ask follow-up questions (which is important to show the interviewer how the candidate thinks) and then 5 to 10 minutes to propose a few solutions. At this point the interviewer should provide feedback on the solutions and ideally do another iteration based on the feedback.
This section is definitely the one that provides the most signal. It puts the candidate in a day to day experience and really allows the manager to evaluate how the candidate would fare versus problems that the product and the company are facing. It also evaluates the candidate in a teamwork dynamic, this section should be a continuous back and forth of questions, ideas and feedback.
Objective of this section:
- Evaluate strategic thinking
- Evaluate structure of thinking
- Evaluate understanding of the industry / product of the company
Questions the interviewer should be trying to answer during this part of interview:
- Does the candidate reason and approach problems in a way that would lead to success inside the company?
- Does the candidate analyze and understand the situation, before getting into solutions?
- Does the candidate really understand what matters for the customer (and for the business)?
- Does the candidate think in terms of tangible results, or is very empty-UX focused?
- Does the candidate reason in terms of personal experiences (red flag in my opinion) or generalizes to user personas based on their understanding of the industry?
- Does the candidate have structure in how they break down the problem?
5 - Day to day product fundamentals (10 min)
It is weird that in a lot of interviews you are not actually evaluated on what you would be doing tactically on a day to day basis. This section is specifically to test that. In the use case section, the candidate can show their approach to problems, though it is still something that happens fairly rarely in the day to day of a PM. Most of the time is actually spent executing on documents, projects and running the team (with engineering managers and/or design). This section should allow the interviewer to “see” how this person will execute and spot any red flags there. Like the previous section, there isn’t a predefined set of good answers (though there definitely are bad ones) and the “good answers” will depend on the specifics of the role, including who are the people that the candidate will interface with most often.
Hence in this section I like to ask questions related to:
- Artifacts writing (internal / for the team)
- How do you like to structure a product requirements document?
- Which sections they have and in what order, ok to make assumptions and call out differences across different types of products
- How do you like to structure a roadmap?
- Which tools the candidate prefers to use, what metadata they add to projects, how they fill it in
- How do you like to structure a ticket for the tech team?
- Which sections, how detailed, how hierarchically organized?
- How do you organize the knowledge of the team?
- In the age of AI, I’d be curious to see if they use knowledge bases. How they set up collaboration on documents and artifacts, how they organize documents, how do they ensure the artifacts of the team are up to date
- How do you like to structure a product requirements document?
- Progress and prioritization
- How do they track progress against a roadmap?
- Standups? Weekly meetings? Checking the code or tickets directly?
- How do you assign work to the team?
- Through engineering manager, directly to team members, let team members pick up from backlog etc
- Which cadences / meetings do you want to have with the team?
- What dashboards do you like to have?
- How do you structure hierarchically initiatives / projects / sub projects / features etc?
- What’s your approach to tech debt?
- How do you like to structure a design exploration?
- How do you report progress to executives and / or your superiors?
- How do you deal with wild executive asks?
- How do they track progress against a roadmap?
- Documentation writing (for customers of the team)
- What documentation do you like to have for products (and who writes it)
- Answer to this depends a lot on the industry
- What information do you persist to help people understand what the team does?
- How do you present the team to both external customers or other teams of the company?
- Have you written documentation before?
- What documentation do you like to have for products (and who writes it)
Major green / red flags that don’t really depend on the type of role
- Online vs offline management: especially for roadmaps and progress tracking, the PM should be able to know about that without requiring long and lengthy meetings with the team. If PMs mention “I’ll set up a meeting” as an answer to many of the above, then that’s a big red flag. Effective PMs can keep track of a team completely offline (without synchronous face to face time with the team members). This is especially important in globally distributed teams where times can't always overlap.
- Single source of truths: PMs should try to keep a single version of artifacts up to date, rather than allow multiple versions to exist. This usually comes up in the day-to-day life of a PM, so I would expect them to mention something around it.
- Closeness to tech teams: In my experience PMs should be going to standups and most cadences of the team. PMs that don’t find value in those meetings mean that they either don’t understand what the team is doing or don’t actually control the backlog of the team (which usually then the engineering manager would do). Or simply duplicate a lot of meetings to request for the same information that should exist in standups
- Hands off external documentation: Documentation is a key part of products, both if it is part of the customer facing experience (developer products) or if it is part of the team knowledge base (to help people onboarding to the team for example). Effective PMs are deeply involved in documentation in both cases. PMs that completely delegate documentation to engineers or tech writers are a red flag. A PM for a senior role that has never written either internal or external documentation is a red flag.
Objective of this section:
- Evaluate effectiveness of the candidate as a Product Manager
- Evaluate fit with the stakeholders that the candidate will interface with
- Evaluate fit of candidate with the tools and processes used at the company
Questions the interviewer should be trying to answer during this part of interview:
- Will the level of the candidate’s output be at the level I expect it?
- How big of a tech and / or design team can the candidate handle effectively?
- Will they be able to follow the processes of the company? If no, will they be able to change them?
- Are they adaptable or there’s only one way they like to do things?
- Would the team accept their style of product management?
- Would the candidate be ok with the tools and infrastructure the company has?
Great, you got to the end of the interview! Now hopefully you designed a great interview panel and, if the candidate passed your screen, then they’re off to the rest of the interview panel!
At the end of your interview, if you did pass the candidate, you will probably have an initial idea of their strengths and their weaknesses. Do NOT tell this to the next interviewers, since you do not want to bias them, then follow the process outlined at the “Making the final decision” step above.
Major mistakes I’ve seen in interviewing
I have been in a few interview panels where I was a “No” and the candidate ended up being hired anyway. In some cases the candidate ended up doing well, though in most cases, it would have been better not to hire them.
There are some common “mistakes” that I’ve seen which lead teams to hire people which don’t quite clear the bar:
- Timing pressure
- Getting affected by candidates rejecting offers
1 - Timing Pressure
This is usually the case when a non product person pretty high up (director or VP level) decides to hire a product manager. The reason they do so is usually because they have no idea how to manage specific teams (given often more than one team reports to them) or that they want someone to organize the teams’ updates to leadership to explain what actually is happening. Another case is that they are a non technical director or VP and they don’t understand what the technical folks on the team are prioritizing or doing, so they need someone that can bridge the technical and non technical side.
Given they don’t want to worry about the team, or don’t want to be directly responsible for what the team is doing, they want to hire a product manager “ASAP”. As a result, they pressure HR and the interview panel into finding a good candidate, which undoubtedly results in lowering the bar, or lowering the bar for the number of “yes” that are needed in the interview panel to move to the offer stage. The more candidates get rejected the more the bar gets lowered.
This is a big red flag. When a person fairly high up wants to hire someone not in their area of expertise, there is very little that a person in the interview panel can do. This applies to really any role. Case in point: at Lyft, while I was there, a CMO was hired in 2023. I heard (even before they were hired, from someone on the interview panel) that the CEO was a strong yes, though the rest of the interview panel was a no. The person was hired anyway since the CEO was so adamant about it. They stayed at Lyft for seven months (which is not a lot for a CXO).
Don’t be the person that pressures their interview panel in approving candidates they like. You will regret it, other than lose a lot of reputation on it.
2 - Getting affected by candidates rejecting offers
This can be similar to the previous point, though it works differently and mentally affects the whole interview panel without the hiring manager being able to do anything about it. If you get to the offer stage for a few candidates, but they end up rejecting the offer, people on the interview panel (including the hiring manager) start thinking that the type of profile they’re aiming for is simply too good for the company or the role, so they resolve in having to lower the bar to accept candidates that they previously would have been a no on.
This is sometimes the case, especially for startups that can’t afford to pay the salaries of bigger companies.
I don’t think the solution is to lower the bar though. Some solutions here may be:
- The interview panel. Even if the candidate blames salary, it might actually be that you didn’t convince them enough about the company, the role and the growth trajectory. Could you switch people in the hiring panel for someone who could “sell” the role and the company better?
- The responsibilities. Did you design the role to be too narrow? Maybe you can expand the job duties (and the salary) and make this a higher stakes position. It should attract better people, while also requiring to deliver more. In some cases, to get the budget for this, you might need to look at your current product managers (or other employees) and manage out some of them to make more room for this role (and for better people)
- The macroeconomic situation. Just like real estate, in hiring there is a “hirer’s market” and a “job seeker market”. In 2022, during pandemic times, there were so many job openings and not many job seekers. As a result, all companies I know ended up lowering the bar. At that time, while I was at Mythical we were in hypergrowth mode and many teams lowered the bar just because there was so much to do and not enough people. I don’t think that was a good idea (and many of the people for whom the bar was lowered ended up without a job a year later…) and might have been better off with a smaller team focusing on fewer things and preserving the funding received. I kept the bar high for my team and took more than 3 months (and probably around 100 interviews) to find a candidate (which ended up being excellent) for a PM role. While I was hiring at Lyft a few years later, in a different type of market, the level of candidates was much higher. Sometimes you just have to wait for the macro economic conditions to change. It is an incredibly hard decision, though you will regret it later if you end up with an average hire now
- Change the constraints of the candidate pool. This usually refers to geographical constraints. If you can’t find good candidates… try expanding the range? Especially for in office roles, the location of your office may severely impact the candidate pool that you’re fishing from. Sometimes, if the role is critical enough, allowing for remote candidates, or looking for candidates in other countries can drastically change the type of applicants you get (and their willingness to accept a particular job or pay grade). There are many considerations around this, though if your company is scaling and struggling to find talent, opening a new hub in a completely different country may be your only option to keep the bar high. Especially if you're very used to San Francisco Bay Area kind of candidates (who are usually used to lofty lifestyles, salaries and perks) then looking beyond your borders may be the only option you have, if you’re on a budget
Conclusion
This is it!
If you’re hiring, best wishes in your hiring. And always remember, no matter what the circumstances, keep the bar high!
If you’re a candidate, this hopefully will help you rate the interviewing processes that you are going through. The process does make a difference and great interviewing processes do define the long term caliber of companies. If the interviewing process looks random… you have been warned.