Prioritization explained… is there really a need to explain it?
I admit I am baffled at having to write this article. CLEARLY companies have to prioritize. Isn’t it obvious? I’m sure everyone would agree to that. And clearly it is obvious what prioritization means. There are a few things to do, one is more important, another one is second most important etc.
Nevertheless, in my career, I have been blown away by how many people understand the concept above though are completely incapable of prioritizing. And I’m not referring to being undecided about whether project A or project B is more important. People usually are very clear about that. I’m referring to being specifically told project A is more important than project B and then coming up with a plan, or even worse, executing in such a way that project B gets treated as more important than project A, even though it is clear to everyone that A is more important.
“Prioritization” is often cast around as a term to justify why projects are being cut or not done. I would say that is usually a good use of it, though it can be framed in a much more productive lens. Let’s start from there.
Prioritization: what do you actually need it for
There are two major tasks for which prioritization is a key enabler:
- To create a roadmap for a team
- Once a roadmap is created and a new project comes up, in order to decide where it fits in the roadmap
(2) really is just like (1). Once a new project is introduced to the team, whether you like it or not you have to redo the roadmap. You start from something, which helps, though you will have to go through the same reprioritization exercise as (1).
That’s it. Once you do this, most other questions that come up around “why are you doing this and not that” can be answered with the roadmap.
Which takes us to the first learning of today: prioritization is only needed to come up with a roadmap. Once you have a roadmap, that should answer all the questions around what the team is doing.
At this point the question comes natural: what is a roadmap? And in order to do that we also need to answer: what is prioritization?
What is prioritization?
Prioritization is the act of getting all the projects that a team can do and ordering them in a single list, from the most important to the least important.
Very simple. Input: list of projects. Output: ordered list of projects
How mature do the projects need to be in order to make it on the list? Ideally they have enough information and background such that the team can understand the impact they’re going to have, though we don’t need all the details. At Lyft, projects that are ready to be put for prioritization usually just have a “one pager”, which is a document which covers:
- A summary of the project -> a few paragraphs
- Approximate financial or end user impact -> very high level, usually made up by the product manager or a data scientists (we’ll get to later why this matters only in a minor capacity)
- Approximate effort needing to complete it and what type of people or skills are needed to execute it (also very high level)
What is a roadmap?
A roadmap takes a prioritized list, assigns people to each project, assigns an end date to each project (can be just a duration, the start and end date follow from there).
Input: ordered list of projects. Output: ordered list of projects with people assigned and target end dates.
Spoiler: I say “target” end dates because dates change… all the time. Especially in software companies. If you’re Boeing building planes or SpaceX launching satellites then usually you want to try to stick to your dates. In most other cases, whether you like it or not, it doesn’t matter as much. We’ll talk about how to set dates and the tradeoff around those later.
Usually people like to add a lot of other information to roadmaps, especially to justify why a certain project is where it is. Those are not strictly needed, though it does help to have reasoning behind roadmap decisions all in the same place, since roadmaps get debated ALL THE TIME. As a product manager you will have to defend your roadmap (very important to build trust with the team), so being buttoned up about why you prioritized certain projects and why they will be delivered when they will be delivered is important.
How many roadmap and priority lists do you need?
This is the hard part, given that in practice there usually are different teams that work on the same projects.
My guideline is that each individual team should have a prioritized list of projects and a roadmap. Usually I identify a team as a set of people that report to an engineering manager, though in functional organizations usually teams are made up of individuals that each come from different managers.
In an ideal world, the company would have a list of priorities that includes all the projects of the company. Such that when there is a conflict of resources (industry jargon for too much work for too few people), then the higher priority projects are staffed first. This is how startups operate and how they keep the company focused. When it is clear to everyone what project is most important, people will actually prioritize that.
As companies grow big though, prioritization does become more “distributed”. There are usually a set of top priority projects for the company, though then each team is responsible for the rest of their work.
The granularity of prioritization depends on the size and type of the company. At NVIDIA we would have a priority list at the VP level. At Mythical I was part of the Marketplace Platform team and we had a priority list there (then there was one for game platform and one for game development, which were the other main verticals). At Lyft instead, we have a set of priorities at the company level and then each team (usually reporting up to an engineering or data science manager) has their set of priorities.
It is effective to tie a priority list to a person in charge of other people, so that if there are resource conflicts, that person can decide and the team moves on. When:
- there are teams made up of people coming from different managers
- there is a disagreement on the relative priority of two projects
- the chain of command of the two people intersects at the CEO
then you’re up for a big delay and long decision making times.
At NVIDIA we were effective because most of the team rolled up to the VP so he would decide on any conflicts, though we were good at prioritizing, so there were few confusions on the team regarding what was the most important project. At Lyft instead there are a lot of projects that touch many teams across the whole company, so often what is a top priority for team A is not for team B, requiring their Directors or VPs to meet and solve the conflict of priorities. That escalation usually takes weeks to happen and so delay in execution creeps in.
How to prioritize
How do you decide on how to prioritize projects? I have bad news that unfortunately there isn’t a single way to do this. A lot depends on the stage of your company and what it does. Overall the usual metric is some sort of benefit / effort. In financial terms it could be described as “ROI” (return on investment). You want to do first projects that provide a lot of benefit for the company and minimal effort to execute.
Regardless of the method or framework that you decide to adapt, let’s first learn a few rules that can help
If project A needs to happen before project B, then project A has to be ranked higher than project B
Even if project B delivers more impact than project A.
This is one of the things that companies and teams get wrong the most. In my opinion, it is also one of the things that leads to companies executing slowly. It's fairly normal to have to do some platform or internal product development before being able to deliver big incremental customer facing product experiences. These can vary from working on internal platforms to internal tools to fixing tech debt or redoing certain parts of the product. Usually execs see these things prioritized and think that they are a waste of time and, as a result, shift people to other teams or priorities. Another case is execs criticizing the team for not prioritizing customer-facing projects.
This has terrible consequences, either:
- The team progresses slower towards the foundational projects, delaying the customer experience further
- Instead of solidifying their shared and foundational infrastructure, the team shifts to solving for only the customer experiences, adding tech debt and making the team slower in the future
At Lyft, we had been working on redoing our mode ranking recommendations model, one of the key ML models powering that ranks what type of car you see in the app. As a first step, we had to undo a lot of hard coded rules that had been added there previously. This was necessary since any model improvements we were going to add in the future were still going to be overwritten by those archaic rules. The objective of that part of the project was to replace the rules while not having any meaningful impact on the end user experience. People were not understanding why we were doing that and not focusing on delivering incremental rides and bookings. Multiple times I had to defend the ordering. Well, thankfully we did because else we would never have actually been able to deliver any business value through the model improvements.
If we had not done the foundation layer and just improved the model on top of the manual rules we would not have shown business impact and probably we would have had to focus on something else entirely without ever getting to the rules removals. This exemplifies why it's important to prioritize projects that logically come first higher.
Externally promised delivery dates matter
Product managers and teams love to complain about their business counterpart making impossible promises when it comes to ship dates to either the board, external partners or the public in general. The truth is though, that externally promised dates do matter. As soon as something has been told to the public or to a 3rd party which is counting on it, then the closer you get to that date, the higher the priority of the project becomes. There is a lot of finger pointing and blaming around who decides when to communicate those dates, but, once the communication has been done, the team, whether they like it or not, has to be held accountable to it. As a product manager, this immediately holds you accountable for prioritizing the project correctly in the priority list, especially as that time nears. The opposite is also true. If there is a very important project which, however, has been scheduled to be released further out in the future, it might make sense not to prioritize it right now, because, even if it is ready early, then it would not be shipped for a while. I'll do another article on urgency and how that drives execution in the future.
Now let’s get into prioritization.
A priority list just provides relative ordering of a project versus another. Sometimes product managers struggle to understand which is their most important project or what are the top 5 important projects that they are doing. They want to start there and then build the rest of the list. In practice though you just need to understand pairwise between projects which one is more important and the list then will come together.
An easy way I tackle this is to just write down all the possible projects that the team may be doing. This is great to do not by yourself but with the whole team,since it gives them the possibility to voice what projects they want to do. Teams usually feel excited when the product manager asks about what they think is important and considers it in the priority list.
- Start with all of the projects written in a random order
- Then from the top or the bottom, start looking at each project and their neighboring projects. Is the project you're looking at more or less important compared to the project above it? What about the project below it? Is a particular project dependent on another project in this list? If yes, is that project above it?
- Answering these questions will make it clear as to which projects you have to move and where
- Continue iterating through the list. Every time you move a project check it again versus its new neighbors
- Once you can go from top to bottom without doing any changes because all of the projects and their neighboring projects look good, then you have your priority list!
This should have highlighted one thing regarding priority list. The priority between projects is always relative, never absolute. It is a lot harder to pinpoint what is your most important project or top three important projects, but it should be fairly easy to pinpoint why project A is more or less important compared to project B.
This ties back to the concept I've alluded to earlier regarding impact estimates.
Often executives want to understand how much a project is going to influence the bottom line of a company or progress towards a specific goal. I've seen many teams, analysts and data scientists spend countless hours trying to understand the right impact. However, in my opinion, that is wasted time. Given the prioritization process is relative, the impact estimates are important only to the extent that they can relatively identify which project is more important. A company usually doesn't need to know exactly the impact of Project A, however, it is helpful to understand in percentage how much project A is going to be more impactful compared to Project B.
If you're not sure about the exact impact but you're very certain that project A is going to be more impactful than project B then the team is gonna do project A anyway, regardless of the total impact.
That's why instead of mobilizing a whole team to pinpoint the exact impact estimates, it's usually better to have a specific person work at the estimates for all of the projects such that the estimates are consistent. Consistency is the key. If the same people come up with all the estimates, they are usually more consistent and that avoids conflicts around different teams estimating different projects in different ways, such that one project seems artificially inflated compared to another.
That's also one of the reasons that usually, in a prioritized list, you have the impact metrics associated with each project. It makes it absolutely clear why one project is prioritized higher compared to another.
Prioritization doesn’t reflect capacity
There is one distinct benefit that prioritization lists have over roadmaps: they are not bound by capacity and can hence be infinite!
This is something that has driven a few of my engineering manager colleagues mad. I would present a prioritized list and they would highlight that there's too many things there and that the team couldn’t possibly do all the projects and that I should remove a few before getting to the roadmapping phase.
I wouldn't, and there’s a reason why. A prioritization list is a clear way to show everyone else in the company what your priorities are, regardless of whether the project is going to be done or not. Having a project in a prioritized list and then not featuring it in the roadmap (which is instead capacity constrained, we’ll get there right after this) sends a clear signal that the project was considered but is not as high priority as others and hence will not be done.
When other teams or customers ask about a particular feature or project, they expect that the team does consider their ask. Even if it is clear that the project or feature is not a priority, it helps to have it featured in the priority list and make it public for all to see. It makes your job a lot easier as a product manager. If there are questions around it, you can just point back to the priority list showcasing the projects that are more important than it. This gets us to another golden rule of prioritization:
Priority lists should be public for everyone outside of the team to see
There is no point in doing a prioritization exercise if it is not public for everyone else to see. Having a priority list ready and handy will save you a lot of time when people ask you why the team is working on A instead of B and what happened to their request. This is particularly helpful if you're part of a platform team serving many customers.
The main other reason to have a public list though is that it allows people to challenge your prioritization, which is something that you should always welcome. You can't possibly know everything. There could be people in the company that know something that you don't, and by showing the priority list they might realize that, based on the information they have, the prioritization reasoning could change. This is why I always try to do public reviews of my priority lists and roadmaps, ideally early before we start projects on them and frequently as they change.
At NVIDIA for example, at some point I had a priority list where a particular project was in the middle of it. It wasn't going to be too impactful, but we were going to do it, though not immediately. When someone in operations saw the list, they gently pointed out to me that, for that particular project, we already had a lot of stock of a key (and expensive) component that we didn’t know where else to use. That changed the equation. Given we didn't have to buy more of a component, the cost of producing the hardware was going to be significantly less, increasing margin and hence making it more valuable to do that upfront. As a result, we shifted the priority of that to the top of the list.
Nobody had directly challenged the prioritization and it wasn't really going to be the job of the operations person to review everyone's priority list. Having it ready and showing it whenever I could though, I got valuable feedback and made a great decision for the business.
So you have a priority list… what next?
Great, now we have a priority list. However, no one is assigned to specifically any project and it's not clear what is going to be done and when.
From a product side, this is where I take the priority list and pass it on to my engineering, design, and data science colleagues. I want to understand from them what will actually be doable.
I expect to get back:
- Which project will get done
- The order in which the projects will get done (spoiler, sometimes it is different from the priority list and I’ll explain when that’s ok)
- Start and end date (or at least an approximation) of each
- Who is going to be working on each project
Who exactly is in charge of road mapping changes from company to company. As a product manager that frequently overreaches into engineering territory regarding architectural decisions, roadmaps is where I feel there is a clear separation between product and engineering / design / data science management. If a priority list was well done, the product manager should just give it to the team managers and then get back a roadmap.
Unfortunately, in the past I've had to overreach also in this direction because I realized some managers don't know how to take a priority list and make a roadmap out of it.
So let's cover that.
How to roadmap
Making a roadmap out of a priority list is a surprisingly mechanical process which should not require too much creativity or improvisation. It mostly has to do with:
- How many people are on the team
- What are the capabilities of the people on the team
Capabilities here stand for both:
- What each member of the team is specialized in (frontend, backend, machine learning, mechanical engineering, thermal engineering etc)
- How good at execution each team member is
Usually b is correlated with seniority. However, that is not always the case and managers should be realistic when identifying how fast an individual person can work on a project.
A few prerequisites before starting the roadmap exercise:
- The team has to decide what is the timeframe of the roadmap. Are you roadmapping a whole year, a particular quarter, one month? This depends a lot on the size of a company. Usually, the bigger the company, the more further out you roadmap. Startups with a few months left of runway would probably just roadmap the next month or so. I found that one quarter is a great time frame for a roadmap horizon. In technology, things change fairly often. Usually, after a quarter, the underlying situation changes such that you would have to anyway roadmap again
- What's your discrete unit of time for a roadmap. Are you considering days, weeks, months? This is highly dependent on the horizon you decided in point a. For a roadmap that looks a month out, you usually want to plan in days. For roadmaps of a quarter out, I found that weeks work well
- You calculate the team and each person's capacity. There's many ways to do it and it really depends on the company's culture (especially the propensity to holidays). You would usually multiply the number of people available by how many discrete units of time are in your planning horizon. Then subtract some units for holiday or other reasons.
For simplicity, for the particular process and example explanation, I will assume we are roadmapping two quarters out and our discrete unit of time is weeks (as we do at Lyft). Let's say we have a team of one designer, one front-end engineer, and two back-end engineers, and that we consider one week per quarter of holiday time, that would leave us with a team capacity of:
- Frontend engineering: 2 quarters * 11 weeks (12 - 1 of holiday) * 1 person = 22 weeks
- Backend engineering: 2 quarters * 11 weeks (12 - 1 of holiday) * 2 people = 44 weeks
- Design: 2 quarters * 11 weeks (12 - 1 of holiday) * 1 person = 22 weeks
It's fundamental that we do not mix up team capacities of different capabilities. Unfortunately one designer cannot do the work of a back-end engineer (and the opposite is true). With AI it is increasingly the case that people can do work beyond their capabilities, though it helps to be clear. You can actually consider people that can do multiple capabilities. It does complicate things a little bit more, though still doable.
The process is the following:
- Starting from the top of the priority list, for each project, the effort of each particular capability is estimated. For example, a project that is the creation of a developer sandbox for a new platform feature might require 1 week of frontend engineering and 2 weeks of backend engineering.
- Once that is done, starting again from the top, people are assigned to each project, scaling down from the capacity available for each function
If a priority list is super long, then there is no need to go through all the projects. Once you have reached the team capacity per capability, then you can stop estimating effort and assigning people. It's going to be clear that once everybody has been assigned, then everything that has not been staffed will not be done.
The difference in capability capacity is what drives the possibility that certain projects lower down in the list might be staffed instead of projects higher up in the list. If I have a project at position 7 that requires two weeks of backend engineering and one week of frontend engineering, but I do not have any weeks of backend engineering left, then I will have to go lower down to find a project that the frontend engineer can work on. Let's say that a project in position 9 only requires front-end engineering. This will result in the project in position 9 being staffed, whereas the project in position 7 will not be. And that’s ok. That highlights that there is too much front-end engineering capacity for the back-end engineering capacity in the team and that should inform future team design decisions.
The art of capacity estimation
By far, the hardest part of a roadmap is to estimate the effort that a particular project needs and the speed of execution of your team. Those effectively dictate what will fit in the capacity and what won't. I'll write an article on urgency, which is how I usually approach things. It is really up to the managers to estimate that and the product manager should not be too involved in it.
However, excellent product managers can do very good estimation for engineering, data and design work. That is a very important skill to have to ensure that the team managers are not “sandbagging” and adding buffer to effort, to make them look good when they hit dates. Ultimately, if a team signs up for too little work, there will be questions from higher leadership, and trust me, it’s a spot you don’t want to be in. It makes the team look slow and lazy.
Conversely, estimating all projects too little will make you consistently miss dates and deliverables, which will make you lose trustworthiness.
As a product manager, I encourage you to let your managers come up with effort and execution speed, though you should challenge them whenever something doesn’t sound right. Just like a product manager needs to know why project A is prioritized higher than project B, an engineering / design / data science manager needs to be able to defend why they think that project C will take their team 3 weeks to complete.
Just like the impact estimation, it helps to have a single person on the efforting and execution speed estimation. I've seen engineering managers that pre-assign projects to a team member and then expect the team member to come up with the effort estimation. This, though, skews the estimates given better team members will estimate projects lower compared to less capable team members. Overall, you only have one reading of capacity for each project, so the unit of measure should be an average of what the team can do. Then better people of the team will execute faster and worse people will execute slower., but on average you should be able to hit the team capacity.
Usually only the manager of the team can make that judgment, so they should be the ones doing the first round of estimation, then asking the team to check on their correctness.
The backlog, the place of no return
It's perfectly okay to leave all the projects that have not been staffed in the roadmap at the bottom. Infamously, this is called “the backlog”. It represents the list of projects that may be done in case one of the staffed projects gets cancelled or, miraculously, gets done ahead of time.
Ultimately, once you have a roadmap, you can get rid of the priority list if you keep all the other projects that weren't staffed in the backlog (in priority order). A roadmap is more powerful than a priority list given it has specific timings on when projects will be done, which is a question that, as a product manager, you get all the time.
It helps to still have the capacity estimates for the top projects of the backlog, such that if there is the possibility of getting more capacity, it is clear what will be the next thing to be staffed.
The backlog test
The backlog is also a great place where to send all the requests that are made to the team that cannot be prioritized. In particular if you're a platform product manager (like me) you get low priority requests all the time that doesn’t make sense for the team to do (since there are many other things higher in the priority list).
It is a lot easier to keep good relationships with the customers if you “add them to the backlog” instead of saying “we won’t be doing it”.
Naturally, when things are not important, people forget what is actually in the backlog and won't bother you about it. So putting things in the backlog is a lot faster and efficient rather than convincing them that the project is not worth your time. If they do come back and bother you relentlessly about it, then maybe it is something that should be prioritized after all… Insistence about a feature usually shows that it is very important for the customer. I call this the “backlog test”.
As this section is called, rarely things make it out of the backlog.
To avoid an ever increasing backlog, it is good, usually once a year, to do a backlog purge. I tend to do it during the New Year's holidays to start the year clean. The team is always nervous about it, since there is a perception that if something made it to the backlog, then it must have been important at some point.
Don't let yourself be scared about it. Never have I had to fish up a feature from a deleted backlog backup. The backlog is the place of no return and there’s a reason for it.
Regardless, if a feature is important, it can always be added to the priority list and roadmap again.
Onwards to execution
At this point you should have a prioritized list of projects and a roadmap which tells you specifically what your team will do and when. You should have enough information to defend your roadmap against objections and a handy place called the backlog where to send all requests that don't fit in the team priorities and capacity.
Now, the key is to handle execution and make sure that you hit those dates or, even better, that you deliver ahead of them. You'll find a few more articles on that.
Another good one to follow here will be the article on urgency given that is the core to making sure that your team overall delivers on the capacity that they have.