How to Hire Great Product Managers, Part 3: Writing Interview Questions
The product manager role is a little bit different at every company and even across teams within a company. It’s okay to use generic interview questions you get out of a book, but you can get even better signal if you write your own questions specific for the role. That will help you say yes to candidates who have the strengths you need most, and no to the candidates who won’t have the skills to be successful in your company’s culture.
For example, at Asana, we‘re built for teams, so we need PMs who can think about multiple customer-types at once. We’ve also got a strong focus on ease-of-use and good design, which isn’t always the case at enterprise startups. On the culture side, we value approaching situations with curiosity from above-the-line and being able to explain the intentions behind decisions. The best PMs for Asana aren’t the best PMs for every company.
Basic approach for coming up with interview questions
My favorite way to come up with interview questions is to take a real work situation I’ve encountered and then switch it to not be about my product. I’ll think about times when a PM did something great that really helped the team to see if a candidate could do the same, or times when I’ve seen people really miss the mark, to make sure this candidate won’t make the same mistake.
Starting with a real situation helps ensure that I’m testing for something that really matters. For example, PMs at Asana don’t need to code, but they do work with engineers to get past technical roadblocks, so we can hone in on the level of skill we need by starting from real examples. Basing on real examples also helps the interviewer know how to probe deeper and recognize the difference between a good and a bad answer — this is especially useful when you introduce a new question and aren’t calibrated yet.
Switching away from your product helps ensure that you’re testing for the skill, rather than specialized knowledge. If you use your own product, it’s easy to forget how much an outsider might know or not know, and you might think certain ideas are obvious when they’re not. I like to switch to a product or area I’m not very familiar with so that I’m sure I’m not assuming too much about what kinds of background knowledge people have. This also makes for a better candidate experience because they won’t feel like they’re doing free work for you.
For example, a real product challenge I had at Google was making a tradeoff between changing the Contacts API to give better expected behavior on the iPhone contacts sync or maintaining backwards compatibility. Rather than directly asking about that situation, I might make up a hypothetical scenario where it’s the Facebook Events API, but the same tradeoffs are at play.
Spicing it up a bit
Once you’ve got a really good grasp on what skills you want to test for, you can have some fun by looking for examples in the real world that would test those same skills.
For example, I *hate* how most TV remotes are designed. Every time I’m running into problems with one, I console myself by turning it into an interview question: How would you design a tv remote?
Sometimes I’ll read a book where I’m sure the author drew the wrong conclusion or misread the data… interview question!
When you’re going more abstract, be sure to double check which skills you’re looking for, and test it out on a few people before you start trusting the signal. Especially be careful for trends of what kinds of background info you’re assuming people know so you don’t accidentally exclude people.
Write up a Rubric
After you have your questions, a great way to get the most out of them is to write up a rubric for grading the answers. This makes it much faster to decide how someone did on the question, and helps calibrate across multiple people asking the same question.
There’s 3 parts to a rubric.
First there’s a breakdown of which skills you’re testing for on the question. For example it could be “Big Picture Design”, “Design Details”, and “Collaboration”.
Second, there’s a description for each skill of what excellent, ok, and, bad answers look like. Abstractly this can be whether they came to good or bad solutions, but it helps to call out specific cases, especially once you’ve interviewed several candidates and can detect patterns.
Last, there’s the instructions on how to combine all those pieces into a final hire or no hire decision. This could be a numeric threshold, or there could be certain answers that are an automatic no-hire. There can be some judgement involved, but it’s helpful to say something like “usually people we hire were excellent in at least one area”.
Want to learn more? Check out my first post about hiring great PMs and second post on How to hire for product mindset.
Got any questions or more advice? Share in the comments! Want more from me? Follow me on Twitter.
We’re hiring experienced PMs at Asana: https://asana.com/jobs/product
