[Editor’s note: This piece was originally published on Medium in 2018.]
Here’s a quick list of questions to use when workshopping your product roadmap, prepping for a pitch/kickoff, or quickly filtering for well-thought-out ideas.
Some of these are tough…but will hopefully leave you smarter (and wiser, and more open to opposing views). I absolutely recommend workshopping your ideas with other team members and inviting them to challenge your thinking/assumptions.
[Related: 8 Hacks to Escape the Feature Factory and Focus on Impact]
Q: Would you bet $5,000 of your own money on the success of this effort? Why? Why not? On what terms? How would we know whether you had won/lost the bet? What might we learn early on that would encourage you to increase your bet to $10,000? Or decrease your bet to $1,000, or $0?
Q: Fill in the blanks. With this effort — in the next 6 months — there is a 95% chance we’ll generate [some outcome], a 50% chance we’ll generate [more of that outcome], and a 10% chance we’ll generate [even more of that outcome].
Q: How have we tried to solve this problem in the past? What happened? Do you mind sharing some data?
Q: What status quo are we hoping to disrupt? What is actually wrong with the status quo?
Q: Imagine you had to judge an internal competition to pick the best intervention to solve this problem. You’re responsible for writing judging criteria for your fellow judges. How would you rank submissions?
Q: What efforts have you taken to defeat confirmation bias, the availability heuristic, information bias, the IKEA effect, and other cognitive biases? How might a less-biased person view this bet?
Q: Describe the “good news” you hope to elicit as a result of this effort. How might you describe it in a company-wide presentation in a non-success-theater-like, non-fluffy way? Write the dream customer feedback tweet. How might the good news change in the short, mid, and long term as we realize the benefits?
Q: Every idea has a “backstory.” What’s the backstory here? How might you describe this effort to a new team member without the “back story”?
Q: Explain how this connects to the broader company strategy. Why is this a critical part/piece of the puzzle? Together with other initiatives, are we telling a cohesive story?
Q: Why now? Why is this the most important problem to solve right now? How might the financial outcome be different if we did this in six months, one year, or never? Explain how it “beats” a handful of other things you are considering.
Q: You’re about to occupy some % of the careers of a couple fellow human beings. Why should they come along for this adventure?
Q: Do you imagine a team sticking around to push the actual benefits here? Or do you expect a hand-off? Or a hybrid? What early indicators might indicate that we’ve placed a good bet and would signal that it is safe to move on to other things?
Q: Let’s say we don’t do this. What will actually happen to the business in the short, mid, and long term? To our customers/users/partners/team?
Q: Does this effort rely on other efforts to be successful? Describe how the efforts are related. If they are truly dependent, can/should we pursue them concurrently, or combine them somehow?
Q: Challenge yourself to cut the scope here by 75%. Would that deliver some value? Should we pursue that first, even if it expands the overall scope a bit?
Q: How much money are we losing each week (new opportunities or cost savings) by not solving this problem? How does that compare to the money we are losing each week by not solving other problems?
Q: In the spirit of challenging the sunk-cost fallacy, what might happen part-way through this effort that would persuade you to stop work?
Q: Describe the various forces and shifts that must come together to make this successful? What do we control? What don’t we control? What can we influence?
Q: Play your own Devil’s Advocate for a moment. Give me three good reasons why this isn’t a good idea. Now give me three good reasons why solving another problem is a better idea.
Q: Who will this impact? Say I wanted to identify the customers/users this will impact, what query would I run? How might I quantify the impact over time?
Q: Can we design some safe-to-fail experiments to help us solve this problem? Overall, how can we expand our “portfolio” of bets here, and get faster feedback?
Q: Can you give a brief summary of the data and insights underpinning this bet? How did this data, and these insights inform your beliefs?
Q: What are the known unknowns here?
Q: Where are you making leaps of faith in terms of user behavior? What is your plan to close the learning gap? When will you get this into the hands of real humans, with real data, trying to do their real job?
Q: Do you have a plan for regular usability testing? How often? How early? Have you set aside time to act on what you learn during these tests?
Q: How will you instrument your solution to measure outcomes and learn? Can you draw out a sample dashboard for this effort?
Q: What’s your plan to work “end-to-end” across the problem and the solution, such that we don’t arrive, finally, at a solution and discover the parts don’t fit together as expected?
Q: What is the customer/user behavior you hope to change? What will customers/users do more of, less of, start doing, and stop doing as a result of this work? How will that behavior change benefit the customer/user and the company?
Q: What information would make solving this problem easier? Are we missing insights that might improve our “batting average” here? How might we obtain that information?
Q: What problem might we solve, such that this problem would be a non-factor? Why aren’t we trying to solve that problem?
Q: How will we measure the impact and success of this effort in the short, mid, and long term?
Q: What is your plan to regularly reduce “benefits risk” (the risk this effort will not achieve the desired benefits) as the effort progresses?
Q: How might you describe the various other risks in this effort? How will you incrementally reduce those risk levels?
Q: What must we “get right” to succeed at this effort? Where can we be less-than-awesome, and still succeed? What should we ignore? What can be “suck at”?
Q: Who do we need involved to make this a success? Any special skills? Any special insights?
Q: What assumptions must hold true for this initiative to remain the most important thing we can work on?
Q: Is this the lowest hanging fruit? If I asked your team (or a team) to spend the next week fixing “small things with a big impact” would this top the list? Would it have a greater cumulative value? Say you only had two weeks to solve the problem (or chip away at the problem)…what would you try?
Q: Can you commit to a “pivot/proceed” decision point? When will we stop iterating on this? Please draw a line in the sand.
Q: It is a six months from now and this effort has failed. Describe three plausible reasons why it failed. Tell a good story.
Q: What is the leap of faith here? What must I believe without supporting data?
Continue Your Learning
Watch the on-demand webinar, “Framing Outcome-driven Product Bets,” to learn more from John Cutler about how to scope and hone your product bets.
