Notes on Josh Seiden’s Replacing Requirements with Hypotheses

Video from Agile UX NYC 2012, link here

Quick distillation of the talk:

• Every decision you make about your offering is a design decision
• Every design decision is an hypothesis stemming from unspoken assumptions
• Declare then test your assumptions
• Engage the entire team in the feedback loop

Intro

Example Requirement (real, from the mid ’90s): Create an Internet Mouse that people use when surfing the Web on their TV from their couch

Repositioned as a hypothesis: We believe that people will pay for a device that makes it easier and more fun to surf the Web from couches in front of their TVs

Most of the time, hypotheses are a more effective way to manage your work than requirements

Flaws with requirements:

• Business owners express needs as “requirements”
• The dev team has no visibility to user/market need
• “The business” does the thinking alone; the dev team does the implementing alone
• This stool has only 2 legs

When you’re in production, building to a known standard, you want requirements

When you’re in an environment of uncertainty, you want hypotheses

• They are understood to be only provisionally true
• They express assumptions that need to be tested
• They are put forth in the spirit of a question

Lean UX has long held that “working software is the primary measure of progress”; Eric Ries’ Lean Startup evolved this into “validated learning is the primary measure of progress”

By constantly testing and iderating, you lower the risk of failure

A hypothesis is a proposed explanation of the way things work:

• We believe that __________, and we’ll know that we’re right when we see this signal: _____________.
• Sample signals: People use your mockups without trouble, people offer to pay when we offer to give them the mockups

Techniques

1. Identify assumptions
2. Express assumptions as hypotheses
3. Test the riskiest assumption first
4. Break your hypotheses down into testable parts
5. Use MVP concept to test your hypotheses
6. Get out of the building!
7. Lather, rinse, repeat

Methods

Declare your assumptions:

• What assumptions do you have about your customers?
• If they are proven false, will that cause you to fail?

Probe deeply for assumptions:

• Who is the user? Who is the customer?
• Where does this product fit into their life or work?
• What user problems does our product solve?
• When and how is our product used?
• What features are important?
• How should our product look and behave?
• How will we make money?
• How will we measure success?

Test your riskiest assumptions first–only pay attention the assumptions falling in the high risk/unknown quadrant below (imagine the vertical line!)

oooooooooooooooohigh risk

oooooooooooooooooo|

known _____________________________ unknown

oooooooooooooooooo|

oooooooooooooooolow risk

Keep backlogs of other assumptions and hypotheses so they don’t get forgotten

Method: Write the test first

• We believe that _____________. We will know we have succeeded when (qualitative) and (quantitative outcome). This will improve (acceptance criteria).

Method: Minimum Viable Product

• What’s the smallest thing we can make to test our hypothesis?

Real Life Example

The client is creating a widget that merchants plug in to their eCommerce site that provides a set of features which end-users value, while also giving the merchants insight since the widget generates metrics

• Problem: The company has a long feature backlog and isn’t sure what to prioritize to work on next
• Method: Built a storymap to identify assumptions and requirements (link to Jeff Patton’s work)–this got them all down in one visible diagram. Then voted for the biggest risks.
• The biggest risks were around transitioning from a freemium offering to a paid offering, so the value hypotheses was formed:
  • Our widget adds a valuable feature to a site’s page (free)
  • Our widget generates site traffic (free)
  • Our widget generates additional sales (free)
  • Our widget generates usable data for the merchants (paid)
• The riskiest assumptions were identified:
  • Customers will value our widget’s free basic functionality enough to choose it over free competitors
  • Customers will value our analytics enough to upgrade to a paid version of our product
• They created an acquisition funnel (from top): Awareness, Installation, Purchase, Repurchase, Refer. Then created hypotheses mapped against this funnel
• Experiment 1: Will they choose to install it?
  • As a requirement (boo!): Build an installer
  • As a hypothesis (yay!): Our customers will value our free widget enough to choose to install it.
  • MVP Experiment: Build a page that supports a concierge installer. (When users clicked the button, a member of the dev team called them and walked them through installation–thus testing “Will they choose to install it?” and not “Is the installer easy to use?”

Josh Seiden is the Managing Director of Neo and was a founding partner at Proof. He is the author of “Lean UX” with Jeff Gothelf, buy it here.