More Notes on Tomer Sharon’s Validating Product Ideas Through Lean User Research

(The second of a two-part summary of Chapter 5 of Tomer Sharon’s new book,  Validating Product Ideas Through Lean User Research. Posted online at uxmatters.com, link here.)

This post covers the second part of Chapter 5 of Tomer’s book (how to answer the question “Do people want this product?”). (Read the first part here.)

So how do you use the Concierge or Fake Doors MVP experiment to answer the question “Do people want this product?”

Step 1: Choose an experiment type (see below all the steps for yet another method, the Contract MVP)

Select between the two types of experiments:

Choose the Concierge method (steps 2-4) when:

• You’re in exploration mode
• When you don’t yet have a product
• When dev hasn’t started
• When you don’t yet know a lot about how to solve the problem
• When you are very unsure of your data
• You want to discover other ideas that might be better perceived by your intended audience
• You want to discovery a new audience

Choose Fake Doors (steps 5-6) when:

• You want to learn about people’s reaction to the idea of a product or feature
• You want to collect data about the overall level of interest

Step 2: Design a Concierge MVP

Concierge MVPs manually provide the functionality of a potential product to the customer

To create one, think about:

• The value proposition of your idea
• The core benefit your product will offer its users
• Why would people need and use your product
• Look at your product’s hypothesis–it will possible already contain an idea for your Concierge MVP
• Example: Your idea is to create a recipe app for using what’s already in your pantry. You could offer a service where users sent in photos of their fridge and pantry, and a chef would email back a recipe for them
• Remember the quality of a manual service can be higher than your ultimate problem–make sure your manual service is achievable by your app

Click here to get  a Concierge MVP Board Plan template.

Tomer goes on to list several other examples of Concierge MVPs.

Step 3: Find customers and pitch the Concierge MVP

After you’ve created your pitch on the Concierge MVP board, find places nearby where your intended audience is likely to be and go pitch your product

If there aren’t such places nearby, pitch the product over social media. (Chapter 9 of Tomer’s book linked to above is about how to do this.) But ideas inlude:

• Post on Facebook groups and pages relevant to your product
• Tweet a short pitch and use hashtags
• Post on relevant Google Plus pages, LinkedIn pages,
• Post on any other relevant sites (ie special interests, other languages etc)

Once you’ve amassed about 5 emails/phone numbers from willing participants, pitch your idea. Make sure you’re clear about expectations and what’s going to happen in the beginning.

If not enough people agree with your pitch, have someone else pitch it. If that doesn’t help, try pitching in more locations. If non of this work, reassess your idea–people just might not be interested in it.

Step 4: Serve the Concierge MVP

You might serve it over the phone, via email or SMS, in person etc. Really consider SMS if possible: it’s the lowest fidelity, quickest method since it strips out all UI and UX considerations and most broadly accessible to a variety of folks

As you serve the MVP, be sure to:

• Keep customer interaction to a minimum–keep noise to a minimum
• Track key events (usability of your instructions, first impressions, first interaction etc)
• Proactively seek feedback! Ask them to share information with each step of the process
• Make changes. If something isn’t working, change it rather than continuing to test it
• Ask for payment. Make sure you prepped them that the service would eventually have a price. Fantastic if they are willing to pay. If they aren’t, try to get to the root cause of their resistance. See if you can change anything to lessen their resistance.

Click here for a form to use when tracking reactions, here’s a sample:

Step 5: Design a Fake Doors experiment

This experiment is usually a website that pretends to provide a product, feature or service to site visitors. It allows you to gauge potential user interest in your idea by collecting email addresses

• A landing or crowdfunding page: Launch a landing page and attempt to prove some commitment on behalf of its visitors, like requiring an email address. Crowdfunding pages can also give you a measure of interest levels, but remember their audiences are somewhat specific
• The button to nowhere: Design a button for the new feature and when users select it, give them a “coming soon” notice
• 404 testing: Launch an ad campaign with Google Adwords or Facebook Ads. But you should have some idea of what you are doing–a poorly written/designed ad campaign might miss its intended audience completely. The ads can lead to a 404 error page–requiring very little effort, but also being obnoxious and ungrateful to users. Much better to write a nice “coming soon” message and perhaps offer a $5 gift certificate from Amazon in exchange.

You can also do this in person, going to your org’s top users, pitching the idea and asking for feedback.

Step 6: Determine a Fake Doors threshold

It’s the ratio between how many folks were exposed to your message and how many responded. Think carefully about what a minimum ratio should be before launch–and stick to this ratio

Step 7: Make a decision

You’ve collected your data. Are you going to move forward? Remember, the majority of experiments fail. Use failure as a chance to rethink your messaging, consider reaching a different audience, tinker with the value proposition. Or pivot.

If your test fails, you shouldn’t just just pretend it never happened and move on as you had planned. Use this data to make some informed decisions about what try next–it doesn’t mean you have to abandon your dream!

You can also do this in person, going to your org’s top users, pitching the idea and asking for feedback.

The Contract MVP experiment

• Go to your intend audience and ask them to sign a contract for your product or service. Be totally honest that this is only an experiment, that it might never come to fruition, and they aren’t actually expected to fulfill the contract. If you get enough contracts, make your next move.
• Example: Order.In is a food ordering platform. They got a certain number of businesses in the area to sign a contract that they would be interested in an intranet widget for their corp sites which employees could use to order food. They got 500 signatures and went ahead to great success

Other methods to consider:

• Interviews
• Paper prototypes
• Pre-order page
• Wizard of Oz (Looks real on the front end, but the backend process is manual to avoid dev time at this early stage)

About Tomer Sharon (from uxsalon.com):

Tomer is H​ead of U​ser E​xperience at WeWork in New York City leading a team that designs work and living spaces, communities, and services around the world. Formerly a senior user experience researcher at Google Search, Tomer is the author of the book, “Validating Product Ideas through Lean User Research” (2016) and author of, “It’s Our Research: Getting stakeholder buy-in for user experience research projects” (2012). He founded and led The Israeli Chapter of the User Experience Professionals’ Association and has been preaching and teaching UX  at Google’s LaunchPad program, a bootcamp for early-stage startups around the world, in conferences, and at Treehouse and General Assembly. Tomer holds a master’s degree in Human Factors in Information Design from Bentley University in Waltham, MA. He is @tsharon on Twitter and Instagram.