Subtitles section Play video Print subtitles SPEAKER: 86% of app ideas are born from a developer's personal pain. These ideas are form apps nobody needs. Developers believe research with users is a waste of time. They perceive their app as a coding exercise. To validate their idea, they ask their sister if she likes it. She says yes. TOMER SHARON: Meet Will and Dana. Will and Dana are the co-founders of Noteo, a note-taking app nobody needs. Dana is 26 years old. She's an MIT Computer Science graduate, she's a Trekkie, and she has been coding since she was eight years old. Will is 25 years old. He's a Stanford Computer Science graduate, and he loves Star Wars and LEGOs. They met over a weekend hack-a-thon here in San Francisco about a year ago, and they liked each other's way of thinking. So about six months ago, they started Noteo, and they've been working on it since. Will and Dana are failing. They're crashing, they're burning, and they're sinking $200,000 of seed money that they got, without even knowing what went wrong. I'm here to help you avoid their mistakes. I'm here to help you execute the right plan. Hi, my name is Tomer Sharon. I'm a Google Search User Experience Researcher, and I have been studying dozens of thousands of users, of people. Learning about what they need, what they want, and how they use apps and other products. I've been helping Google Search and startup teams come up with products that meet human needs. Going back to Will and Dana. They have six big, big problems. Their number one is they did not fall in love with a problem. They rushed into launching a landing page for a product they weren't sure of. They had personal pains related to note-taking, and they were sure that this is something people needed. Just to be very sure, they launched this landing page. And I did put an arrow here, but I'm going to use my lightsaber. They launched this landing page. And they collected people's email addresses, trying to figure out if they're interested, therefore if the app is needed. But the only question that they were able to answer is the one that you see down there. Are people interested enough to give them their email addresses? They were never getting or gathering information about what users need. Their number two problem is that they learned from friends. They interviewed seven friends and family members. Now family members and friends are always happy to give you feedback. The problem is that they're biased. How can they hurt your feelings? They're your friends. They're your family. Of course they like your idea. Sure, they'll use it. No doubt they'll pay for it. A lot. Third problem. They listened to users. Now I know coming from a User Experience Researcher, it sounds kind of weird, but bear with me here. The first rule of research is don't listen to users. Instead, observe their behavior. When Will and Dana asked their friends and family, would you use this app, would you pay for it, how much you pay for it, they got really good answers. People liked it. But they forgot-- or didn't know, or ignored-- what social psychologists know for almost 100 years now. We humans are very, very bad at predicting our own behavior. Here are two studies that were done during that time. This is a study from 1937. The researchers went into classrooms and asked students-- they passed along a questionnaire-- and they asked students, would you cheat in an exam? And the students answered. A few weeks after that, they came back to these classes, to these students, and together with the teachers, they were giving an opportunity for the students to cheat without them knowing. And surprise, surprise, there was close to zero correlation between what people said about their behavior and what they actually did. 1937. Now Will is kind of sarcastic about studies from 77 years ago, so here's a study from 2012. This was done in the UK with dozens of thousands of people. The researchers went into public bathrooms in gas stations. And they asked people coming out of the bathrooms, did you wash your hands after you finished your business? 99% percent of people said of course, yes. But the researchers, they installed electronic recording devices on the faucets in the bathrooms. And they actually knew exactly how many people did wash their hands. So 32% of men and 64% of women actually did wash their hands. There's a very, very big difference-- very big difference-- between what we say we do and what we actually do. There are many reasons for that. Some people would say, they're just liars. They're not. We are having trouble predicting our behavior. There are many reasons for that. When Will and Dana ask their friends and family, or anyone else, would you use our app? They're asking them to predict the future. Again, we humans are very bad at it. Their fourth problem is that they didn't test the riskiest assumption. Every product or idea comes with a set of assumptions or beliefs. The riskiest assumption is the one that is core to the idea. And it's also unknown. Or, the riskiest assumption, if that's not true-- if the riskiest assumption is not true-- the whole idea falls apart. What they could have done, Will and Dana, is-- and this is just an example-- they could have assumed that this is risky. Smartphone owners are aware of their ineffective note-taking habits. If this is not true, they have no reason to develop Noteo. They didn't do anything to validate or invalidate this riskiest assumption. Their fifth problem is what I call, that they're having a Bob the Builder mentality. They rushed into developing a product. This is what they know best. This is what they know how to do. They rushed to developing a product, and they launched a minimum viable product without even knowing what for. They kept asking themselves, can we build this note-taking app, instead of asking if they should. In their mind, this was a coding exercise. And a coding exercise doesn't require any insights from users. And their last problem is that they were perfectly executing the wrong plan. They developed a beautiful app nobody needs. They could have validated or invalidated three things. The problem. Is there a problem, a note-taking problem in this world that people care about? They could have validated the market. Are there enough people who have this problem and care about this problem? And later on, they could have validated their product. Is our product solving this problem for this market? And I give credit to Laura Klein, who's sitting right here. Thanks for that. They are doing a few things very well. I want to mention two of these things. So in the past year, I interviewed 150 app developers and startup founders. And I wanted to know what are the questions that they ask themselves about their users, or potential users. And the good thing that I found was that they ask the right questions. These are some of the results. I'm just going to go over a couple. 97% of them ask, who are my customers? 95% ask, do people need my product? Probably the most important question to ask. 89% ask if the product is usable. These are very good and important questions to ask. The second thing that is going well for them is that they understand priorities. They understand what are the questions, what are the most important questions they need to ask. And they know when to ask these questions. I'm completely ignoring the invalid, unreliable way they answered those questions. But just having the right questions and knowing when to ask them is a very, very positive thing. So up until now, I talked about their six problems and a couple of good things for them. What I want to do next is suggest a solution. Suggest a way to execute the right plan. So say hello to lean user research. Lean user research is a discipline that is providing insights into product users, their perspectives, and abilities to the right people at the right time. Excellent lean user research is of high quality. It's not crappy research. It's impactful, meaning it's not just interesting, but you actually have something to do with it. And it's fast, because nobody wants to wait for research. Next, I'm going to introduce you to three lean user research techniques. The first one is called experience sampling, the second is observation, and the third is fake doors. Let's start with experience sampling. During an experience sampling study, research participants are interrupted several times a day to note their experience in real time. This is a very unique way of mining their reality. Experience sampling is coming from a research technique that was called pager studies. This was developed in the 1950s. Back then, researchers handed pagers to their research subjects and asked them a question several times a day. For example, how do you feel? Where are you? And things like that. Or what do you do? And they collected these responses and understood the lives of their users, or research subjects. The key in experience sampling is asking the same question over and over again. So for example, if you want to know-- if you ask people, what annoyed you in the last couple of hours? Imagine you asked that question five times a day, for five days, and you have 100 research participants. Quick math, you collect 2,500 data points. This is a huge, huge, insightful, useful body of knowledge. I want you to try it out. If you sit nearby the screen, you can use the QR code. If not, access this URL right now. Yes, do it right now. And even if you watch at home, you can do that. And you have an experience sampling question there. A sample question related to Noteo. It works! Answer the question, and we'll go over your answers in a minute. And I'm going to play with my lightsaber. Don't futz with it too much. The URL is working too. All right, I'm moving on. So imagine you are asked that question five times a day, for five days. You're not always going to have an answer, but you will in many cases. Let's go over sample responses. So here we have 31 responses to this question. If you look at it-- just eyeball what you see here-- very, very quickly you can understand that there are several groups of things you can learn here. Let me color them for you. Some people are writing down lists. Others are writing down ideas. And others are just sketching stuff.