Many of my free product help calls are about ways to pursue customer development and gather VOC, bootstrap new product efforts, and apply lean principles to product management activities. I recently had an email Q&A with a fellow PMer in which I shared what’s worked for me. So I decided to post my answers here — of course, your mileage may vary, but perhaps there may be some nuggets that may help you.
Voice of the Customer (VOC)
Besides web analytics, do you have any tools you use to facilitate getting VOC from the market? Customer interviews are great, but my experience is that it can be hard to get this feedback on a continuing iterative basis. Are there tools you use to make sure you have the #1 customer problem identified?
- For identifying the #1 problem, nothing beats interviews. You need qualitative feedback, especially at the early stages.
- Solution hypothesis can be also be validated qualitatively via interviews, as well as quantitatively — e.g., watching engagement metrics on a feature.
- Regardless of the stage of the product, I always dedicate time every week to speak with my customers.
- For now, we capture this feedback in Google Docs. Works well enough.
- That said, lots of tools to choose from: ListenLoop, Survey Monkey, Google Forms, UserVoice, and Intercom.io, to name a few.
- Sometimes plain old email and phone work best!
- UserTesting.com is an option for usability testing. 5 respondents is good enough.
- Can also ask for feedback directly within the app itself. Many apps, like Jira and Basecamp, now do this.
- Which tool you use depends on the type and frequency of the VOC you’re looking for, and the lifecycle stage of your product.
What do you do to create your hypothesis and metrics, and at what stage in the process? Ash’s model in the Lean Stack works to an extent, but it is focused more on startups as you know.
For startup product, yes, at just about every stage we’re testing hypotheses — see my post on using validated learning for a new product idea. If you’re doing problem validation right before the sprint, it’s too late.
For a launched product, depends on the nature of the hypothesis to be tested. If it’s an optimization type hypothesis — e.g., do customers convert at a higher rate with copy A or copy B — A/B testing or engagement metrics may work fine. Tools like Optimizely can help.
If it’s a more pivot vs. persevere type test, then it depends on the specific component of your product strategy or business model you’re testing. For example, if you’re reliant on lead gen to fuel inside sales, testing sales pipeline metrics, product positioning and sales commissions becomes important. If you’re an e-commerce play reliant on SEO/SEM, those are the metrics to focus on.
I have recently tried to combine problem/solution interviews into one interview, because l found that at the beginning of the Solution Fit interview I could test to make sure that I had the #1 problem nailed. As you stated in a response to one of my comments on your blog, you find that with an existing customer base you often have a good idea of what the #1 problem is, and that you can often just verify this in the Solution Interview.
I’ve done both: combined and separate. Pure problem interviews are great for immersing yourself into your customer’s world — what they do today, where they feel the pain, and how they articulate it. However, it is true that sometimes you need to get them to react to something to give some structure to your discovery.
There’s also an aspect of knowing your customer. If it’s a new product to existing customers, I may already have a pretty good idea of their pain points, and how well they articulate their problems. So I may combine the problem and solution interviews. If it’s a new product to new customers with whom I’m less familiar, I may prefer a pure problem interview to allow me to understand the customers better.
Ultimately, just need to use good judgment based on the nature of the problem, the solution you’re envisioning, and your target customer.
As you preach, long MRD/PRD’s make no sense. However, there is some need to provide our business analyst with some form of requirement. What do you do in these cases? I have a Feature Requirements doc I created, but I sometimes feel it is overkill. What would you recommend?
- We don’t write MRDs, PRDs, or any sort of traditional functional spec. It’s a total waste of time.
- If we have time, budget and bandwidth, we’ll always get a Designer to create the screens before getting it to Engineering. Doing otherwise is a waste of Engineering’s time. Only caveat is I keep my Engineering Head / Chief Architect informed and involved.
- Since the Designer is the recipient of the stories, we can write them at a fairly high level. We don’t get hung up too much on granular acceptance criteria — the use case, user goal or job story is much more important at this stage.
- As such, at times we’ve provided just a 1-pager with bullets. Because design is iterative, we lean more heavily on interactive ongoing conversations than paper.
- If we don’t have time, budget or bandwidth for a designer, PM just hacks the screens — Balsamiq, ppt, whatever. Not ideal, but sometimes you just have to make do. I’ve literally sent a photo of a whiteboard doodle, and written the story around it.
- As much as possible, for major new features or flows, we will do customer validation. Yep: phone interviews with screen shares. 5 may be good enough.
- We don’t have a Business Analyst, so PM writes the stories directly, including me. It’s not hard, and removes a middle man. Not saying a BA/BSA is never needed — just saying in our case we haven’t had the need for one, and have managed fine.
- When ready to provide the stories to Engineering, we do write more granular level stories. Granularity is determined by the detail put into the screens. Let’s say we had a designer mockup every detail — every click, button placement, font, color, and even copy —then it’s just easier to add the mockup as support documentation and point to it in the acceptance criteria. If the design was more high-level, like a ppt hack or my silly whiteboard doodle, naturally I need to provide more specificity.
- We use Aha.io to capture ideas, convert them to features, write stories, attach documentation, do prioritization, and map out an executable roadmap. It has some super cool features:
- It will spit out a req doc from the stories you write. This is helpful if you’re using a 3rd party dev firm and need to provide them a written doc during the initial planning stages. So we no longer need to use MS Word.
- It has an awesome Jira integration feature, where I literally click a button and boom, everything is automatically placed in the Jira backlog.
- It allows me to publish out status on our roadmap execution to senior execs. This is extremely helpful, as it gets me away from PowerPoint and Excel crap.
Ultimately what really matters is joint understanding between PM and Engineering. If there’s a good relationship, both sides can come to an agreement on how the reqs should be delivered. I’ve found regardless of how the reqs are documented, ongoing conversation between PM and Engineering is the key.
Want to learn more? Faced with similar challenges? Just pick a slot on my FREE product help time!