What Sherlock Holmes can teach Product Managers!
“Sherlock knows the kind of questions to ask to ordinary people”, says Watson when asked why Holmes is such a successful detective. Replace ‘Detective’ with ‘ Product Manager’ and the answer would retain its relevance!
For Product Managers, asking the right questions is arguably the most important skill to cultivate and implement. Since a PM often acts as a conduit for information flow between various work streams, remove the correct questions and what we’ll be left with are noises from different directions. So how to ensure that the sound coming in from each department consists of coherent notes and not jarring ones?
Knowing what questions to ask is as much a science as it is an art. While there are established frameworks to guide your thought process, a fair amount of customisation is always needed depending on the respondent profile, her answers and even her manner of answering. As PMs, we need to continuously engage our stakeholders with pointed questions to understand not just the issues, but also the concerns behind those issues.
In this post, I want to share one such example in which we used the power of strategic questioning to dive beneath the surface and uncover a latent risk.
A few years back, I was working on a cloud based ‘Enterprise Content Management ‘ system — a fancy name to describe the creation and maintenance of a repository for all content generated by a company. Think of an access controlled storage space for documents like vendor contracts, purchase orders, product blueprints, marketing creatives etc. Our product had started gaining traction and we were witnessing a steady uptick in all metrics. Revenue was increasing. So was the number of active users. As a PM, I should have been contended, right?
Lesson 1: If it’s not broken, why fix it? But, are you sure it’s not broken?
Delving deeper into the data, we identified two trends. First, while monthly user retention was healthy, (meaning customers were coming back to our application), product usage interval, that is the average duration between two sessions for a user, was slowly increasing. And also, the first derivative of this gap was positive, meaning those who were visiting less frequently were doing so with rising tendency.
Lesson 2: Question your top-level metrics. They tell only half the story!
The next step was to analyse the usage pattern on our application. The robust tracking of in-app user behaviour, which we had painstakingly built into our product, helped a lot during this exercise. What we observed was out of five primary modules within our app, there were two whose usage had fallen. Worryingly, past research into user behaviour had indicated that these were the two which generated maximum stickiness. In other words, if we could get customers to use these two modules more, the chances of them moving to a competing product decreased.
Lesson 3: Arm yourself up with data before entering the interrogation room!
We reached out to few such customers. Without disclosing the reason for this meeting, we asked them to list down what goals they wished to achieve through our software. Surprisingly, most answers contained at least a few needs that could be met via the same two modules which they were not using. Aha! So, users are not aware that our app already has these features. We just need to educate them more, maybe highlight these items prominently on the screen. Well, only if life were that simple…
Lesson 4: Build opinions based on feedback, but be ready to question them!
Prodding these users further, we realised many of them knew about these features on our app, had even used them initially, but then slowly stopped. That’s an encouraging observation, we thought. We just need to make these features more user-friendly based on the pain points they share. Can you tell us where you faced a problem, we asked them. Alas! We drew a blank. No one could really give a coherent reason why they felt uncomfortable.
Lesson 5: Some questions might not elicit response. Re-frame your question!
Can you walk us through how you you would achieve the same task outside of our application? This was how we changed tracks to gain more insight. We then asked them to perform the same activity within our app and compared the two processes. One thing stood out immediately — the time taken within our app was lesser. Then why this disconnect? Are you sure this is exactly how you perform this activity in real life scenario, was our follow-up question.
Lesson 6: Some answers are what people want you to hear. Question those answers!
Turns out, that was not the case. Let me draw a sample scenario to explain our finding. Think of a line manager in a factory sending a ‘Product Requirements Document’ to her department head for his comments and subsequent sign-off, via our platform. Now, these senior folks are often reluctant to adopt new fangled software like ours. So, even though they received the document through our application, rarely, did they send it back via ours. E-mail attachments were their preferred mode of communication. What these line managers then had to do, was download the attachment from the mail and re-upload it into our app to maintain tracking history. Ergo, more work. Better to conduct this entire transaction with higher management outside the app.
Lesson 7: Question the motives, not just the actions!
So, now we had it. Our customers were reluctant to admit that few of their higher-ups preferred email communication over the content management system their company had instituted. And thus the low usage of the module associated with sign-offs and approvals. We could finally put a finger to the heart of the problem!
How we decided to tackle this is a story for another day. I’ll end this post here with an apt quote from another favourite detective of mine, Hercule Poirot — “Mon Ami, let the suspects talk. Sooner or later, they’re bound to slip up something!”
Still not convinced about the power of asking the right questions? Well, just have a look at what Einstein has to say!
