Growth experiments: Testing growth hypotheses at low traffic volumes
Product thinking, User researches, Intentionaly
Meta is a mental-health service. It is a marketplace where clients find psytherapists. Smart algorithm allows our service to offer specialists who will best cope with a request. The client then selects an appropriate specialist and arranges a meeting.
In the fall of 2021, we worked on the stage of conversion rate from “Complete the survey” to “Chose a psychotherapist”.
After completing the survey, the client would get to the page of the most suitable specialist.
60% of users who completed the survey don’t choose a psychotherapist
We started by studying the problem — why clients who completed the survey don’t choose a psychotherapist? Together with the product manager, we studied 40 hours of screen recordings, conducted 30 problem understanding interviews and several ux sessions with users. Here’s what we learned:
User interviews — my favourite part of research
88% of users don’t understand whether the therapist will help to cope with their problem
We formulated several hypotheses on how we can help users:
We evaluated the hypotheses according to the criteria “Development time” / “Expected impact on conversion”, and we have prioritized the order in which the experiments shall be carried out.
First, we decided on the feature, which we call the “Top 5”. After completing the survey, a client sees a list of 5 most suitable specialists.
When the prototyping was completed, we conducted 8 interviews with users: we compared the previous and new solutions Side-by-Side to make sure that users see a qualitative improvement in the selection process.
Having received good reviews from users, we launched a feature in production. Later, we conducted experiments for two months on how and which list is best to be shown to customers:
According to the results of the experiment, we settled on the design version, which showed the best conversion results. We show clients the 5 most appropriate therapists with the opportunity to see all the right ones. The experiment helped us raise the conversion in that part of the funnel by 6 percentage points.
A surge in conversion was noted at the end of february, which was caused by the outbreak of war in ukraine
With little traffic, conversion fluctuates — it shows irregular behavior. Under these conditions, it is difficult to conduct experiments on the change in conversion: it is not always clear what affects growth and fall — whether its a product solution or fluctuation.