Improving user retention through personalised learning
My Role
Researcher & Product Designer
User research, Data analysis, Strategy & User flows
Team
Ashwin Manghat, PM
Abhishek Srivastava, DM
Mayank Singh Karadam, IxD
Anuvrat, SDE
Kshitij , SDE
Timelines
3 Months
Overview
Following the acquisition of Coding Ninjas by Naukri.com's parent company Info edge, our core product CN Studio was primed to supplement the learning needs of users on Naukri
I led the research to understand the needs of this new incoming user group and contributed to the strategy of this project.
As a result, we were able to improve user engagement on the platform and reduce our drop off numbers
Context
A huge opportunity for collaboration between Naukri & Coding Ninjas
Using our individual strengths for collective good
The plan was simple, Naukri could leverage CN Studio's massive collection of learning resources, to position itself as a holistic career platform instead of being just a job board.
While CN could leverage Naukri's strong recruiter pool to provide reliable job opportunities for it's users.
The Problem
Our experiments were failing
We were on quest to find the right learning resources for Naukri users
CN Studio has a repository of thousands of blog articles, coding problems, community forums, contests, interview guides and so much more.
Unsure of what would work we started running quick experiments that tested out each of the learning resource available with us against the users in Naukri, but we saw the same concerning results for every single experiment.
CN Studio has a repository of thousands of blog articles, coding problems, community forums, contests, interview guides and so much more.
Unsure of what would work we started running quick experiments that tested out each of the resource category available with us against the users in Naukri, but we saw the same concerning results for every single experiment.
User engagements metrics were at an all time low on our platform for this new cohort of users from Naukri
81% avg increase in bounce rate
97% drop in session time for a Naukri user on CN studio
Research
What were user expectations when it comes to consuming coding related content?
My research process
Qualitative research
Heatmaps & data analysis on user behaviour
Comparative study
Studying Naukri user's behaviour metrics against our user
User interviews
Interviewed 21 Naukri users who dropped off after showing interest in a resource on CN Studio
But then, only 2 people showed up to the user interview calls
I sent out a screening survey and booked 30 user interview calls with participants, but hardly 2 people showed up to the calls.
So I evaluated my approach, and restarted the whole process, this time, we started doing telephonic interviews and it worked!
I sent out a screening survey and booked 30 user interview calls with participants, but hardly 2 people showed up to the calls.
So I evaluated my approach, and restarted the whole process, this time, we started doing telephonic interviews and it worked!
Insights
We uncovered some interesting findings about our user
Some key insights include
User cannot code in 5 - 10 mins
Naukri's users check the app in pockets of free time, they can't start solving coding problems suddenly.
Difficult to code on mobile devices
Naukri’s user base was mobile first while our coding resources were geared to be desktop first
Confusion on how to return to CN Studio
An interested user is unable to return back at a later point in time
Key discovery
Both our user personas had extremely different goals and needs
The challenge
How might we establish a learning habit for Naukri users on CN Studio
Workshop
We put on our thinking hats & started to look for solutions
Bringing the whole team together
I organised a 2 day workshop involving essential stakeholders across product, design and business teams
We collectively brainstormed on various ideas that could help us solve user pain points identified in the research stage
I organised a 2 day workshop involving essential stakeholders across product, design and business teams
We collectively brainstormed on various ideas that could help us solve user pain points identified in the research stage
Wireframes
Developing our ideas further
The coding interview preparation plan
We knew by this point that Naukri users's learning was focused on preparing for upcoming interviews
Based on a user's target company, days until their interview and the level of preparation they intend to do, a personalised set of coding problems, flashcards and quizzes are generated for the user.
We knew by this point that Naukri users's learning was focused on preparing for upcoming interviews
Based on a user's target company, days until their interview and the level of preparation they intend to do, a personalised set of coding problems and flashcards are generated for the user.
What is the best way to establish a return habit to CN Studio?
We had 2 approaches to present this presonalised plan to users.
In option 1 we were dividing the content up into all the progress that a user needs to make over the span of a week and presented it to them.
In option 2, we were approaching the plan by days. We only present in front of the user what they need to do on a given day.
The user can set their pace of learning
Best suited for users casually preparing over the weekend
Users can get overwhelmed by the amount of content
Difficult to establish a daily habit
Potential to establish a daily habit for the user
Best suited for daily practice and consistent preparation
Less overwhelming due to smaller everyday progress
Less flexible implementation of a learning plan
Visual Design
Bringing our concepts to life
Making the learning plan flexible for the user
After the recommended plan is generated, none of the content is locked, so the user has complete flexibility to browse everything and set their learning pace.
We also allow the user to edit their plan to increase or decrease intensity, add more target companies etc based on their need.
Making it easy for users to find their way back to CN Studio
Our research revealed that despite users finding value in our content, they were unable to find their way back.
Due to certain constraints creating a single static touchpoint as an entry way for the user was not possible, so we found a work around with the help of WhatsApp
Conlusion
What we achieved
We finally had some great news!
42.7% decrease in bounce rate
30% increase in returning users to CN Studio
190% increase in time spent on CN Studio for returning users
A renewed sense of direction and learnings
This project was a massive undertaking. We had to unlearn what we thought we knew about our users and re learn our concepts. We now understood our users a bit better and could work towards serving their unique needs.
I am grateful for the chance to collaborate with multi level stakeholders, contribute to strategy, obsess over numbers, research and work with product & my team to execute this project.
Next project: Optimising student enrolment experience
Next project: Optimising student enrolment experience
Next project:
Optimising student enrolment experience
Improving user retention through personalised learning
My Role
Researcher & Product Designer
User research, Data analysis, Strategy & User flows
Team
Ashwin Manghat, PM
Abhishek Srivastava, DM
Mayank Singh Karadam, IxD
Anuvrat, SDE
Kshitij , SDE
Timelines
3 Months
Overview
Following the acquisition of Coding Ninjas by Naukri.com's parent company Info edge, our core product CN Studio was primed to supplement the learning needs of users on Naukri
I led the research to understand the needs of this new incoming user group and contributed to the strategy of this project.
As a result, we were able to improve user engagement on the platform and reduce our drop off numbers
Context
A huge opportunity for collaboration between Naukri & Coding Ninjas
Using our individual strengths for collective good
The plan was simple, Naukri could leverage CN Studio's massive collection of learning resources, to position itself as a holistic career platform instead of being just a job board.
While CN could leverage Naukri's strong recruiter pool to provide reliable job opportunities for it's users.
The Problem
Our experiments were failing
We were on quest to find the right learning resources for Naukri users
CN Studio has a repository of thousands of blog articles, coding problems, community forums, contests, interview guides and so much more.
Unsure of what would work we started running quick experiments that tested out each of the resource category available with us against the users in Naukri, but we saw the same concerning results for every single experiment.
User engagements metrics were at an all time low on our platform for this new cohort of users from Naukri
81% avg increase in bounce rate
97% drop in session time for a Naukri user on CN studio
Research
What were user expectations when it comes to consuming coding related content?
My research process
Qualitative research
Heatmaps & data analysis on user behaviour
Comparative study
Studying Naukri user's behaviour metrics against our user
User interviews
Interviewed 21 Naukri users who dropped off after showing interest in a resource on CN Studio
But then, only 2 people showed up to the user interview calls
I sent out a screening survey and booked 30 user interview calls with participants, but hardly 2 people showed up to the calls.
So I evaluated my approach, and restarted the whole process, this time, we started doing telephonic interviews and it worked!
Insights
We uncovered some interesting findings about our user
Some key insights include
User cannot code in 5 - 10 mins
Naukri's users check the app in pockets of free time, they can't start solving coding problems suddenly.
Difficult to code on mobile devices
Naukri’s user base was mobile first while our coding resources were geared to be desktop first
Confusion on how to return to CN Studio
An interested user is unable to return back at a later point in time
Key discovery
Both our user personas had extremely different goals and needs
The challenge
How might we establish a learning habit for Naukri users on CN Studio
Workshop
We put on our thinking hats & started to look for solutions
Bringing the whole team together
I organised a 2 day workshop involving essential stakeholders across product, design and business teams
We collectively brainstormed on various ideas that could help us solve user pain points identified in the research stage
Wireframes
Developing our ideas further
The coding interview preparation plan
We knew by this point that Naukri users's learning was focused on preparing for upcoming interviews
Based on a user's target company, days until their interview and the level of preparation they intend to do, a personalised set of coding problems and flashcards are generated for the user.
What is the best way to establish a return habit to CN Studio?
We had 2 approaches to present this presonalised plan to users.
In option 1 we were dividing the content up into all the progress that a user needs to make over the span of a week and presented it to them.
In option 2, we were approaching the plan by days. We only present in front of the user what they need to do on a given day.
The user can set their pace of learning
Best suited for users casually preparing over the weekend
Users can get overwhelmed by the amount of content
Difficult to establish a daily habit
Potential to establish a daily habit for the user
Best suited for daily practice and consistent preparation
Less overwhelming due to smaller everyday progress
Less flexible implementation of a learning plan
Visual Design
Bringing our concepts to life
Making the learning plan flexible for the user
After the recommended plan is generated, none of the content is locked, so the user has complete flexibility to browse everything and set their learning pace.
We also allow the user to edit their plan to increase or decrease intensity, add more target companies etc based on their need.
Making it easy for users to find their way back to CN Studio
Our research revealed that despite users finding value in our content, they were unable to find their way back.
Due to certain constraints creating a single static touchpoint as an entry way for the user was not possible, so we found a work around with the help of WhatsApp
Conlusion
What we achieved
We finally had some great news!
42.7% decrease in bounce rate
30% increase in returning users to CN Studio
190% increase in time spent on CN Studio for returning users
A renewed sense of direction and learnings
This project was a massive undertaking. We had to unlearn what we thought we knew about our users and re learn our concepts. We now understood our users a bit better and could work towards serving their unique needs.
I am grateful for the chance to collaborate with multi level stakeholders, contribute to strategy, obsess over numbers, research and work with product & my team to execute this project.
Next project: Optimising student enrolment experience