Newsplace, the news app I’m working on, was born 2 years ago. Just like a newborn it was constantly growing with new features and ideas and I met it when it already had tens of thousands of users on the one hand, while still searching for its core value proposition on the other. Currently the app offers a stories experience and a content categorized feed of aggregated news items which lead the user to the publisher’s website, where they can read the full article they saw in our app.
As a junior product analyst, for me Newsplace seemed like Disneyland, since it was in its early stages and I could influence the core value it offers users . Despite that excitement, I didn’t know how to expand my touchpoints with the product beyond the weekly value proposition meeting when a whole history existed before me and there was (and still is) a strong and talented team which was already running things, let alone to be proactive with initiatives of my own. While retention and engagement are the obvious things to optimize when trying to increase LTV, they are big ambiguous metrics and trying to improve them was too vague of a goal for me at that point. Together with Ran, our Product Manager, I found out that breaking down these words into the journey steps that users go through in our product, and choosing only one step to focus on, makes things much more clear and exciting. The step that got me excited was a user’s first session
A user’s first session with any product is critical to the value they’ll eventually get from it. This obvious fact quite literally hit me when I opened Tik Tok for the very first time and I HATED IT! Without onboarding, without easing into the app and what it holds, one innocent click on the TikTok icon led to a violent hijack of my precious phone with a full screen video combining music. For me, the instant urge was to shut the app down and delete TikTok from my phone forever. The only reason I continued watching videos was the initial reason I downloaded it in the first place- to finally understand what my colleagues are talking about when they are referring to the experiences in TikTok.
But like Newsplace, most products don’t have the luxury of retaining users beyond the first session solely on their solid reputations, and in order to do so, we need to optimize for the best user experience possible. This moment – a user’s first session- in which I found the ownership that I wanted to take upon myself in this product was the moment I started to feel passion and excitement and most importantly, started to love this product.
A user’s first session
So once I had a clear and bounded playground – a users’ first session, many questions could be asked to optimize the user experience in it:
- What does a ‘good’ first session look like (one in which users discover the value proposition and return to)? What is its flow?
- What is the biggest moment of disappointment a user encounters? (like the intimidating feeling I had from opening TikTok for the first time) Is it a long loading screen? Many clicks to get somewhere? A general feeling of being ‘lost’ in the app? Lack of content? Cognitive overload?
- What is the last action a user does before churning?
As the reference to my TikTok experience might suggest, I decided to focus on the reasons that cause users to churn, specifically in the first session.(In) To find these reasons, I worked together with Ran the PM and used Mixpanel to create a cohort of users who churned on their first day, in order to look at their flow in the app and hopefully discover what went wrong- what was the last thing they did/saw in the app that made them leave (or worse- hate us). That cohort with the flow chart were my ticket to the Newsplace Disneyland.
Diving into the data
Digging through the flow chart was a great opportunity for me to get my hands dirty and to really understand how and when we report each event. Things that initially didn’t make sense such as a ‘visible story’ event before the impression on the first screen in the app, became the chance to sit down with Amit, The Mobile R&D Team Leader, for the first time, and get explanations about the technical way these screens are loaded.
After eliminating irrelevant events that are sent automatically or those that don’t reflect the actual progress of the user in the app, something super interesting caught my eye- I noticed that 17% of users who see the user selection screen (which soon will be vastly elaborated upon) drop off. Additional 33% drop off after exiting the app while this screen appears or doing some other esoteric events that come before the stories. That means that more than 50% of the users who churn during their first day, do so during the first screen they see. 50% (!) of users leave us forever before even taking a sneak peak at our value proposition!
The second time I came to Amit with questions was much shorter as it came from a problem that was very easy to spot- a very long loading time for this simple screen that enables a user to choose which content they’d like to read today- top news or lifestyle content.
This first screen that caught my interest so much was added when Our UX Designer, Naama, came up with the idea for it when a user test she conducted revealed that the mere choice of content type showed an uplift in the ratings that users gave to the app. After this idea turned into an actual screen and was rolled out in the latest version- there was an uplift in other metrics as well which aligned with the conclusions of the user test. There was an increase of 13% in meaningful interactions and an increase in time spent in the app (in Mixpanel) among users who actively made a choice (clicked on either news or lifestyle) over those who were auto-transferred to the news. Despite the fact that this screen was intentionally shown for a long time to allow the app to load the stories, we failed to measure the friction this feature created and its impact on churn.
This revelation brought up very interesting questions – is the user drop worthwhile for us? Do we get more engaged users afterwards thanks to this feature? Does this screen increase engagement only on the first session(s) or does it also affect retention? Does this screen increase our KPIs only when a user is actively choosing the content or is just encountering it is enough? If we find out that it contributes to engagement but not retention- is it worth optimizing and if so- what to change in this screen? How to prioritize the ideas? If the user selection screen doesn’t increase measures- how do we integrate choice of content in another way with less friction?
Since I wrote the first draft for this post, many actions have been done to answer these questions, including re-evaluating and altering our north star metric, However many questions are still left to be explored. For me, this screen which I had no contribution to up until this point, was the gateway to the team, to being able to work independently in a proactive manner and to love this product.