Visitor Retention Analysis with Adobe Analytics

This is my first blog post, a 3-part article originally published on LinkedIn as of Oct 2017. It explains how to define visitor retention metrics in Adobe Analytics.


A while back I got inspired by the way Amplitude put in practice the approach to user growth by Social Capital. Which revolved around the definition of Monthly Active Users (MAU) as being composed of three elements:

MAU(t) = new users(t) + retained users(t) + resurrected users(t)

Where new(t) are the acquired unique users this month; retained(t) are the users this month which were also users last month; and, resurrected(t) are repeat users this month that are not retained from last month but from some month prior to it. The following picture should clarify this visually:

The following identity also stands:

MAU(t-1) = retained users(t) + churned users(t)

Which essentially says that unique users last month either have come back and thus are retained this month or have not come back and hence have churned this month.

The value of breaking down MAU this way is that it allows you to segment your users so that you can regularly target them with marketing campaigns geared at retaining new visitors and resurrecting churned ones. In fact there is no point in making tons of investments in acquiring new users if most of them end up churning the following month: it is a lot of effort that quickly goes down the drain. On the contrary, consistent and healthy growth requires constantly engaging users so that they keep coming back month after month. And, as we know, it costs a lot less to retain than to acquire new.

But this is actually only a starting point for more advanced analytics that you can use to understand product retention cycles and therefore the successful patterns of user stickiness that can be leveraged for further optimization and user growth of your product.

So, I wanted to use this approach and turned to Adobe Analytics Workspace to see how much the tool would help me achieve that. The good news is that it is possible to define these metrics and underlying segments, however it is quite a painful process.

First thing first, this is where you will end up at the end of the journey:

Got excited?

Let me show you how I created all these metrics.

Please notice: the method described in the remainder of this post is no longer applicable since Adobe applied changes to Analytics Workspace on January 17th, 2019, which made this approach no longer viable.

For the most up-to-date method, please refer to How to Account for User Growth in Adobe Analytics in 2019.

Defining the New Users(t) Metric

I must give tons of credit to my friend Adam Greco for having defined the foundations for this solution in his post Visitor Retention in Adobe Analytics Workspace. By the way, I am the client he mentions in his article!

In his work, Greco introduces the concept of rolling date ranges to define a segment that captures users that have visited your website this month but not in the previous 24 months.

As Greco mentions, this solution has a drawback, which is “this month” in a rolling date range always means the current month. Therefore, if you define a segment to capture users that visit your site this month but not in the previous 24 months, that segment will capture Visitors only for the current month and will be empty for all months prior to the current. Which requires you to have to create one such segment for each previous month you want to include in your analysis. He has created 24 of them.

Apart from that pain, you will always end up with reports which display data on a diagonal, like Greco shows in his post:

While I haven’t been able to find an alternative to the creation of the 24 different segments, I have however managed to aggregate all of them into one single metric. Let’s see how that works step by step.

Step 1. Define New Date Ranges

Say you are interested in a time window that includes the current month plus the last 24 months. You then need to define 25 new date ranges to capture This month, Last month, 2 months ago, 3 months ago, and so on and so forth until 24 months ago. The idea is that the beginning of time for your analysis is 24 months ago. Or in other words that, as the time goes by, you are always interested to know how your website is doing in the last 24 months + current month (i.e. a rolling window by 25 months). However, you can set the length of this window to your liking. Here’s an example for the date range “10 months ago” when the time of writing is 19 October 2017:

Once you’ve defined 25 such date ranges, you will need to define 24 extra date ranges that you need to use to exclude that a user was ever a visitor to the platform before a given month. For example, at some point you will need to define the segment of New users (10 months ago) (see Step 2 below), and because these would be users that visited the website 10 months ago but not in the 14 months prior (remember, 24 months ago is the beginning of time), then you need to define a date range that includes the full month 11 months ago plus all 13 full months prior to it. Here’s how you do it:

Step 2. Define New Segments

Define 25 new segments, one for each set of new users during the last 24 months + current month. Here’s the example for New users (10 months ago):

Step 3. Define Calculated Metric “New Users(t)”

Finally, in order to create the new users(t) metric, use this simple formula:

Defining Repeat Users(t)

Once you’ve defined the new users(t) metric, defining the repeat users(t) is a piece of cake. What are repeat users on a given month t? Well, they essentially are all unique users that are not new users on month t. In other words:

repeat users(t) = MAU(t) – new users(t)

Or visually:

So here’s how you define repeat users(t) in Adobe Analytics:

Defining Retained Users(t) and Resurrected Users(t)

Now, notice that repeat users(t) are also the summation of retained users(t) (those that we retained from last month) and resurrected users(t) (those that we reactivated from a month prior to the last), as shown here:

repeat users(t) = retained users(t) + resurrected users(t)

Therefore, once you’ve calculated repeat users(t), you can either calculate retained users(t) first and then subtract it from repeat users(t) to obtain resurrected users(t)or, vice versa, calculate resurrected users(t) first and then subtract it from repeat users(t) to obtain retained users, as these two derived identities show:

(1) retained users(t) = repeat users(t) – resurrected users(t)

(2) resurrected users(t) = repeat users(t) – retained users(t)

I’ve tried both methods as a validation to one another and both work. In this article I will present the method suggested by identity (2) above, so I will calculate retained users(t) first and then derive resurrected users(t). Here’s how to go about it:

Step 1. Define New Monthly Segments for Retained Users

Like we did for new users, we need to define 25 new segments, one for each month during the last 24 months + current month. Here’s the example for retained users (10 months ago):

Here’s a problem though. You would expect a segment like this to give you visitors only for the full month if you go back 10 months ago from current month. However, given its very definition, Adobe Analytics will make that same set of visitors appear on both full months 11 months ago and 10 months ago. You can see this visually if you define the following metric:

If you add this metric to a free form table, here’s what you would see at the time of writing (29 October 2017):

However, we are only interested to know retained users for the exact full month that stands 10 months ago from the current month. How do we solve that?

Well, we need some sort of month selector so as to show the retained users on the retention month only (Dec 2016 in this case). That’s what we will do in step 2.

Step 2. Define Month Selectors

month selector is nothing more than a simple metric containing 0’s for all months except for the one we want to select, which needs to contain 1 as value. We will later multiply that metric by the metric calculated earlier for retained users so we will be able to get rid of the double attribution.

Keeping on using the example of 10 months ago, what you need to do is to use the previously created date range 10 months ago in order to build a new segment that only captures the visitors that visited your website on the full month that stands 10 months ago relative to the current month, like so:

Now, apply this segment to the unique visitors metric and here’s what you get:

We are just one step away from the month selector metric we want to build. You see, the value on Dec 2016 needs to become a 1. What’s the easiest way to make that value become 1? Just divide it by itself! However, as easy as this sounds, calculated metrics only allow for operations amongst metrics or between metrics and constant values (called static numbers in AA). So, in order to divide that value by itself, you need a metric that has that exact same value for the month in question. And what is that metric? It is simply Unique Visitors. So, all that we need to do to define the 10 month ago selector is to apply this formula:

And here’s how the result looks like on a free form table:

Exactly what we wanted.

Step 3. Calculate Retained Users(t)

Putting it all together, the formula for the the retained users(t) metric is the following:

Step 4. Define Resurrected Users(t)

As explained above, you now simply calculate resurrected users(t) as:

resurrected users(t) = repeat users(t) – retained users(t)

Defining the Churned Users (t+1) Metric

Last but not least, here’s the process to define the churned users(t+1) metric.

Step 1. Define New Segments

Similar to what we’ve done for new users and retained users, we need to define 24 new segments, one for each month during the last 23 months + current month. You will immediately notice a difference though: we’ve got one month less here, and that’s due to the very nature of this metric. We can’t in fact have churned users at time t. You easily see that as you define one of the segments. Keeping on using the example I’ve used so far, say you want to build the segment of churned users (10 months ago). You would define it this way by making use of date ranges previously defined:

As you can see, while we have defined these visitors as churning 10 months ago, this segment will however contain visitors attributed to the date range 11 months ago but contains no visitors for the date range 10 months ago. Why? Because of the mere fact that the segment definition above excludes users that have appeared on the platform 11 months ago. This means that we are in fact calculating churned users at month t+1 for any given month t.

Step 2. Calculate Churned Users(t)

All new segments having been created, here’s how you then compute the churned users(t) metric:

The multiplication at the end of the formula is required to make churned users a negative number (good for visualisation). It is a bit dodgy: I wish there was a way to multiply by -1 but there isn’t.

Conclusions

While getting to this point requires a heavy lifting, it is worth the effort if you are looking to use these segments and metrics as a basis for more advanced product analytics.

Please bear in mind that the retention cycle I’ve focused on for my analysis is monthly based. Therefore, these metrics will work fine if you break them down by month. But please notice that while you can scale them up to focus on longer intervals of time, such as quarters and years, as well as scale them down to analyze shorter intervals of time, such as weeks, days, hours, etc, you won’t be able to break them by quarters or days straight away; if you want to do so, you will need to redefine them by using longer or shorter intervals of time in your data range definitions.

8 Comments

  1. Hello Piermarco,

    Thank you for this. Its really helpful. May i ask you something..

    Is that retention rate real? 10,82% It means that about 90% of the trafic is all new users? I already implemented this on my website metrics and y i have got something similiar. Perhaps it doesnt make me sense at all, does it for you?

  2. Hello Sergio,

    I am really glad you find this work interesting.

    As to your question, I believe you are referring to the image I posted which I took from the article by Adam Greco.

    So the problem with defining new versus repeat traffic lies with the limitations due to the rolling window. I do not know how long your rolling window is. However, if it is about 24 months (like Adam suggested, and which I am suggesting too), depending on the type of website/business you run, I guess defining a new user as someone that hasn’t showed up in the last 24 months is pretty close to real. Right? I mean, if someone was there 2 years ago and is coming back only after 2 years, you probably can call it a “new” user now.

    Having this said, it does not surprise me too much that for certain websites 90% of the traffic on a given month might be made by new users. Which, I believe, is a great sign that you are getting a big potential for growth at the top of the funnel. The whole point about doing visitor retention analysis is to know then how your website is behaving with regards to that mass of new users: is that growth real (retained percentages raise over time) or not so much (all new visitors churn the month after)? I guess it really depends on case by case. Now, in this specific case Adam reported, to be honest, I would not know which website he ran his stats onto. However, those figures really don’t surprise me as I have seen them too.

    Hope this helps and please let me know how it goes with your website đŸ™‚

    Cheers,
    Piermarco

  3. Hi Sergio,

    That’s pretty straightforward; you need to define a new date range with start date set to “Start of current month minus 24 months” and end date set to “End of current month minus 24 months”. Then you need to define a new segment to capture all visitors of that date range. Essentially all visitors 24 months ago will be considered as new.

    Hope this helps.

    Best,
    Piermarco

  4. Hi Piermarco,

    A few members of my team have been looking through the Amplitude playbook lately and we LOVE that you’ve been able to replicate this view in Adobe. We found a hack that allows you to show churned visitors in the same month as the other metrics for that month.

    Create a calculated metric that looks at the cumulative of churned visitors for the last N timeframes, where N = 2. This will show the combined churn of this and the last month. Then subtract the current month’s churn from this cumulative metric, and you will end up with last months churn number showing up for the current month.

  5. Hi Nick,

    That’s interesting. I would love to see how you define the metric.

    Thanks!
    Piermarco

Comments are closed.