How to Account for User Growth with Adobe Analytics in 2019

In this post I will discuss a new method to calculate user retention metrics in Adobe Analytics following the changes that Adobe implemented with the January 2019 release.

---

On January 17th, 2019 Adobe released a new version of Analytics Workspace that added more powerful user retention features to Cohort Tables. While this is great, I think these new features are still lacking an overcomprehensive approach to user growth analysis whereby retention metrics are made available out-of-the-box and can be used over and over again as any other metric within the Workspace. In other words, the retention/churn rates that you can now compute in Cohort Tables
(which are a great step ahead!) are ingrained within the Cohort Table visualization and can’t be taken out and applied elsewhere in Workspace like you would be able to do with the rolling segments and success events
I have previously defined.

Because of that, I think that my method is still valid since it provides one with a way to define retention metrics in the form of (rolling) segments and calculated metrics, which can then be used for further calculation, analysis and visualization within the Workspace.

One other thing that happened with the January 17th, 2019 release was that Adobe changed the boolean logic used in segmentation. In fact, several peers of mine who had previously implemented my method, contacted me since the last Analytics Workspace release mentioning that my retention metrics, which had worked for several months, no longer worked as they used to.

I’ve looked into it and found that the root cause was a change in the AND logic used in segmentation involving date ranges. While the AND operand was, before this release, producing similar effects as the THEN operand when applied to date ranges, it no longer does after the aforementioned release.

Though this is unfortunate, it also opens up opportunities for simplification.

In fact, I’ve come up with a new method that brings all the user retention logic to the segments and simplifies calculated metrics a lot. Hopefully this is also future-proof.

The reminder of this post is structured into paragraphs, one per each of the MAU rentention metrics (see definition of MAU and its component metrics here). But before, let me just introduce a new definition: in place of MAU we will be talking about MUV, which stands for Monthly Unique Visitors. This definition makes it clear that it is Unique Visitors (hence user devices), and not users, we are talking about. Nonetheless, though I haven’t tested it yet, I believe my approach should also work with the Adobe Analytics People metric. In that case, it makes sense to talk about MAU again.

Metric 1. New MUV (t)

Step 1. Define New Date Ranges

Compared to my my previous method, this step hasn’t changed. So, those that have defined new date ranges before can skip this step.

Say you are interested in a time window that includes the current month plus the last 24 months. You then need to define 25 new date ranges to capture This month, Last month, 2 months ago, 3 months ago, and so on and so forth until 24 months ago. The idea is that the beginning of time for your analysis is 24 months ago. Or in other words that, as the time goes by, you are always interested to know how your website is doing in the last 24 months + current month (i.e. a rolling window by 25 months). However, you can set the length of this window to your liking. Here’s an example for the date range “10 months ago” when the time of writing is January 20th, 2019:

Once you’ve defined 25 such date ranges, you will need to define 24 extra date ranges that you need to use to exclude that a visitor ever visited your website before a given month. For example, at some point you will need to define the segment of New MUV (10 months ago) (see Step 2 below), and because these would be users that visited the website 10 months ago but not in the 14 months prior (remember, 24 months ago is the beginning of time), then you need to define a date range that includes the full month 11 months ago plus all 13 full months prior to it. Here’s how you do it:

Step 2. Define New Segment

Define one segment that captures hits by new users in all the months included in the rolling window; make sure it’s defined at hit level and that you add 25 new containers to it. I am showing you here how this segment should look like when all the 25 containers are collapsed and, for simplicity, I am only showing the top 6:

And here’s an expanded view of a container; the example provided here is for New MUV (2 months ago) but you can use this same structure for all containers:

This says to Adobe Analytics: “capture all hits from visitors that did not appear in the rolling time window before 2 months ago and then appeared 2 months ago for the first time; make sure to only give me hits made by users 2 months ago”. This definition ensures that Adobe Analytics select those users, and only those users, that comply with this sequential definition by excluding their activity from other months in the time window. Notice that the sequential logic within this container (i.e. the Then relationship) must be set to capture visitors Only After Sequence.

Step 3. Calculate the Metric

Simply apply the segment defined at step 2 above to the metric Unique Visitors to calculate the New MUV (t) metric:

Metric 2. Retained MUV (t)

Step 1. Define New Segment

Like for metric 1, we need to define a new segment that captures all hits by retained visitors for each month in the rolling time window. Here’s how the segment should look like if one collapses all its containers (only top 6 containers shown):

And here’s an expanded view of a container; the example provided here is for Retained MUV (2 months ago) but you can use this same structure for all containers:

This says to Adobe Analytics: “capture all hits from visitors that appeared 3 months ago and then again 2 months ago; make sure to only give me hits made by users 2 months ago”. This definition ensures that Adobe Analytics select those users, and only those users, that comply with this sequential definition by excluding their activity from other months in the time window. Notice that the sequential logic within this container (i.e. the Then relationship) must be set to capture visitors Only After Sequence.

Step 2. Calculate New Metric

Simply apply the segment defined at step 1 above to the metric Unique Visitors to calculate the Retained MUV (t) metric:

Metric 3. Resurrected MUV (t)

By now it must be clear to you that we are using the same method over and over for each metric. So, for this metric too, step 1 is the segment definition.

Step 1. Define New Segment

Same as done before, this is how the Resurrected MUV (t) segment should look like:

And here’s an expanded view of a container; the example provided here is for Resurrected MUV (2 months ago) but you can use this same structure for all containers:

This says to Adobe Analytics: “capture all hits from visitors that visited the website sometime before 3 months ago, then did not visit it 3 months ago, and then visited it again 2 months ago; make sure to only give me hits made by users 2 months ago”. This definition ensures that Adobe Analytics select those users, and only those users, that comply with this sequential definition by excluding their activity from other months in the time window. Notice that the sequential logic within this container (i.e. the Then relationship) must be set to capture visitors Only After Sequence.

Step 2. Calculate New Metric

Simply apply the segment defined at step 1 above to the metric Unique Visitors to calculate the Resurrected MUV (t) metric:

Metric 3. Churned MUV (t+1)

Like already explained in my original method, this metric looks one month ahead. This is due to how I am capturing churn via sequential segmentation and exclusion logic. Keep reading and you’ll see why.

Step 1. Define New Segment

Like all previous metrics, here too the definition of the segment with collapsed containers should look like this (only top 6 containers); notice that it contains no churned MUV for the current month since they cannot be computed yet:

And here’s an expanded view of a container; the example provided here is for Churned MUV (2 months ago) but you can use this same structure for all containers:

This says to Adobe Analytics: “capture all hits from visitors that visited the website 3 months ago but then did not visit it 2 months ago; make sure to only give me hits made by users 3 months ago”. This definition ensures that Adobe Analytics select those users, and only those users, that comply with this sequential definition by excluding their activity from other months in the time window. Notice that the sequential logic within this container (i.e. the Then relationship) must be set to capture all visitors (Include Everyone).

Also notice that, capturing 2 months ago churn means capturing visitors that visited the website 3 months ago and didn’t come back 2 months ago. However, given the exclusion logic used in this segment/container, visitors from 2 months ago will effectively be excluded from the segment. But it will contain visitors from the start of the sequence; that is, those that were there 3 months ago (and then churned the following month). This means that we are in fact calculating churned users at month t+1 for any given month t.

Step 2. Calculate New Metric

You want to define this metric to be negative all the time, so that it shows up properly on visualizations such as a bar chart. Do that by applying this simple formula:

Conclusions

While Adobe has taken great steps ahead in enabling user retention/churn analysis capabilities within Analytics Workspace’s Cohort Tables, they haven’t released these in the form of segments and success events yet.

My method to define user retention metrics is still valid and in this post I’ve explained an easy way to do it.

It still requires a bit of work, but it is still worth the effort if you are looking to use these segments and metrics as a basis for more advanced product analytics, which I am going to talk about in upcoming posts.

Please bear in mind that the retention cycle I’ve focused on is monthly. Hence, these metrics work fine if you break them down by month. And while you can focus on longer intervals of time, such as quarters and years, as well as shorter ones, such as weeks, days, hours, etc, you won’t be able to do that straight away. If you want to do so, then you need to redefine them by using longer or shorter intervals of time in your data range definitions. For example, if you are interested in weekly retention rather than monthly retention, your date ranges will be 1 week ago, 2 weeks ago, 3 weeks ago, etc., rather than 1 month ago, 2 months ago, 3 month ago and so on and so forth. At the same time, you would be focusing on WUV (Weekly Unique Visitors) rather MUV (Monthly Unique Visitors).

Please let me know if this is all clear and/or if you have any questions or comments.