How to Find your Most Engaging Features with Cohort Tables

Wanna know how to identify features that impress your first time users while getting onboarded and hence increase the odds they will become your active users?

Keep reading!

In Jan 2019, Adobe released a new Cohort Tables feature in Analysis Workspace whereby one can group visitors in cohorts using any dimension (eVars and sProps in Adobe language).

Next to the classic acquisition cohort (based on a given start time), this enhancement introduces a new type of cohort: the behavioral cohort; that is, a cohort that is not only defined based on a given start time but also on a given dimension of behavior (a user or event property). In fact, before this enhancement, you would be able to retrieve an acquisition cohort by its inclusion (start) day/week/month and then you could follow its trend over time. Because different acquisition cohorts start at a different time, you cannot really compare their retention curves against each other’s as they would not be aligned on the time dimension. Visually, this used to be the typical cohort table before the enhancement:

And if you wanted to plot its cohorts’ retention curves, you would end up with a chart like this:

With the new ability to group visitors by their actions, cohorts can now be defined over the same period of time; the action being what discriminates a cohort from the other ones. For instance, suppose you want to identify cohorts of visitors by browser type. In this case, the cohort table would look like this:

And here’s a visualization on top of it:

See the difference? You can now compare like for like, retention curves with retention curves, making it possible to see which cohort retains users best!

This feature opens up a new host of possibilities, including the ability to find your website’s most engaging features; that is, the features that make a good first impression during the onboarding and value discovery phase of a new user to your webiste, and which therefore increase the likelihood they will form a habit and continue to come back time after time (becoming an active user).

In this post, I am going to show you the few steps required to discover your most engaging features with the new cohort tables. Here they come:

Enable Success Event Pathing

Make sure success event pathing is enabled for your Adobe Analytics implementation by passign your website’s success events to a traffic variable (sProp). Your Adobe administrator can easily enable this via processing rules in the DTM. You can also check out an old post by Adam Greco to get you started with this topic. But, it should really be a piece of cake for your experienced admin (no extra JavaScript is required).

Define Cohorts by Success Event

Follow these simple steps:

1. Go to your Analysis Workspace and start a new project
2. Add a cohort table to the blank panel
3. In the Inclusion Criteria of the cohort table configuration pane, add a new segment with Visitor container to capture all new visitors (those for which Visits Number = 1)
4. Drag the Visits metric and drop it in the inclusion criteria; do it again for the retention criteria
5. Under Settings, select the Advanced checkbox
6. Select the Custom Dimension Cohort option
7. Drag and drop the traffic variable containing your success events (in my case, it is called KPI Pathing)
8. Select a time granularity, e.g. if your product is a monthly product, so you expect the value discovery phase to last a month, select Month.

It should all look like this:

Now hit the Build button and, voilà, Workspace will crunch the numbers and present you with your brand new cohort table (see picture above with caption Behavioral cohorts for an example).

Compare Cohorts’ Retention Curves

At this point, all you have to do is to add a line chart on top of the cohort table. Because cohorts are different in size, this chart will present all retention curves as worlds apart, each being of a different scale; like this:

In order to be able to compare apples with apples, we need to normalize; that is, configure the line chart by checking the “Normalization” checkbox. Now all retention curves start at 100% at the start time of the cohorts and you can see which features utilized during the onboarding drive the highest retention in the long run (the higher the curve, the higher the retention rate):

And here comes the gold mine: this chart tells us that those visitors that have made use of facet search and link out during their first month of usage are a lot more likely to come back the following month and become active as opposed to, e.g., those that have only used the search feature (most of which churned the month after).

Which means we now know which features we should be looking to optimize the user experience for and which aspects of our website we should promote more to impact our user retention and, hence, our user growth.