My Summary
I'll start with my conclusion.
Laura didn’t explicitly say this, but what I took from her talk was that you should try to put yourself inside the head of a single real user who isn’t you. Gather the evidence of what they do but then use your imagination to re-live their experience.
You can use your left brain and right brain to gain insights from data. There’s art in among the science.
My Notes
There are so many tools to use to gather and analyse data that many people are turned off and just don’t engage in the process of gathering data or analysing it with purpose.
How to take the first step
So how do we get started? Especially in the case of startup apps and new websites where we have very few users to start with.
Well, there’s data that we can use from zooming in on just one user. We don’t need to aggregate massive quantities of user data to gain useful insights.
Some accessible tools which we can use to do that are:
- Google Analytics for websites
- Google Firebase for mobile apps
- Fullstory for e-commerce websites
These tools now lean towards gathering ‘event’ data rather than page views or clicks. They are, therefore, better at helping you visualise how people behave on your website in greater detail than the broad strokes of which pages they visit and which links they click.
So the tools are there and are getting more useful and easier to use but what next?
Strategy
You need a plan of what to measure. To help decide what that should be you should plot a map of the person’s journey through your website or app. Then you can decide where to focus.
Check out Google's HEART framework as a good method to organise your thoughts.
For example on our Rubadub app project a goal/signal/metric set might be:
| Goal → | Signal → | Metric |
|---|---|---|
| Task success | Save a record on Rubadub app | Record is saved |
What we’re doing here is clarifying our goal first, stating what we think signifies when a person has achieved that goal and then defining the metric that we will track in our tool.
In this case, we’ll need to add some tracking code to the app to record an event when the person taps the ‘save’ button on a record in the Rubadub app. We’ll then be able to track that in Firebase.
This approach is good for gaining quantitative data (the what of people’s behaviour) but not necessarily the why, the qualitative data. For that, we need to talk to people which we can do with short surveys. Even if it’s just asking someone a basic in-context question like “is this page useful?”. If they say “no” then you can take it further and ask why.
Using the data
Once we’ve got metrics planned and data coming in to review, how can they be used?
Here are some go-to techniques.
Chart the data as a funnel to see where people are falling off or giving up
A funnel chart shows the number of people who completed each step in a journey. Here are a couple of examples that show the kind of data that reflect a system that's working normally and one which might have a problem.
This feeds back into the planning stage. If you set out the person’s journey and want to use a funnel analysis to see where people drop off then you need to make sure you’re tracking the events that make up that journey.
Use A/B testing to make predictions about proposed changes
In this case, once we are able to measure a certain metric then we can test a proposed feature or layout change on a fraction of the people using the website to be able to predict its impact if it were introduced across the whole site.
If you want to try an A/B test then it stands to reason that you need to be measuring what was happening before you introduce the variable journey. You can then continue to measure the control group.
A/B testing doesn’t have to be run across massive levels of data. Use the sample size calculator to get a handle on whether or not the stats you would generate are meaningful. And pick small but realistic factors to test against a control.
How do you really use these new-found techniques?
So how do you get everyone on the project to buy into the process of gathering metrics and trying to influence them? Try to make it fun. You can get people who are invested in a website to bet on the changes in metrics to try to make it a shared experience and a shared responsibility.
You can use dashboards to highlight user data but be careful to not create meaningless screens full of widgets. Decide as a team what is important and build dashboards that centre on the metrics that matter to the people using the website. Use this as a chance to reinforce the team’s commitment to caring about the people who use the website or app.
But remember, no-one likes a creep
The most important factor to keep in mind when working with data is to be very careful about how you collect it. And don’t be scared to delete data that you don’t need.
