UX KPI - Measuring User Experience Design - Eye Studios

UX KPI – Measuring User Experience Design

Reading Time: 8 minutes

UX KPI – Measuring User Experience Design

The work does not end when the project is published.

Often there comes a time to ask a core question:

How do we know if the design will work?

Being able to measure UX Design is an essential step in creating a successful product.

How can you “measure” User Experience?

What data needs to be analysed and how to interpret it?

Good news: If these are the questions you that you have, you are finally in the right place.

To determine the success of a UX Design process, we need to know how and why people use products and evaluate the results with UX KPIs.

Why should we do this?

The simplest answer is: better decision making.

Nobody has an infinite budget for design, so it is important to know where a product needs improvement the most.

This is one of the major goals when measuring User Experience, to be able to understand which areas can improve business learning and decision making.

How to choose KPIs

  • Which metrics should be monitored?
  • What is the difference between KPIs and UX KPIs?
  • When to define them, How and Who should do it?

Let’s start with a fundamental assumption:

KPIs can tell you exactly WHAT users are doing, but they cannot tell you WHY they are doing it.

The Google HEART Framework

There are five categories to brainstorm on from the user’s point of view:

  • Happiness – Measuring users’ attitudes, often collected through surveys
  • Engagement – Level of involvement
  • Adoption – Acquisition of new users
  • Retention – Rate of Return of current users
  • Task Success – Efficiency, effectiveness and errors

Google recommends starting with one or two things that are really important for the product.

After choosing a category, you need to follow a three-step workflow: Objective> Signal> Metrics

For example: YouTube, Engagement

Objective
In this step you need to define the “big picture” of the metric – the most abstract level.

What does Engagement mean from the perspective of a YouTube user? The Engagement objective for YouTube is “that users enjoy the videos they watch and discover other videos”.

Signals
After defining the objectives we move on to the Signals.

These answer a very simple question: How do we know that we have achieved the goal? Is there anything to prove this?

The Signal for the Engagement objective, for example, is “the amount of time users spend watching videos”.

Metrics
Last step is to turn the signals into metrics that can be measured.

The metric in the YouTube example is “average number of minutes spent watching videos / user / day”.

The HEART framework is great for finding KPIs for any (or almost any) product.

But how can you turn the data into useful design insights ?

KPIs vs UX KPIs

UX KPIs are more focused on measuring the user experience related to the use of the product.

For example?

Here are some UX KPIs we’ll look at in a moment:

  • Usability
    • Task Success Rate
    • Time to Task
    • Retention
    • Abandonment Rate
  • Conversion Rate
  • CSAT – Customer Satisfaction Score
  • NPS – Net Promoter Score
  • SUS – System Usability Scale

Usability

Usability is the source from which we draw many UX KPIs.

What do we mean when we say that a product is “usable”?

According to ISO (International Organization for Standardization), usability is “can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience.”.

Why measure it?

Because when products do not meet user needs or make it difficult to achieve goals, users leave the product and go away.

The only way to know if we are really solving design problems, is to test.

1. Task Success Rate

Task Success Rate is a UX KPI based on usability, basic but very useful.

In order to measure the Task Success Rate we need to define when a task is defined as “done”, and then set a ratio where the denominator is the total number of attempts and the numerator is the number of successful attempts (i.e. tasks completed).

The result, after being multiplied by 100 (*100), shows the percentage of participants who have completed the task.

Some examples of measurable tasks:

  • Adding a product to the shopping cart on an ecommerce website
  • Completing the registration process on an app
  • Issuing an invoice on an accounting software

Remember that the percentage is a value, and as such shows an objective figure. It tells you how many users have completed the task, but not How they completed it or Why.

2. Time to Task

This metric measures the amount of time it takes for a user to complete a given task.

We can say that in general, the less time a user spends on a task, the better the UX will be.

Remember that there are many factors, and each user may have different velocity.

Depending on the evaluation method and the type of project, there are different approaches to determine the average time of the activity:

  • Completion: only consider users who actually complete the task 100%.
  • Before error: the average time taken by users to give up on the activity or to complete it incorrectly
  • Activity: The total average time users spend on an activity.

3. Retention

In general, the loyalty (or retention) rate is the percentage of users who continue to use a product over time.

It is very common and always on everyone’s lips.

But how exactly is retention measured as a UX KPI?

In order to measure the retention rate of a product, there must be a clear definition of which main actions represent the use of the product.

When can you say that a user “uses” your product?

The answer can include actions such as logging in, visiting a specific page on your site, downloading/uploading files, using a key feature, etc.

4. Conversion rate

It is often associated with a financial meaning, linked to sales.

Conversion rate as a UX KPI measures the percentage of users who perform a specific action.

That is, they go from “those who do not perform an action” to “those who perform an action”.

Actions can be linked to product goals or can be important steps in the process.

This UX KPI is useful if there is a specific action that can be influenced by an improved user experience.

For example, completing a form, subscribing to a newsletter, using a feature in your app etc.

As NN/g suggests:

“The conversion rate measures what happens once people are on your website. It is therefore strongly influenced by design and is a key parameter to monitor to assess whether your UX strategy is working. “

Example: an e-commerce website is visited by 100,000 people during the month of April. During that month, 2,000 users purchased something from the site. Therefore, the conversion rate of the site is 2,000 / 100,000 = 2%.

The main problem with the conversion rate on websites is that the number of traffic can be strongly influenced, and the users who visit the website can be very different.

You must always consider how qualified the traffic is that your marketing brings to the site/product.

If too many unqualified users arrive, the conversion rate drops, but it does not depend on the UX

5. Churn Rate

The churn rate is the percentage of users who have stopped using your product or service in a given period.

There is no universally ‘correct’ churn rate.

It depends on the sector in which your business operates.

If you’re a SaaS and you’re serving B2B customers, you’ll probably experience lower churn rates.

But if you offer consumer services such as entertainment services, your churn rate may be significantly higher.

UX attitudinal KPIs

Unlike the previous metrics, attitudinal UX KPIs return different data.

The main difference?

In previous metrics, the data was monitored. In these ones, they are collected.

CSAT (Customer Satisfaction Score)

The Customer Satisfaction Score (CSAT) is a UX KPI that measures the loyalty of your users and is used by companies to assess how satisfied a customer is with the overall experience.

The main problem with the CSAT, is that usually most of the users who fill it out are on the extreme end: they either love your product, or they hate it.

But if used correctly, it is very useful:

For example, if you are testing a new feature within your product, you can put in one of these simple tests as soon as the user has finished using it.

In our opinion, this test is much more useful when used in specific situations to analyse micro steps and new features, and not to evaluate the overall experience which might lead to unrealistic data.

NPS: Net Promoter Score

Net Promoter Score (NPS) is a survey that you can include at the end of guided UX tests.

In short, it helps you measure loyalty based on one question: how likely are you to recommend this company/product/service/experience to a friend or colleague?

The data is interpreted according to the scale:

  • Those who answer with a score of 9 or 10 are called ‘promoters‘. (loyal customers who recommend your services and products to other people and will continue to buy from you in the future).
  • Those who respond with a score of 7 or 8 are called “passive”. They are satisfied with your service but have no real loyalty to you, so they are likely to turn away.
  • Finally, there are the ‘detractors‘, the customers who answered with a score from 0 to 6. These are dissatisfied people who never want to see your product again.

The final NPS score is then calculated by subtracting the percentage of customers who are detractors from the percentage of customers who are promoters. Promoters – Detractors = NPS.

SUS: System Usability Scale

Probably the best known and most widely used questionnaire in UX Design and UX Research is the System Usability Scale (SUS).

The SUS is a post-test tool, which is provided to a participant at the end of a usability testing or comparison testing session.

Users fill in a short questionnaire and a score is derived from this. It is on a Likert scale, which helps to assign a quantitative value to qualitative opinions.

Participants will rank each question from 1 to 5 according to how much they agree with the statement they are reading. 5 means they completely agree, 1 means they completely disagree.

Here are the 10 model questions that you can adapt to suit your website:

  1. I think I would like to use this site frequently
  2. I found the site unnecessarily complex
  3. I found the site very easy to use
  4. I think I would need the support of a person already able to use the site
  5. I found the various features of the site well integrated
  6. I found inconsistencies between the various features of the site
  7. I think most people can learn to use the site easily
  8. I found the site very difficult to use
  9. I felt comfortable using the site
  10. I needed to learn a lot of processes before I could use the site to its full potential

The System Usability Scale is non-diagnostic and will not tell you which specific problems you need to address.

The average SUS score is 68.

If your score is below 68, there are probably serious problems with the usability of your website that you should address.

Here is an overview of how your scores should be measured:

  • 80.3 or higher is an A++. People love your product and will recommend it to their friends.
  • 68 or less is a pass. The product works, but you should improve
  • 51 or less indicates major usability problems. Make usability your priority now and fix it quickly.

Conclusions

UX KPIs are numerous, and these you have seen are the main ones in our opinion.

You can find many and varied ones, depending on your needs.

It also depends on your business goals and the results your stakeholders want to see.

The key is to be clear about what is being measured and why.

If you need support in making better design decisions or understanding where you are going wrong, contact us and we will be happy to help.

Let's talk about your idea

Let's talk
about your idea

Request

After sending your request we'll get in touch within 24h

Meeting

We schedule a meeting to discuss your idea

Start!

If our proposal fits your need, we’ll start working together on your project