User Interviews – Collecting Data and Uncovering Problems

Reading Time: 5 minutes

User Interviews – Collecting Data and Uncovering Problems

Introduction

Within Eye Studios, in the last period, we are focusing on the Research and Analysis part of the UX Design workflow, in order to standardize a process that can be scalable and replicable.

This document aims to bring together a lot of information that we are collecting and that we believe will be useful to anyone who wants to learn more about this topic.

It is not a finished document or a guide, but a collection of useful information that we are gathering and that we think can be shared.

This will be part of a series of documents, visit Eye Studios Linkedin profile to be updated on all available documents.

Understanding what data to collect

Quantitative data

“Quantitative data are used to collect the bare facts, the figures. It is statistical and structured data, which supports drawing general conclusions from research.”

en.surveymonkey.com ‘ quantitative-vs-qualitative-research

Qualitative data

“Qualitative data collect that information which attempts to describe a topic rather than measure it: it is about impressions, opinions and views.”

en.surveymonkey.com ‘ quantitative-vs-qualitative-research

	Quantitative Research	Qualitative Research
Questions	How many and how much?	Why?
Goals	evaluate the usability of an existing site – track usability over time – compare the site with competitors – calculate ROI	Making design decisions – identifying usability problems and finding solutions
When to use it	When you have a working product (at the beginning or end of a design cycle)	At any time: during the design phase or even when you have a working final product
Result	Significant data and statistics that can be used to conduct other analyses	Meaningful data and statistics that can be used to conduct other analysesDesigner interpretations based on impressions and prior knowledge
Methodology	Many participants – Defined and precise approach – Not usually “thinking aloud”.	Few participants – Variable approach depending on the needs of the team – Usually “thinking aloud”.

Questions to get to know the user

What does your typical working day look like?
Describe your typical day in [role-environment]?
When do you normally use the Internet for the first time on a typical day?
What are some of the apps and websites you use the most?
Tell me about your role in your company?
Any lifestyle questions related to your topic/product
Tell me about yourself and your relationship with [product name, app, site].
Tell me about your relationship with [project topic]?

Example scripts

inspired by Evan Bowers – Zapier

Hi {name}, how are you today?

Thank you so much for your willingness to help us with this, have you ever taken part in a usability study before?

I’m just going to give you an overview of what we’re going to do today …

I’m going to show you a prototype design where I’m going to have you complete some tasks and ask you some questions along the way. The first thing I want to make clear is that we are not testing you, we are testing the design, so there are no wrong answers you can give.

Our goal is to get your perspective. To get your perspective, I’m going to ask you to try and think out loud as much as you can – to say what you’re looking at, what you’re trying to do and what you’re thinking – it will help.

If you have any questions during the session, please ask – just know that I may not be able to answer them straight away, as we are interested in how people behave when they have no guidance.

One last thing before we start… do you mind if I record this session it will help my teammates who are not here and so that I can review it.

Questions for analysing user behaviour, impressions and expectations

From Objectives to Tests

To be able to prioritise questions and create new ones, you can start from the functionalities and objectives in the Product Backlog (or simply those defined by the team).

The following diagram is an example of how to go from an objective to a clear and defined test

What are the most important tasks you or other people have to perform when using [project website or application]?
How would you describe your past and current experience with [project topic, website or application]?
How often do you use or see [project website or application] being used?
How do you normally get to [project website or application]?
What devices do you usually use when you visit [project website or application]?
Have you used other websites and resources in the past for the same purpose as [project website or application]?
Is there something you often look for on [project website or application] that is missing or hard to find?
Is there a way in which [website or project application] does not currently support your needs?
If you have a question about [project topic] do you know who to contact?
What are you thinking while looking at this project?
And your (first) impression of this product / feature?
What do you think this product / feature does or will do?
Where do you start from?
When and where do you think someone would use this product / feature?
What do you expect to gain from using this product?
What would prevent you from using this product?
Do you think this product is similar to another one?
Do you trust this product?
Did you [“do this action”] when you [“saw/do something”], what caused this reaction?

Questions during and after use

How would you perform [task]?
What do you expect to happen if you performed this [task]?
What alternative method would you use to perform [task]?
Was there anything surprising or that didn’t work as expected?
Was the interface easy to understand?
What was the easiest task to do?
And the most difficult to complete?
What is the main function of [project topic]?
What is the main function of [project website or application]?
What do you like about the current [project website or application]?
What do you dislike about the current [project website or application]?
How long do you expect the [process in the project] to take?
Do you remember communication or any follow-up after performing the [process in the project]?
Under what circumstances would you like to be notified in [process in the project]?
How do you use the information on the [project website or application]?
Would you ever need to share these metrics with others?
If so, who, what format and method of sharing?
Would you ever need to export [information or resource in the project]?
If yes, when, why and in what format?
Why do you think someone would use this product?
How do you think this product will help you?
Would you use this product today?
What might prevent people from using this product?
What is the maximum amount you would be willing to pay for this product?
Does this remind you of other products?

Organising data during interviews

Notes and Templates

To take notes during the sessions we experimented with different templates used by large companies using Google Sheets (template at the bottom).

The note-taking sheet consists of:

Objective (the macro task the user has to perform)
Emotion (what the user feels)
Tester (which member of the UX team conducts the tests, in case a follow-up is needed)
Notes (individual notes from the tester).

➡️ Here is the Google sheets template which is used in a very similar way by companies like Zapier and Airbnb ⬅️

Follow-up and question-answer integration

Example

User response: “I thought the design was very simple.”

Follow-up question: “What made you think it was simple?”

User response: “This is not a feature I would use.”

Follow-up question: “What was the situation that made you not use it?”

User response: “I didn’t expect it to work this way.”

Follow-up question: “How do you expect it to work in that situation?”

User response: “I don’t like that colour of blue.”

Follow-up answers: “Why don’t you like that colour of blue?” or “What is that colour you don’t like in relation to the product?”

User response: “This is not something I would like to download and use.”

Follow-up response: “Can you tell me why this application is not useful to you?”

Top UX Research Interview Questions to Ask Users, Andrew Smyk, Adobe