Quantitative UX research & Qualitative UX research — do you know the difference?

7 min readNov 28, 2020

All methods used in user research can be classified as either qualitative research or quantitative research. Truth be told, this distinction isn’t binary — some methods find themselves somewhere in the middle of the qualitative-quantitative spectrum, as they bear some aspects of both research paradigms. (This includes session replay!) It is sometimes quite difficult to determine where a research method belongs, much less determine which one would be better to use in a particular situation.

What is qualitative research?

In a world where everything revolves around numbers, qualitative research might seem a little odd. Nevertheless, qualitative methods of research explore things in depths that numbers and statistics could never hope to reach. To use qualitative methods means that you’re looking for the whole truth about the researched subject. This whole truth could include things like details about the respondent’s actions, the reasons behind them, and their thoughts and opinions. These can be learned through interviews and open-answer questions.

There are many types of people, many variations of possible situations, and nothing exists in a vacuum. Even though numbers and statistics barely scrape against the complexity of the real world, qualitative research allows us to collect data that includes all of its intricacies. When we analyze things like user behaviour, reasoning, or past experiences, we can form a big picture and discover implications that would stay hidden if we only relied on quantitative data.

For example, while statistics can show that 99% percent of people leave your landing page without registration, an interview in a usability study will tell you that it’s because users accustomed to registration pages looking differently on other web sites, and the way your navigation works confuses them.

What is qualitative research good for?

The answer is, in layman’s terms, in any case or situation where you wish to understand things. Do you want to know what users think when they first get their hands on your product? What they would do with a new feature in your app? What’s the reason behind their unexpected behaviour? In a qualitative study, you have space and freedom to go in-depth and grill the respondents for all the details of their experience.

For example, you could run a qualitative study to find out what’s stopping users from purchasing things on your e-commerce website. Some typical qualitative questions that you could ask during the interview include:

Why haven’t you bought anything?
Were you expecting something on the website to work differently?
What does your usual experience of buying goods online look like?

Qualitative research methods

Qualitative data can be collected in a variety of ways. Qualitative methods usually involve giving the respondents a lot of space for expressing themselves while we record all they do and say using transcripts, audio and video recording, or even specialized tools such as session replay.

Here are a few typical qualitative methods, including quick summaries:

User testing — We ask the respondent to complete a number of tasks so we can analyze how they did, and whether there are any usability problems that need fixing.
User interview — Talk with the user. Usually, there’s an agenda of topics that need to be discussed (e.g., the user’s views, thoughts, and experiences regarding specific subjects), but some questions can and should be more open-ended.
Contextual Inquiry — The users are first asked a list of standard questions, and then the researcher observes them while they’re using the tested system in their natural environment. Thanks to this method, the analyzed data is more reliable than laboratory data, although the negative is that the method can only be used if there’s a system already in place.
Focus Group — Often used for getting feedback about products, concepts, prototypes, etc. A moderator leads a group of seven to ten respondents in a directed discussion that uses group interaction and nurtures different points of view.

Note that some of these methods can also be combined (e.g., we have respondents complete a task in a usability test, but we also try to learn more information about their experience by conducting a user interview).

Qualitative research and the number of respondents

In usability testing, the recommended number of respondents is six. Anything beyond that does not add very much in terms of discovering new insights about the experiences the users have when interacting with the tested system. Focus groups usually involve a group of seven to ten respondents. Qualitative research makes use of a lower number of respondents in general, ranging between five and ten respondents being representative of most qualitative studies.

To those who are used to surveys and calculating the statistical significance of findings in quantitative research, this might seem like too little. However, the low respondent count is by design and well justified. In the first place, the purpose of qualitative research isn’t to impress anyone with statistical data; it’s to generate insights into the researched topic. Secondly, this is done to maximize the return on investment of each respondent. Large scale studies are more difficult to plan, and, with each session, the study takes longer to run, and all the data becomes difficult to process, with relatively little additional information gained. Running smaller, iterative qualitative studies are more agile and therefore, the preferred option.

What is quantitative research?

The foil to qualitative research, quantitative research, replaces in-depth dives into the respondent’s experiences with data being expressed by precise numbers. Quantitative methods gauge data in the form of metrics, measuring phenomena such as the number of successful transactions and customer satisfaction. Quantitative analysis uses statistics to support the reliability of its findings.

Having data expressed by numbers gives us a plethora of options for working with them. Statistical methods allow you to look for trends in customer behavior. Numbers offer themselves up to being compared — between versions of the product in A/B testing, between various segments of the market, when looking for the best month for releasing a new product, or when looking for which new feature added would have the most impact.

For example, a quantitative analysis would be measuring the bounce rate on the landing page of your e-commerce website. You could use this data to determine how successful the redesign of the landing page was in leading respondents to the next page towards purchase.

What is quantitative research good for?

Quantitative research is at its most powerful when used to investigate an existing user base and their behaviour. By collecting quantitative data as implicit feedback from use, we gain realistic insights while minimizing the extent to which we disturb user flow. Of course, you might also want to do a survey before you start development to gain insights in advance. One benefit of quantitative research is that it’s often less expensive to do than qualitative studies, regarding both monetary and time investment (small or no incentives at all, no need to moderate sessions, etc.).

A quantitative research expert can be expected to answer questions like these:

What percentage of users don’t buy a product after placing it into the shopping cart?
At what time of day does our e-commerce see most completed orders?
What percentage of people add products to the shopping cart from search results, and what percentage opened product detail first?

Quantitative research methods

Numbers and statistics can be collected by different methods, depending on the type of information we want to learn.

Here are a few popular ones:

Card sorting — Ask respondents to place cards that represent various content into categories. Data provides insight into the conceptual models that form in users’ heads, which is useful when looking for ways to structure information instinctively.
Tree testing — The reverse of card sorting, here, respondents look for information in a tree that represents the structure of information on a website. Useful for validating information architecture models.
First click analysis — With just a simple prototype, give the respondents a task and observe which element they would click first. The first click often determines whether the user completes the task successfully or not.
A/B testing — Better than just testing one version of a design, test two variants, and find out which one performed better according to scores (success, completion time).
Clickstream analysis — A clickstream is the path of pages that the visitor passes through while traversing a website. Clickstream analysis aggregates this data to extract knowledge about the individual pages and their order.

Quantitative research and the number of respondents

As the name implies, quantitative research demands a high quantity of research subjects to be really useful. Optimally, you would want to base your actions on the data collected from every relevant person in the world. That’s obviously not possible, but that’s what statistics are for. The more data you have, the higher its statistical reliability, and the better facts you can learn from it.

Different methods have a different optimal number of respondents. Web analytics tools such as RePlay Visitors can collect data about thousands of real users, while for a card sorting study, the recommended number of respondents is at least thirty.

Qualitative, Quantitative and Hybrid

Sure, sometimes the research we conduct can be purely qualitative or qualitative in nature. However, as we’ve mentioned, this distinction is often only artificial, to help us better understand various aspects of quantitative and qualitative methods and how to make better use of them. While some methods contain aspects of both paradigms in their very nature, some others can be combined together to enrich the insights we can learn in our research.

As an example of a hybrid research method, let’s take a survey, which can contain a number of yes/no and multiple-choice questions, but also allows respondents to express their opinions in free writing at the end. An open card sorting study gives us numbers about the similarity of the cards, while also allowing respondents to name categories by themselves (and even allowing them to comment on their own way of thinking, if they wish to do so).

As for the combining of methods, let’s return to the previously used example of an e-commerce website. Web analytics can tell us how many people leave the website right away from the landing page, while user testing can help us discover why exactly that is. One piece of information is actionable and tells us how to improve the website, while the other one tells us exactly how much of a problem it is (so we can predict the return of investment for the redesign).

If you like to use some tools for both methods, have a look to our comprehensive platform for UX research, Uxtweak.