Having feedback on your product and understanding the interaction of participants with it is essential for improvement. Good and effective questions will be the key to high-quality feedback. Learn how to ask proper questions and when to ask them.
This article will give you tips and advice about writing the most effective usability testing questions for your research. We will explain why it is important to ask good and effective questions and how to do it. Let’s dive in.
Why is it important to ask the right questions?
The role of usability testing is to find issues, technical difficulties, bugs, and errors. Those mistakes wouldn’t be exposed by testing because respondents will focus on the inaccuracy of questions and task instructions. Incorrect or insufficient wording can lead to inaccurate results of testing. The attention of respondents should be on the task, not the syntax errors of the task. Otherwise, it’s a loss of finances, time, and energy.
Right questions can lead to the most accurate answers which represent how your product is perceived by the audience. Good questions can identify issues or difficulties with the design but also how are people interacting with the design.
Let’s say that you added a button to your design that will make certain processes easier. Is the function of the button understood by the users? Is the button really making their life easier?
How to write a good usability testing question?
When writing questions for tests, one could have a feeling that writing simple sentences is really hard. We understand your concerns. Sometimes even UX professionals have a feeling that their native language is some foreign language. Asking the right usability testing questions is the art of language.
Questions should be:
- short and clear
- well defined
- explicit
- without complicated grammar or syntax
- no suggestive questions
How to avoid bias in your usability testing questions
When preparing a questionnaire, bias is your worst enemy. Poorly formulated questions will mess with the respondent’s minds and lead them to answer the opposite of what they actually think. Thus, skewing your research and overcomplicating the data analysis process.
We’ve gathered some helpful tips to follow in order to avoid bias in your next study.
What type of questions ask and when?
Let’s talk about a theory first. There are two kinds of questions that you can ask your respondents
- Open-ended questions: There is no suggestion of the answer, you can ask about reasons, feelings, and opinions (What is your opinion.., What do you think about..). You will receive rather qualitative data.
- Closed questions: scales, numbers, choosing one option (multiple choice questions A., B., C., on a scale from 1–5) and therefore, quantitative data.
We recommend planning out your questions and tasks in advance by creating a usability testing plan you can follow throughout the whole study. You can create one by choosing a usability testing template from this list. Enough of theory and back to real life. Testing has various phases and each phase requires a different set of questions. With UXtweak Website Testing, you can prepare questions that would be asked before and after the study and even right after the task. Here are 4 tips on how to do it and when.
1. Screeners
Screeners will help to select the most ideal testers based on one or very few questions. Why do you need to ask questions before the actual testing? The role of screeners is to set boundaries in your test.
What is your target audience? What kind of respondents do you need for the test? What characteristics should they have?
Sometimes you need to ask only one question to find out the right respondents, such as: are you using *insert name of the bank product*? The answer is simple: yes or no. On the other hand, some situations require a deeper selection of respondents and questions like: How old are you? What is your occupation? What is your highest level of education?
You can read other tips for the successful recruiting of the right respondents in our blog section.
PRO TIP: Think about those who should be included in the test, but also those, who should be excluded: such as people under 18.
Screening question example
Suppose you’re testing a future banking app for usability. The target audience of such a study would be people who use online banking and are familiar with some of the main functionalities of similar apps.
A great screener, in that case, would be:
How often do you use online banking?
- Never
- Once a month
- Once a week
- 2–3 times a week
- Daily
If in such a questionnaire a respondent chooses the “Never” option, they will not be allowed to proceed with the study.
2. Pre-test
The main aim of the pre-test is to understand the background of respondents. In the case of moderated testing, you have a chance to know your respondents and ask questions regarding their current mood or thoughts. This is your chance to get to know your testers.
The role of a pre-test is not to ask basic questions- quantitative data, but rather qualitative data. It’s important information that can influence the final result. The pre-test can be in a form of an interview (moderated testing) or a questionnaire (unmoderated testing).
You can ask questions regarding brand awareness or the way how respondents interact with the product: Do you know this brand? Have you ever brought anything from this brand? How often do you use internet banking? What kind of device are you using while using internet banking?
Pre-test questions example
Following the example with an internet banking app, after we filter out the respondents not representative of our target audience, it’s time to get to know them and their habits better.
Some of the pre-test questions, in that case, would be:
- What type of device are you using while using internet banking?
- What are some of the main features you’re using the app for?
- Have you ever had experience with this brand?
3. D-Day: Test day
Asking questions during testing is the domain of moderated testing. It means that the researcher is asking questions while respondents are completing the task. The main goal is to collect information about the intentions of respondents, and why they did a certain thing.
Testers are sitting in a room with the device working on the task. They are focused on the task so you have to think twice about questions. When conducting a moderated test, be aware of the wording. Try to use simple questions without suggestions, that would lead to a certain action.
Don’t ask questions that would influence the behavior of respondents. You can learn more about how to write effective tasks for your usability testing here.
Examples of questions to ask during the test
During the study you can see the behavior of respondents - they are clicking on buttons, their body language, and facial expressions, but what is happening in their minds?
Questions will support respondents to verbalize their mental processes, to be authentical, and express their inner monologue.
This can be supported by questions such as:
- What do you think about it?
- What is your opinion about this?
You can also ask questions about personal opinions such as:
- What did you think about the design of the product?
- What did you think of that feature?
- I see that you clicked on this item. Could you tell me why you did that?
4. Post-study
Tasks are finished, and you have data from the tests in your hands. However, there is still some room for additional questions. If there is a possibility to reach respondents after the test, don’t hesitate and ask them some questions. This is a great way to find out what is the authentic impression directly of the task and the design.
Give them enough time and space for expressing their thoughts, positives, or limitations they experienced with your product.
Post-study questions example
If you conduct a moderated usability testing study, after the test, while you’re still connected with the respondent, you can ask their opinions and thoughts on the product and tasks. In the case of the unmoderated study, you’ll have to create a questionnaire and send it to the participants to fill out after the study.
Some of the great post-study questions to ask are:
- What is your opinion about the product?
- Was the task easy?
- Did you feel confused at any point in the study?
- Would you recommend the product to your friends?
You get what you ask for
Asking good questions leads to high-quality answers. It’s easy. You can gain valuable insight into the design and overall impressions, and opinions about your product with effective questions. Did your expectations meet reality? Or in other words, is this cool gadget that you added to the design that cool for users? Ask a good question and you will find out.
