A Guide to Conducting Effective Usability Testing

Usability testing is a crucial process for any product or service that aims to provide an optimal user experience. It involves testing a product with real users to identify any issues or areas for improvement. By conducting usability testing, companies can uncover user pain points, improve the user experience, and ultimately drive better business results. In fact, research has shown that every dollar invested in usability testing can result in up to $100 in increased sales.

Usability testing is not only important for creating great user experiences, but it can also have a significant impact on a company's bottom line. Let’s explore the ins and outs of usability testing, including the best practices to follow, common methods to use, and the most effective ways to analyze and report your findings.

What’s usability testing

Usability testing is the process of evaluating how easy it is to use a product, such as a website or an app, by watching real people operate it and gathering their feedback. The goal is to identify any problems or areas where the user experience can be improved. During a usability testing session, participants are typically asked to perform a set of tasks on the product being tested. At the same time, the researcher (a moderator of the study) observes and takes notes. The tasks usually represent the typical actions users would take while using the product.

Usability testing can be conducted in various ways, such as in-person, remote, moderated, or unmoderated, depending on the goals and constraints of the project. The cost of usability testing can vary depending on the size and complexity of the product and the testing method. In-person testing can be expensive, but remote testing methods can be more cost-effective. The cost should always be weighed against the potential benefits of improving the user experience of the product.

While recruiting employees as test participants is not a replacement for external recruiting, they can be involved in conducting pilot testing sessions or if your application is developed for internal use and team members are the direct representatives of your target audience.

Why conduct usability testing

Even the most experienced designers with years of expertise can never be completely confident that they know exactly what users need, think, or how they behave. Usability testing plays a crucial role in addressing this uncertainty.

In addition to this, usability testing is conducted for several reasons:

• To identify usability issues: Usability testing helps identify any problems or challenges that users may face when using a product, such as confusing navigation or hard-to-read text.
• To ensure user satisfaction: By testing a product with real users, the product team can ensure that it meets the needs and expectations of the target audience. This can lead to increased user satisfaction and loyalty.
• To increase adoption and revenue: A user-friendly product is more likely to be adopted by users, which can lead to increased revenue and success for the product.
• To inform design decisions: The feedback and insights gained from usability testing can inform design decisions and help ensure the product is designed with users in mind.

While usability testing has many benefits, it is important to consider whether it is necessary and appropriate for the specific product and stage of development. Usability testing should not be conducted solely to validate preconceived ideas or assumptions.

When to conduct usability testing

Usability testing is something you'll want to consider doing throughout the development of your product, from the initial design stages to launch and beyond. Here are some key moments where usability testing can be especially helpful:

• Early in the design phase: This will help ensure that the product is on the right track and uncover any major usability issues.
• During prototyping: Testing early prototypes can help you identify usability issues before you invest too much time and resources into development.
• During development: Testing throughout development can help you catch problems early and make sure your product is meeting user needs.
• Just before launch: Final testing before launch can help you identify any last-minute issues or opportunities for improvement.
• After launch: Usability testing is an ongoing process even after launch, as feedback from real users can help you make updates and improvements to the product over time.

Sometimes, you won't need to do usability testing, especially if your product is a straightforward design that doesn't require much input from users. In some cases, it might not be productive to conduct usability testing during the very early stages of development when your product isn't fully functional yet.

Plus, if you're working with a tight budget or timeline, you might need to focus on other aspects of development before diving into usability testing. It's all about finding the right balance for your specific situation.

Facilitator

In usability testing, the facilitator is like the captain of the ship. Their job is to make sure everything runs smoothly during the testing session. A facilitator's responsibilities include:

• Setting up the testing environment
• Recruiting participants
• Explaining the tasks to participants
• Collecting data on the participants' performance
• Debriefing them at the end
• Analyzing the results to identify any issues or opportunities for improvement

Who can become a facilitator? Usually, it's someone who works in user experience, design, or product management and has a good understanding of the product and the target user group. They should have experience in conducting usability testing studies and be able to lead discussions and moderate the session.

Sometimes, companies might bring in a professional facilitator to make sure the testing is done properly. However, in smaller organizations or startups, someone on the team might take on the role of facilitator if they have the necessary skills and experience.

To keep your testing process unbiased and fair, have a facilitator who is not part of the design or development team.

Tasks

Tasks in usability testing are specific actions or scenarios that participants perform while interacting with the product being tested. These tasks aim to simulate real-world situations that users may encounter when using the product, and they typically focus on the product’s key features and functions.

Tasks can vary depending on the product being tested and the goals of the testing session. For example, in a usability test of a mobile app, tasks may include signing up for an account, searching for a product, or making a purchase. A website usability test may involve finding specific information, navigating to different pages, or completing a form.

Ensure that tasks are realistic, considering participants' experience, skill level, and familiarity with the product. For instance, if you are testing an accommodation rental website with a college-aged participant, don't ask them to book a hotel for a business trip.

Write clear and concise tasks

To develop effective tasks for usability testing, several considerations are essential:

• Start by defining the most important goals that users want to accomplish using your site or app. This can help you identify areas where users may encounter issues, struggle, or experience friction, providing valuable insights into how you can improve the user experience.
• Add context to engage users with the interface. For example, instead of asking participants to "search for a product," provide a scenario where they are looking for a specific product type for a particular use case.
• Keep tasks clear and concise. Participants should understand what they need to do without confusion or ambiguity. Avoid using technical jargon or overly complex language that could make tasks challenging to understand.
• Include a mix of tasks covering different features and functions of the product. This can help identify areas where users may struggle or encounter issues.
• Test both common and uncommon scenarios. While it's essential to test tasks that are likely to be performed frequently by users, less common testing tasks are also important. This can help identify issues that may only occur in specific scenarios or edge cases.

Instead of giving users step-by-step instructions on how to perform an action (like go there and click here), it's better to ask them to achieve a final goal and observe how they go about it on their own.

Participants

Usability testing participants are people who are asked to test a product or system for its ease of use. They can be anyone who is the target audience for the product, such as potential customers or users. However, participants who have prior experience with the product being tested or knowledge of its design and development may not be suitable as they may exhibit biased opinions or behaviors.

Keep in mind that it's crucial to follow these ethical principles while conducting usability testing to ensure the safety and well-being of participants:

• Informed consent: Before taking part in the study, participants must be fully informed about the study's nature, risks, and benefits. They must also give their informed consent.
• Voluntary participation: Participants shouldn't be forced or pressured to participate in the study. They must be free to withdraw at any time without any penalty.
• Confidentiality: Personal information of participants should be kept confidential and not be shared with anyone outside the research team.
• Debriefing: After the study, participants must be told about the study's purpose and allowed to ask questions.
• Respect for participants: Participants should be treated respectfully and with dignity throughout the study, and their beliefs and values should be taken into account.

Usability experts recommend testing with at least 5 participants to identify the most common issues with the product.

Thinking aloud technique

The thinking aloud technique is a popular way to learn about what people are thinking when they use a product. In this method, a facilitator asks a participant to explain their thoughts and actions as they complete tasks. The facilitator may ask questions like, "What are you thinking right now?" or "What are you trying to accomplish?"

Advantages of using the thinking aloud technique include:

• It helps designers and researchers understand how users interact with the product, which can inform design decisions and improve the overall user experience.
• It provides insights into what users may find difficult or confusing when using a product.
• It is a low-cost and relatively easy way to get user feedback, so it can be used by small teams or those with limited resources.
• It encourages participants to be more reflective and aware of their own actions and thought processes.

However, the technique isn’t without limitations. Participants' comfort levels with speaking aloud while being observed may influence the results and reduce their validity. Skilled facilitators are required to properly administer and interpret the results in a way that avoids influencing participants’ behaviors or thought processes.

It’s important to remain neutral and avoid leading participants or providing them with hints, as this could influence their behavior and impact the validity of the results.

Remote usability testing

Remote usability testing is a way of testing a product or system with participants who are in a different location than the facilitator. It allows companies to test products over the internet. The benefits of remote testing include reaching a wider audience, saving time and money on travel expenses, and testing in real-world environments where participants would normally use the product. It's also convenient for participants, who can take part from their homes.

Different methods of remote testing include video conferencing, screen-sharing software, and tools like Zoom, UserTesting, Lookback, Maze, UserZoom, and Optimal Workshop. It’s important to choose the right method based on your specific testing needs.

Before starting any remote usability testing session, the facilitator should ensure that the technology is working correctly, including verifying the audio and video quality, confirming the screen-sharing tool is functioning, and making sure all necessary software is installed.

Talking during usability testing

During usability testing, it's crucial to ensure that participants feel comfortable and provide honest feedback. Sometimes, participants may feel pressured to please the facilitator and answer questions in a way they think the facilitator wants to hear.

To elicit honest feedback, use encouraging phrases that allow participants to speak their minds, for example:

• "Please feel free to speak your thoughts out loud as you go through the tasks."
• "There are no right or wrong answers — we just want to understand how you interact with the product."
• "We appreciate your time and feedback — it will help us improve the product for future users."

Facilitators should avoid using leading questions that can bias the results, such as:

• "Did you find that task easy?"
• "Was it frustrating when the system didn't work?"
• "Did you notice any design flaws during the task?"

These types of questions can suggest a particular response and influence participants to provide feedback that is not truly reflective of their experience. Instead of leading questions, facilitators can use methods such as the boomerang technique, which involves asking non-threatening, open-ended questions like, "What would you normally do in this situation?" or "Where do you think this button might lead?" This encourages participants to share their genuine thoughts and experiences.

Sometimes, when users think aloud, they may ask rhetorical questions that don't require an answer. If you're unsure whether to respond, silently count to 10 before talking.

Analyzing results

Analyzing usability testing findings can be complex and overwhelming. To stay organized and focused on the most critical insights, follow these steps:

1. Organize your data: Create a structured way to keep track of all identified usability issues. Use a spreadsheet or database to log each issue, including its severity, the number of participants affected, and any relevant comments or observations.
2. Categorize the issues: Group similar usability issues based on the area of the product they affect, such as navigation, layout, or content. This helps identify patterns or trends.
3. Prioritize the issues: Prioritize usability issues based on their severity and impact on the overall user experience. Focus first on the issues that have the greatest potential to improve the product.
4. Create an action plan: Develop an action plan to address each issue. Assign tasks to team members, allocate resources, and set deadlines. Regularly track progress and make adjustments as needed.

Visualize your data by creating charts or graphs. This can make it easier to communicate your findings to team members and stakeholders.

Ethical behavior during usability testing

Obtaining informed consent from each participant is a critical component of usability testing. It ensures that participants fully understand the purpose of the study and their role in it. Consent is especially important if participants are asked to share personal information or complete sensitive tasks. Participants should feel secure knowing they can withdraw from the study at any time without penalty.

Additionally, consent fosters trust between researchers and participants. When participants feel respected and treated fairly, they are more likely to provide valuable and honest feedback.

If the session will be recorded, inform participants during recruitment and again before the session starts so they are aware of how the recordings will be used.

Always remind participants at the start of the session if it is being recorded, even if they’ve already provided consent earlier.

Qualitative vs. quantitative usability testing

Qualitative and quantitative usability testing are two different methods of gathering data about user experience:

• Quantitative testing focuses on collecting numerical data, such as the number of clicks, time on task, and completion rates. This type of testing typically requires a larger sample size to ensure that the data is statistically significant.
• Qualitative testing focuses on collecting descriptive data, such as user feedback, opinions, and observations. It is typically conducted with a smaller sample size, as it provides more in-depth insights into the user experience.

To decide between qualitative and quantitative usability testing, consider the following:

• Research goals: If you want to measure task efficiency or success rates, quantitative testing may be more appropriate. If you’re looking to understand the user experience or gather feedback, qualitative testing is more suitable.
• Design stage: During the early stages of design, qualitative testing may be more beneficial to gather insights and feedback. Quantitative testing is useful later on to validate designs and measure performance.
• Resources: Quantitative testing typically requires more participants, time, and resources, while qualitative testing can be done with a smaller group of participants and fewer resources.

In some cases, combining both methods may provide a more thorough understanding of the user experience.

Remember that usability testing is an iterative process, and continuous improvement based on user feedback is key to developing successful products in today's competitive digital landscape. With practice and dedication, you'll be well-equipped to uncover critical insights that can drive meaningful improvements in your product's user experience.