UX Research Analysis

Master the art of UX research analysis by learning effective methods for analyzing qualitative and quantitative data
UX Research Analysis Lesson

It doesn't matter what method you use to conduct user research, analysis should be the next step. You can't just take raw data and apply it to your product. Use analysis for sorting out the wheat from the chaff — i.e., defining what data is relevant and what isn't.

Regardless of the data type you analyze — web analytics, interview transcripts, field notes, or heatmaps — your focus should be on your research objectives. You should keep in mind your product's target audience.

Keeping this information at hand will help you stay focused during the analysis stage and define what's actually important for your project.

Why analyzing data is important

Performing UX research for the sake of UX research is never a good idea. The main goal is to gain valuable findings, sort them out, analyze them, and, finally, generate insights that can be applied to the development process.

One of the biggest mistakes UX practitioners may make is jumping to hasty conclusions based on raw, statistical data and acting without proper consideration. Data analysis allows seeing the bigger picture, making more meaningful decisions, and thus, saving money and time. Instead of fixating on random numbers or users' quotes, UX researchers should seek an answer to why users behave or think in a certain way.[1]

For example, if your user research reveals that 80% of users tend to ignore your Subscribe button, try to find out why this happens. Maybe, instead of redesigning the button, you might have to change its placement or reconsider the benefits you offer users through your subscription.

Thorough data analysis helps teams clearly understand users’ pain points and make more rational decisions.

Attitudinal data

The analysis of user research data produces a lot of qualitative data. Attitudinal qualitative data represents users' thoughts, beliefs, feelings, or assumptions based on what users say. This data type usually includes quotes and anecdotes collected during user interviews, focus groups, card sorting, or diary studies.[2]

One of the disadvantages of attitudinal data is that users don't always do what they say they're going to do. This means attitudinal data can be confusing and lead to wrong assumptions at times.

Behavioral data Bad Practice
Behavioral data Best Practice

Behavioral data is considered the most valuable data gathered during user research. It demonstrates user behavior during contextual inquiries, observational studies, and other ethnographic approaches.

Unfortunately, UX practitioners don't always have the opportunity to interview participants to understand why they behave in a certain way and what makes them choose one user path over another.

Think about analysis early

Ideally, you should start thinking about the analysis phase early in the process — before the research has even started. Remember that your research goals and hypotheses allow you to stay on track during your main analysis and find relevant answers.

Imagine your research goal is discovering why your customers abandon a shopping cart. In this scenario, you’d likely have certain assumptions about the rationale behind this behavior and would look for certain variables when analyzing research data.

At this stage, you can also create tags or codes that will help you categorize and identify data according to your research goals at the final stages. For example, if you're testing the usability of a landing page, you may create codes like "Navigation," "Aesthetics," "Critical errors," "Recommendations," and other pain points that you might discover during the study.

Analysis in the discovery phase

Analysis is necessary even in the discovery phase of your research. UX practitioners are humans too, and some things can slip away from their memory if not logged down. Taking notes, reviewing records, and jotting down first impressions while they're still fresh in memory help researchers remember and retain critical thoughts or ideas.

Write down the words participants choose, their facial expressions, body language, and overall behavior — everything is important to get a deep understanding of their rationales, feelings, and needs.

UX researchers usually conduct more than one session, so analyzing data immediately after each session prevents it from blending together. Team discussions after each session can help look at the process from different angles and spot inconsistencies and questions that don't work.

Analyzing data in the discovery phase can save you a ton of time and resources before you work on the final analysis. In the long run, it helps you identify users' pain points and needs and create better products.[3]

Pro Tip! Make sure you have a 15-minute break between sessions to review your notes, discuss with your team, and write a summary.

Setting priorities and objectives

Analyzing large amounts of data collected during the user study can be intimidating, and you might not know where to begin. Setting priorities and understanding your objectives makes it easier to focus on relevant data when analyzing results.[4]

Before you start, write down why you've decided to carry out UX research in the first place. For example, you're designing a yoga app and want to find out if it is enjoyed by users and fulfills their needs. Keeping this research goal in mind, you'll be able to prioritize findings and decide which ones are "nice-to-haves" and "must-haves."

For example, you may discover your users are interested in having meditation practices in addition to yoga. These findings are "nice-to-haves" and can be put aside in the non-urgent category. On the other hand, "must-haves" should have the highest priority when analyzing. For example, if users are unsatisfied with the quality of your yoga videos, it must be fixed first.

Pro Tip! Keep the "nice-to-have" findings in your backlog so you can get back to them when you have the time and resources.

Analyzing quantitative data

Once you've collected numerical data via surveys, polls, or web analytics, you will have a large dataset that can be analyzed with special software (SPSS, JMP, Stata, R, etc.) or simply with a spreadsheet.

Various methods can be applied to gain helpful insights from quantitative data:

  • Cross tabulation allows UX practitioners to analyze the relationship between multiple variables and determine patterns, trends, and probabilities within data sets.
  • Max-diff analysis allows measuring the preference/importance score of different items. It's often called the best-worst method as it helps identify what features matter most to your users.
  • Conjoint analysis helps identify the optimal combination of features in a product or service, ranking them on a scale from most to least desirable. It consists of factors (e.g., class of service or airline brand) and its levels (economy and business; Southwest Airlines, Delta Air Lines, American Airlines, and United Airlines).
  • Gap analysis demonstrates the difference between the desired state and the current state. For example, you can compare the actual and expected state of customer satisfaction.
  • Trend analysis shows the change of a value over time and what parameters influence this change.[5]
  • Sentiment text analysis helps researchers process users’ feedback from textual data using NLP (Natural Language Processing) tools.
Questions when analyzing quantitative data

The analysis of quantitative data implies working with numbers. Although it might appear a tedious and overwhelming task, quantitative UX analysis allows researchers to see patterns and tendencies and investigate how users interact with a product. Based on gathered insights, we can decide what can be improved to help people achieve their goals faster and more effectively.

When analyzing quantitative data, you can define things like:

  • The success rate of a specific task
  • Time users spend to complete a task
  • The bounce rate of a webpage
  • Users' demographic profile
  • Features that users use the most
  • User satisfaction with a feature or product
  • User needs that are not met by the product
  • Critical features that require the greatest attention
  • Different experiences of different user groups[6]
Analyzing qualitative data

Qualitative research data deals with human behavior, which might be harder to analyze than numerical quantitative research data. It may take a lot of time to read long transcripts and extensive field notes and decide what details matter and what can be skipped. Also, participants' feedback can be conflicting, and researchers should remain objective and try not to ignore viewpoints that don't fit their beliefs. One of the best methods of breaking down and organizing rich data is thematic analysis.

Thematic analysis groups the collected data into themes by tagging individual observations and quotations with appropriate codes (like labels or keywords). While coding, researchers review each text segment and give it a name that describes the data. As they look for themes, some codes can be collapsed or expanded.

The last step involves evaluating your themes — a belief, practice, need, or another phenomenon that appears multiple times across data findings and can be supported with multiple instances.[7]

Thematic analysis can be performed using software, journaling (which involves manual coding and essential thought processes by researchers), or affinity diagramming techniques.

Questions when analyzing qualitative data Bad Practice
Questions when analyzing qualitative data Best Practice

Qualitative research data analysis provides in-depth insights and answers to why users behave a certain way.

When analyzing data, keep in mind the following questions:

  • What do users like most about this product?
  • What do they like least about this product? Why?
  • Which functions are more valuable?
  • Which functions get unnoticed?
  • Do they have an emotional response to certain features? When?
  • Are they satisfied with the product? Why?
  • How does the product fit into their daily lives? How important is this product to them? Why?
  • What are the major patterns or common responses noticed in users' behavior?
Qualitative analysis mistakes to watch for

Even the most experienced UX researchers can make mistakes in analyzing qualitative data. Look out for the most common ones:

  • Having too much uncategorized data to stay objective. Ask your teammates for their opinions, prioritize user feedback, and decide what details can be omitted and what data is essential to be kept.
  • Having biases about research outcomes that can lead to misinterpretation of important data. To avoid biases affecting your results, write down all potential ones before the discovery phase and discuss them with your team.
  • Over-reduction, or "flattening," of the data gathered into close-ended survey responses like "yes or no" questions. Your focus should be on analyzing only the most meaningful and usable data. Also, make sure you include open-ended questions during user studies.[8]
Synthesize your findings

Once you're done analyzing qualitative and/or quantitative data, it's time to synthesize your findings and come up with insights. Synthesis helps researchers turn the identified themes into something meaningful.

Don't confuse findings with insights. A finding is a fact or a statement drawn from user research. For example, the statement that 70% of users abandon their shopping cart at the checkout page is a finding.

An insight is a conclusion about human behavior or user motivation based on this statement. An insight from this example can be that the checkout page is too confusing or that it contains certain distractions that prevent users from making a purchase.

How do you synthesize findings?

  • Prioritize findings and select the ones that seem the most relevant based on your research objectives.
  • Organize your themes, findings, and insights using sticky notes, whiteboards, or any other system you prefer.
  • List the most important insights in a document.
  • Share them with your team and brainstorm together what can be learned from these insights.
Contradictory results

Sometimes, different research methods can produce inconsistent or contradictory results. For example, the task success rate among users could be 100%, but they could still report that they're not satisfied with the app and would switch to another if they found a better alternative.

What can be done to prevent such conflicting findings? Start with checking the methodology:

  • Respondents: Have the same respondents participated in both studies? Different people can respond differently, which can lead to contradicting results.
  • Tasks: Have participants had the same amount of time for a task? Was the task the same for all participants?
  • Environment: Have users been performing a task in the same environment, or was there something that could have distracted them or influenced their responses?
  • Data analysis: Is the statistical significance high enough? Is there a chance that data was overcorrected?

If you find that none of your research methods have inaccuracies, you might have to consider conducting another study. Contradictory results are common, so don’t be discouraged.

Make recommendations

The final step of analysis implies providing actual recommendations based on the key insights and supporting data. Including recommendations in your research report represents a valuable takeaway for stakeholders and motivates them to act immediately to solve users' problems.

You may also share a document with the list of gained insights and have an open team discussion in person or remotely. Instead of providing off-the-shelf solutions, you can brainstorm together and turn your insights into "how might we" questions.

For example, the insight that "users abandon their shopping carts because they don't see a total amount unless they click the Pay button" can be turned into a design opportunity — "How might we help users review the total price including delivery costs and fees before they click the Pay button?”

Insights and recommendations help your team define what to focus on and can be potentially turned into design solutions.

Complete the lesson quiz and track your learning progress.
Start
<?xml version="1.0" encoding="utf-8"?>