Tree Testing

Card sorting and tree testing are crucial research methods in information architecture, each with a distinct purpose.

• Card sorting: It involves giving users a list of content items to group and label. This helps understand users' natural organization and naming conventions, capturing their mental models. It's great for discovering possible content groupings but not ideal for evaluating navigation hierarchies.
• Tree testing: It evaluates a navigation hierarchy by asking users to find specific items within a predefined category tree. This method requires a complete category structure beforehand and reveals whether users can effectively locate key resources, highlighting navigational issues. For example, card sorting might show that users think "FAQs" and "Support" belong together, but tree testing will indicate if users can find the "Contact Us" page within that structure.

The best times for conducting tree testing are at several key stages in the product development process to maximize its effectiveness:

• At the beginning of a redesign project: Tree testing at this stage provides clear insights into the existing structure’s performance, helping you identify problem areas. Unlike traditional usability tests, tree testing focuses solely on the IA and helps demonstrate return on investment (ROI) by showing improvements in navigation.
• After a card-sorting study: Card sorting generates ideas about how users group content but often results in multiple possible IA structures. Tree testing helps validate and refine these structures, ensuring the chosen hierarchy works well for users.
• Before generating content or layouts: Tree testing allows you to test different IA options without investing in design, coding, or content development.

Tree testing can be both qualitative and quantitative, depending on whether you're looking for formative or summative evaluation. Qualitative tree testing focuses on early-stage feedback and iterative improvement. It's ideal at the start of a project when you need to refine your information architecture quickly without designing full layouts or content.

Like usability testing, qualitative tree testing involves a small number of participants and often uses moderated sessions. This allows facilitators to ask probing questions and understand participants' thought processes. It's excellent for gaining insights on how to improve navigation but isn't suited for gathering statistical data like success rates or time on task.

Quantitative tree testing is used for benchmarking and comparison. This method requires a larger sample size and provides precise measurements, such as how long it takes users to find a specific resource and the success rate of users navigating through the IA. Quantitative tree testing doesn't delve deeply into the reasons behind users' actions but offers valuable data for comparing different navigation structures. It's particularly useful when you have multiple IA options or are benchmarking against competitors or future redesigns.

We recommend beginning a quantitative tree test with a small qualitative study for two reasons:

• Piloting your study design: Ensure that your tasks are clear and do not bias participants. It'll help identify any issues in your test design that could affect the reliability of your quantitative results.
• Gaining deeper insights: Obtain insights about why users chose the answers they did, which category labels are confusing, and ideas for how to address these issues. Understanding the "why" behind user choices can help refine your IA before conducting the larger quantitative test.

The first step in the tree testing process is defining the tree. This means creating a comprehensive list of all the categories and subcategories in your global navigation, and possibly in your utility navigation as well.

Start by listing the top-level navigation categories, but don't stop there. You'll need to break down each category into its subcategories, going all the way to the lowest level where the actual resources or content items are located. For example, if you're testing the navigation for an online bookstore, your top-level categories might be "Books," "E-Books," and "Audiobooks." Under "Books," you might have subcategories like "Fiction," "Non-Fiction," and "Children's Books." Within "Fiction," you could further break it down into "Mystery," "Romance," "Science Fiction," and so on.

This detailed approach helps you identify any issues or confusion at every level of your navigation, providing valuable insights for improving your information architecture.

Selecting the right tool for tree testing is essential for efficient and effective results. While you could use a paper prototype or any clickable prototyping tool, specialized tree testing services can significantly speed up data analysis and provide more accurate insights.

• Professional tree testing tools: You can consider services like UserZoom and Treejack. These platforms are designed specifically for tree testing and offer robust features for tracking user paths, measuring success rates, and analyzing data quickly. They are user-friendly and can handle complex navigation structures, making them ideal for thorough tree testing.
• Free tools: If budget is a concern, you can use free tools like Google Forms to create a basic tree test. While it may not have all the advanced features of dedicated services, Google Forms can still help you gather useful data on how users navigate through your proposed structure.^[1]

The tasks you set in a tree testing session are key to gathering useful insights. Here are some common types of tasks:

• Warmup task: Begin with an easy task to get users comfortable with the test. This also helps screen out participants who aren't paying attention. If they fail this simple task, they might not be fully engaged.
• Resource-finding tasks: Ask users to locate important items like products, services, or crucial information, such as finding the return policy or contact details.
• Sleight-of-hand tasks: Have users find less important items to test how well they can navigate parent categories.
• Potential problem areas: Focus on new or disputed categories from card-sorting studies to identify where users might struggle. This helps highlight problematic areas in your navigation.
• Label or location comparisons: Test different names or locations for the same category to see which is more intuitive. For example, compare "Support" versus "Help Center" to see which is easier for users to find.

For each task, specify the correct answer within the tree. This helps testing tools automatically calculate success rates, showing where users navigate well and where they get lost.

Mistakes in tasks during a tree testing session can significantly impact the validity of your results. Here are some common errors to avoid:

• Using exact category labels: This gives away the answer. For example, asking "Find information about pet adoption" directly leads users to the "Pet Adoption" category. Instead, rephrase it to avoid direct hints.
• Overly complex scenarios: Long, detailed stories can distract users. For instance, "You are adopting a pet and need to know the legal requirements and procedures for pet adoption in your area" is too lengthy and may cause users to miss crucial details. Keep it simple and to the point.
• Unclear or vague instructions: Avoid tasks that don't clearly direct users. For example, "Find important information about pets" is too broad and doesn't guide the user effectively.

A better task about pet adoption would be, “You are thinking of adopting a pet. Check if there are any resources on this site that can help you understand the process.”

Just like any testing method, tree testing has its limitations, with the major one being the lack of contextual data. Since the study is often conducted remotely and unmoderated, you miss out on observing user behavior, hearing their comments, and asking personalized follow-up questions. This means you might not fully understand why users make certain choices or what specific issues they encounter.

Here’s how to overcome this:

• Start with a few moderated pilot sessions: These sessions help ensure that task wording is clear and allow you to observe nuances in user behavior that might be missed in an unmoderated setting.
• Include a short survey after the tree test: Instead of asking users to recall confusing labels from memory, provide a list of labels and ask them to check which ones were difficult to understand. Follow this with an open-ended question for additional comments and feedback, allowing users to share insights that might not be evident from the click data alone.

The success rate metric in tree testing measures how well users navigate through your categories to find the correct information. It is calculated as the percentage of users who correctly identify the right category for a given task.

To determine the success rate, define at least one correct answer for each task. If users find the correct location, it's counted as a success. If not, it's a failure, even if they initially clicked the right spot but then backtracked. For example, if 100 participants are asked to find contact details for customer support and 75 select the correct category, the success rate is 75%. While a higher success rate is better, what counts as “good” depends on the task's importance.

Consider the following:

• Mission-critical tasks, like finding purchase options or emergency contact information, should aim for over 90%
• Less critical tasks, like finding corporate history, can have lower acceptable rates

Compare the success rate of the task to similar tasks within the same hierarchy level. This context helps determine if a 75% success rate is acceptable or needs improvement. Tree testing removes design elements like search functions and visual cues, providing a clear view of your navigation structure's effectiveness and helping you decide where to improve.^[2]

In addition to measuring the success rate, it’s crucial to consider how much users struggled to find the right information by looking at the time spent on tasks. The time spent metric indicates how long it took users to locate the correct category, providing insight into the mental effort involved.

If users took a long time to complete a task, even if they were ultimately successful, it may indicate that the process was challenging and mentally taxing. This can reveal areas where your navigation structure may be confusing or inefficient.

To analyze task times, start by comparing the times for various tasks in your study. Identify which tasks took significantly longer and investigate why.

Directness is a crucial metric in tree testing that measures how many users navigate straight to the correct category without backtracking or changing categories. This metric, often referred to as the "happy path," indicates a smooth, intuitive interaction with minimal confusion or detours.

Direct navigation is an important indicator of how well your information architecture supports users in finding what they need effortlessly. If a high percentage of users find the correct category directly, it suggests that your navigation structure is clear and logical.

However, it's important to note that users can also directly fail a tree-testing task. This means they confidently navigate to the wrong category without backtracking, believing they have found the correct answer. This type of failure is particularly concerning because it suggests a fundamental misunderstanding of your navigation structure. For instance, if users consistently go directly to "Privacy & Settings" instead of "Account Privacy" for managing account privacy, it indicates that the category labels or organization are misleading.

First-click metrics are vital in evaluating tree testing results, as they provide insights into user navigation behavior. The first click is the initial category users select when starting a task. In tree testing, this is always a top-level category because subcategories aren't visible until a parent category is selected.

When a task has a low success rate or directness, examining first clicks can reveal where users initially expect to find information. For example, if users consistently start in "Support" when looking for "User Guides," it suggests that these categories are too similar and might benefit from clearer labeling or separation.

The last click is the final category users select, indicating where they believe they’ve found the correct information. If users' first clicks are correct but the success rate is low, it suggests that lower-level categories overlap too much, confusing users as they navigate deeper into the structure. For instance, if users initially click "Vegan Products" correctly but then struggle to find vegan yogurts among "Tofu Products" and "Lactose-Free," it indicates a need for clearer distinctions between these subcategories.