Tree Testing vs Card Sorting Showdown

Tree testing evaluates the navigational structure of a website by having users find specific items in a hierarchical tree, Card sorting organizes content by letting users group and label cards to understand how they perceive information architecture.

It delves into the user’s mind to uncover the hidden psychological patterns and instinctive tendencies that shape their expectations for navigation and usability.

The tree testing vs card sorting debate does not claim that only one of these two UX research techniques is invaluable. Both techniques offer extremely valuable insights and they are both equally important for professional UX designers.

Most professionals combine the diverse insights these research techniques reveal with data analytics, user interviews, and usability testing to create truly user-centered experiences.

However, as mentioned above, tree testing takes a top-down approach and card sorting takes a bottom-up approach. In this article, we will explain what that means.

We will also analyze the two UX research techniques in great detail and settle the ‘tree testing vs card sorting’ debate once and for all. But first, let us clarify the ‘top-down’ vs ‘bottom-up’ aspects of these two UX research techniques.

Tree Testing vs Card Sorting

Dictating the navigation of a website is one of the main tasks of a UX designer. This task is pretty complicated because organizing any type of information is complicated. To understand why, consider this example: imagine you are organizing a bookshelf. You have two options:

• You can either organize the books based on the publisher’s categories (date, fiction/non-fiction, and other categories)
• Or, you can take a more personal approach, grouping the books based on your favorite authors, genres, etc.

Which method of organization is the most user-friendly? This is where two research techniques, Card Sorting and Tree Testing can help.

Tree Testing

Let us say you have meticulously organized the books by genre. Now, it is time to put your organizational skills to the test. Enter your family members, i.e., the test participants. You present them with the predefined structure you have created and ask them to navigate that bookshelf and locate specific books.

This top-down approach tests how well your pre-defined structure aligns with users’ expectations. It is like checking whether the assumptions you made while organizing the books in a specific, logical way match your users’ needs and expectations.

Based on the test results (how easily your family members locate specific books) you can make changes to the way the books are arranged on the bookshelf, making it more user-friendly.

Card Sorting

Card Sorting takes a bottom-up approach. So, let us take a step back and imagine that you are organizing the bookshelf from scratch. This time, instead of following a predefined structure based on your assumptions, you let your family members create their categories and organize the books based on their understanding.

You study how each family member organizes the books. In doing so, you uncover their mental models & and instinctive tendencies when it comes to organizing information. Then, you use this understanding to build a user-centric structure (specific way of organizing the books) from the ground up.

This analogy perfectly describes tree testing vs card sorting:

• With tree testing, you get to validate an existing information architecture (from the top down) and reveal areas where users feel lost/confused.
• With card sorting, you get to discover how users think (from the bottom-up) and then build an information architecture based on their mental models.

This analogy also encapsulates the complexities UX designers face when designing website navigation structures:

• They can either impose a predefined structure based on their assumptions (regarding how users navigate through information)
• Or, they can delve into users’ mental models to create user-centric experiences.

There is a third option- they can do both. That is because both tree testing and card sorting share the same goal- to help UX designers create the most user-friendly navigation structures and information architectures (IA) for their websites.

Each UX research method approaches this challenge from a different perspective and professional UX designers use them both at different stages of their research process. For example, UX designers often use card sorting to generate new, user-centric ideas on how to organize information in their web designs.

Then, they conduct tree testing to evaluate these ideas. Now that we have discussed the core tenets of the tree testing vs card sorting debate and clarified that they are not opposing forces, but partners in creating user-friendly web designs, let us dive deep into each of these UX research techniques.

Tree Testing

Tree testing is a UX research technique used to assess the effectiveness and usability of pre-defined information architectures (IAs). UX designers present users with a pre-defined IA (i.e., the ‘tree’). The users are then asked to navigate through the IA to find categories, labels, and cards that are relevant to their needs.

Their success/failure rate at finding what they need in the pre-defined IA is used to measure the effectiveness of the design.

Here is a screenshot of the navigation menu from an academic library website. This menu has a pre-defined IA. On it, you can find many options including: “Home,” “Find & access,” “Study & create,” “Teach & learn,” “Research & publish,” and “About us.” 

Each of these options has its own set of subcategories. By asking the students of this university to navigate through this menu and complete specific tasks, we can evaluate the effectiveness of the page’s pre-defined IA. For example, let’s conduct a basic tree test on this pre-defined IA:

• Let us ask 10 students to find and read all the content under the “Study & Create” section in the menu
• Then, let us use software to track their behavior.
• Let us track all the navigation paths taken by the 10 students.
• Then, let us assess how much time each student needed to complete the task.

Ideally, all 10 students should be able to navigate the menu without any usability issues. But, if some of them fail or face usability-related issues, we can use that info to make adjustments to the IA. We might use the info revealed by this basic tree test to adjust the menu labels or to cross-link certain pieces of content in different places.

The best thing about tree testing is that it allows UX designers to fix issues in the IA long before any major design work is performed. Most professional designers run tree tests solely to optimize the information architecture itself, long before developing any content for their website.

How to Run a Tree Test

To run tree tests, UX designers first assemble groups of participants. A typical unmoderated tree test will feature around 50 participants. Then, they present the participants with interactive prototypes of their website’s IA or navigation menu. Then, they assign certain tasks to the participants.

These tasks are based on actions that the average user typically performs on the website. For example, the participants can be asked to navigate to specific signup pages or find certain pieces of information in the interactive prototypes.

The UX designers then observe the participants’ behavior using remote usability testing tools like Treejack or UserZoom. These observations log every step and misstep the test participants make and they help UX designers understand which aspects of their predefined IA are the most and least user-friendly.

Here is how you can tree-test your website’s prototype structure or IA:

1. Define Your Research Plan

Start by verifying your prototype site’s structure or IA, i.e., the “tree” itself. Make sure it’s ready for users to explore. Since participants will interact with ‘text-only’ labels and options, prioritize clarity. Make sure each label is properly termed and easily visible.

Aim to create concise test sessions (lasting no more than 15 minutes) with less than 10 tasks to ensure your participants remain engaged throughout the test. Then, define the metrics you’ll use to measure the success of this tree test.

Typically, UX researchers use software tools to track:

• The different navigation paths users take
• The number of times users succeed or fail in completing specific tasks
• Average time spent completing each task

UX researchers aim for high rates of ‘direct path’ decisions, which refer to the types of decisions the design team expected the users to make. They also aim for high success rates (in completing specific tasks) and low time-on-task rates.

2. Map the Tree’s Branches

Outline the categories/subcategories/pages that’ll be presented in the tree test. Make sure these categories are rightly worded and provide sufficient context for users to navigate through them. For example, if you’re tree-testing an eCommerce site, avoid using interchangeable terms like “Browse Products” and “Start Shopping Now” as top-level categories.

Even subtle inconsistencies like this can lead to confusion among test participants. Instead, distinguish clearly between each label/option using terms like “Featured Product Categories” and “Shop Now.”

3. Create Tasks for Test Participants

Create tasks intended to guide participants toward specific locations/branches of the tree. For example, you can ask the participants to navigate to a specific help page. Make sure your questions are clear, and concise, and effectively prompt users to take specific steps.

4. Assemble the Participants

Recruit participants (at least 50 people) who accurately represent your target audience.

5. Select Mode of Exploration

Choose between in-person or remote testing. With in-person testing, you can observe participant behavior and collect feedback from them in real time. But, it’s more expensive and difficult to conduct than remote testing which can be performed via online tools like Maze.

6. Pilot Run and Launch

Conduct a pilot run with your design team to identify any potential issues with how the test is set up. Once your pilot run is complete, launch the main test!

7. Analysis

Analyze the results. Compare multiple tree structures to uncover patterns in user behavior. Verify all the metrics you defined earlier in the test. Gather additional insights through follow-up surveys and user interviews.

Tree testing will reveal the usability, findability, navigability, and overall effectiveness of your website/app’s IA and navigation structure. It will answer questions like:

• Is the content on your site organized in a way that matches your target user’s logical expectations?
• Is the terminology used in the labels comprehensible?
• Can users find the info they need on their own?
• Which areas of navigation need to be improved?

Once you’ve gathered these insights into your target user’s mental models, you can prioritize issues for iteration and filter your design ideas based on your target user’s needs/expectations. But, what if you’re designing a new website with a novel IA from scratch and you’re unsure of how your target audience would categorize and organize the information?

That’s where tree testing stops being effective and UX designers turn to card sorting.

Card Sorting

Card sorting is a UX research method used to analyze the mental models that users apply when they organize information. Unlike tree testing where users are given pre-defined information architectures, card sorting allows users to be their information architects. Users are given the building blocks (information items) and asked to build their architectures/hierarchies.

This bottom-up approach to observing how users categorize and label info on their own enables UX designers to uncover the hidden pathways in the users’ minds. This assessment helps UX designers understand exactly how their target users want to experience their designs. It gives them the foundation they need to create an IA that feels intuitive and easy to navigate.

But, what is card sorting? Imagine shuffling a deck of cards featuring different labels titled “features,” “product categories,” etc. In card sorting, users are asked to shuffle and categorize these cards based on what’s logical to them.

UX researchers track their decisions to understand how their minds organize information. But, card sorting is not a one-size-fits-all type of research method. UX researchers like to vary these tests to reveal the different shades of logic users apply to organize information.

Sometimes, users are given pre-dealt categories to sort into. Other times, they’re asked to sort completely random cards from scratch. Here are five such variations of card sorting that UX researchers perform:

• Open Card Sort: Users independently create categories based on how they think. They’re given total freedom so this method is great for understanding mental models. However, the process can be time-consuming and user inputs can be disorderly.
• Closed Card Sort: This method is used to test existing information architectures. Users are asked to sort ‘cards,’ i.e., pieces of information not in any way they like but into pre-defined categories.
• Hybrid Card Sort: Users start with pre-defined categories but can create new ones during the test. It offers the best of both open and closed card sorting. But, users must be given clear instructions on what they can/cannot do in such tests.
• Modified Delphi Card Sort: This method involves multiple sorting rounds. After each round, experts review user decisions to make refinements to their IA. This combination of expert knowledge and user input helps researchers validate and refine complex IAs.

Here is a brief overview of these Card Sorting Techniques as well as of their strengths & weaknesses:

Card Sorting Technique	Description	Level of User Control	Structure Control	Benefits	Drawbacks
Open Card Sort	Users randomly create categories, and label them on their own	Users control both the way they sort the cards & the way they label them	None	– Uncovers user mental models freely – Flexible and extremely insightful	– Results can be subjective – Requires a ton of analysis from the researchers
Closed Card Sort	Users sort items into pre-defined categories	Users only control the sorting, not the labels	Researchers provide pre-defined categories	– Efficient and standardized – Useful for validating pre-designed IAs	– Limited user input on how the labels should be categorized – High risk of researcher bias
Hybrid Card Sort	Users either sort the cards into existing categories or create new ones	Users control both, with clear guidance from the research team	Researchers provide initial categories but give users the freedom to customize	– Combines the pros of open and closed sorting – More flexible than its counterparts	– More complex than its counterparts – Requires clear instructions
Modified Delphi Card Sort	It is an iterative process with numerous sorting rounds; each round is reviewed by experts	Users and experts work together to refine the categories	While experts dictate how different categories are developed, users guide them via their inputs	– Strong way to validate and refine pre-existing IAs – Data-driven approach	– Time-consuming, expensive, and requires high levels of expert involvement – Unsuitable for obtaining quick insights

Each card sorting technique offers unique benefits/drawbacks. By understanding these pros/cons, UX researchers can perform multiple card sorting tests until they create information architectures that truthfully resonate with their target users. Some other key benefits of card sorting include:

• It generates new, user-friendly ideas for organizing content on the website/app
• Helps UX designers understand how their target users like to navigate digital products
• Reveals how well users will receive the design

Can UX designers create the perfect IA after reviewing the results of a single card sorting test? No. Card sorting will not help UX designers magically create user-friendly information architectures. It can only serve as a guide, giving designers brief insights into how their target users’ minds work.

To validate these insights, designers must perform other tests (interviews, surveys, etc.). But, the ease with which card sorting can be performed makes it a favorite among professional UX designers. Here’s how anyone can perform card sorting:

• Define research goals
• Set aside any assumptions related to how you’d like the info on your site/app to be organized
• Focus on what you want to understand; select specific sets of info (design elements, product categories, etc.) that you want the test participants to organize
• Write content on cards; maintain clarity, avoid bias and repetition
• Choose your test type (check the chart)
• Conduct pilot tests with colleagues.
• Launch the test
• Let the test participants categorize the cards naturally.
• Conduct interviews with the test participants to gain deeper insights into their mental models
• Analyze the results, look for common themes, outliers, and other patterns

Perform different types of card sorting tests to get a varied understanding of your users’ mental models. Then, design your information structure based on your findings. Validate your design decisions with users before finalizing them!

Conclusion

Both tree testing and card sorting are essential in UX research. As long as designers know the right time/context to use each research method, both can help UX designers create more user-centric web, app, and IA designs. That is why at Design Studio, we use both of these research techniques as part of our UX research services to make well-informed design decisions. It is also why our designers are so proficient in delivering usable, easily navigable, and engaging digital experiences! To learn more about how we perform tree testing and card sorting, contact us now!