Shortening the Trail on a Rail Site

Background photo by ad agency Arnold for Amtrak’s “Somewhere Somebody Misses You” campaign in 2008.

What to UnXpect When You Are UXing

In the spring of 2026, I ventured out of my quantitative comfort zone and adventured into an alien territory of user experience (UX) design, using qualitative methods.

This case study highlights a few memorable differences between quantitative and qualitative research that I have struggled with this semester. I hope this case study will be of help to those in similar situations.

The project my team worked on for this Information Architecture & Interaction Design course was redesigning certain aspects of Amtrak’s website. I will use our design process and our final redesign proposal to illustrate areas that quantitative researchers should not expect or take for granted when conducting UX research and creating a user-centered design or product.

Motivation

Amtrak served a record-breaking ridership of 34.5 million trips in 2025. Redesigning Amtrak’s website presented an opportunity to improve the user experience for tens of millions of passengers every year.

Problem

Amtrak users expect to find information related to their trips easily, but some are frustrated in the process. How can we surface the needed information sooner and improve certain aspects of the website to align with travel industry standards?

Complications

A team of four aspiring UX designers and researchers undertook this project, each with a different working style and personality. One of us (me) has worked and researched with only quantitative methods and has no prior UX experience.

Iterations of Insight Discovery and Testing

The process of our user search started with user interviews, followed by card sorting, tree testing, and usability testing. (Figure 1) In every testing, participants were asked to think aloud and we also asked follow-up questions.

Figure 1. User search consists of a series of iterative insight discovery, synthesis, planning, and testing.

In a quantitative study, it is typical for researchers to start by developing a research question and a hypothesis to test for, say, its statistical significance, get results, and discuss them.

In contrast, a qualitative user experience research process does not start with a hypothesis; it starts with insight discovery through exploratory methods, including interviews of target users.

In fact, it is important to avoid making assumptions (“hypotheses”) when conducting interviews or any kind of testing. Instead, one needs to have open ears to listen actively and open mind to capture and synthesize the insights that emerge during the process.

As a “quant,” I was surprised by how many tests we had to do in user research. Unlike quantitative methods, in which a researcher obtains results after hypothesis testing, user research is an iterative and interconnected process involving a series of insight discovery and testing. More importantly, from each test, one does not get results but insights, and those insights start to surface during testing. (Figure 1)

The Irreplicability of UX Research and One-Size Does Not Fit All

Replicability is a core component of many quantitative analyses: using the same data and following the same methodology, anyone should be able to get the same results as the researchers. This, however, is not the case with qualitative studies.

First, depending on the interviewer and the follow-up questions, different insights could be derived from the same interviewees. Different researchers reviewing the same interview transcripts can also discover different insights. In other words, there is no universal truth from the data. (Figure 2)

Figure 2. Different insights and design directions can be derived from the same interviews.

Secondly, even when following the same methodology and drawing on the same interview insights, different researchers can arrive at different design proposals. This is because different teams may have different priorities and constraints regarding which issues to address first, resulting in different tasks to test and, consequently, different features to improve further. One good illustration of this is the treatment of Accessibility / Accessible Travel on two rail monopolies in Figure 3:

Figure 3. VIA Rail Canada places Accessibility on the first row of its navigation bar, while Amtrak buries it under “Planning & Booking” in the Plan mega menu, reflecting different priorities regarding accessibility.

In user research, it is especially important to remember that “one size does not fit all.” Even if your research is (miraculously) replicable, it does not mean that what works for one user group will also work for another. This is true even among your target users — primary users may have different priorities than secondary users, and primary users may have drastically different mental models. In short, no matter what you design, some users might absolutely adore it, but some might not care for it at all. (Figure 4)

Figure 4. Users may find the second entry point for Manage Trip confusing or convenient, depending on whether they are on the homepage or other pages on the site.

The Point About Shared Pain

As someone who has used statistical analyses heavily in her professional work, I was very concerned about sampling methods and sample size. I frowned when we were asked to interview merely 6~8 users at the beginning of the project.

A magic I came to appreciate about qualitative methods, too late in the process, was that as few as 5 participants would be needed in a qualitative usability study to find most usability problems; to have some sort of statistical power, you would need as few as 20 users. These small numbers would be non-starters for traditional inferential statistics!

Surely enough, after conducting semi-structured interviews with eight participants, we were able to identify two main pain points that were shared by multiple users:

• Amtrak’s current interface interactions do not align with industry standards.
• Finding specific details and information is difficult before and after booking.

I was amazed by how important pain points were shared by most of our interviewees. It is no wonder you would need as few as 5 participants to discover most usability issues, since many users experience the same pain when navigating a site.

Participants Aren’t Subjects…They Are Co-Designers!

The magic number of at least 5 worked like a charm, but I didn’t appreciate another aspect of it until I read the first chapter of “Rocket Surgery Made Easy” to prepare for our prototype testing. In this chapter, subject-matter expert Steven Krug explained that, in quantitative research, people are often measured and thus are called “subjects.”

In user research, it is your design that is being evaluated, and people are there to help with the evaluation, thus they are referred to as “participants.” It is very important that every effort is made to make participants understand that they are not being evaluated. This is why, before every single test, we emphasized to our participants that there were no right or wrong answers and reminded them: “We are not evaluating you in any way. We are evaluating the website.”

Get Betty Yang’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Sometimes, participants make ideal co-designers, and many user research projects invite them to join the design team. Once I started to think of participants as co-designers, I appreciated the magic of at least 5 even more: although having little statistical power, imagine how powerful your design can be if five more designers joined the team.

Figure 5. Participants aren’t subjects in your usability study; they are your co-designers!

No Need to Fidget with the New Booking Widget

Multiple users expressed their confusion and frustration with Amtrak’s booking widget, specifically the date picker. One user even commented on how it took him “a few minutes of trial and error to understand what is going on here with the date calendar.” Another user thought she needed a return date for a one-way trip.

Currently, Amtrak’s booking widget requires users to select the trip type (one-way, round-trip, etc.) from a dropdown menu on a banner above and across the main area of the widget. “One-Way” also serves as the default item label for what should be categorized as “Trip Type.” The most confusing aspect of this widget’s design is that the Return Date field remains visible even for one-way trips. (Figure 6)

Figure 6. Amtrak’s existing booking widget design, with problematic areas marked in red.

We performed a comparative review of four companies in the travel industry. This was one of the areas where I could contribute meaningfully and significantly, given my business background and MBA training.

Because Amtrak is a monopoly of national passenger rail service in the U.S., we included the “Amtrak of Canada” as a comparable, an intracity rail company as a regional competitor, a national airline, and a national bus company, as flights and bus rides were considered by some users as alternatives to train travel:

VIA Rail Canada, owned by the Canadian government, operates the only public passenger rail across Canada.

Brightline is the only privately owned intercity passenger railroad in the U.S., connecting major cities in Florida. It is considered a regional competitor for Amtrak.

Southwest is a major U.S. airline operating a point-to-point network focused on high-frequency, short-haul routes. Some users mentioned it both as a benchmark and an alternative to Amtrak.

Greyhound is the largest provider of intercity bus transportation in North America, serving over 1,600 destinations. Bus transportation was mentioned as an alternative to train travel in user interviews.

After examining the booking widget across all companies in our comparative review, we observed an industry standard: the trip type selection is on the widget itself, and the Return Date field disappears when a one-way trip is selected. (Figure 7)

Figure 7. Companies chosen for our comparative review seem to have formed an industry standard on the booking widget.

We have all heard the advice: “Just because your competitors are doing it, it doesn’t mean that you have to do it, too.” However, when enough sites in your industry offer a certain way to interact with a widget, it becomes an industry standard, and users are likely to expect similar interactions from your widget. (Figure 8)

Figure 8. Participants compared their user experience on Amtrak’s site to that with airlines during user interviews

It was an easy decision for us to redesign Amtrak’s booking widget to be aligned with the industry standard: trip type selection is now on the widget itself and the Return Date field is now dynamic and disappears for one-way trips. (Figure 9)

Figure 9. The redesigned booking widget displays the trip type on the widget itself and has a dynamic Return Date field that disappears for one-way trips.

What’s on the [Mega] Menu? Users’ Mental Models!

During the open card sort, I encountered users’ mental models for the first time. The most impressionable example was seeing how “onboard dining” was grouped as “onboard” by some users who thought of their trip in terms of stages of travel (e.g., before, during, and after), and as “dining” by other users who grouped it according to functional categories such as Tickets, Food, and Help. (Figure 10)

Figure 10. Two distinctly different mental models regarding “onboard dining”: “Onboard the Train” as a travel phase or “Food & Dining” as a functional category.

How exactly did we figure out our users’ mental models? We conducted open card sorts using UX Metrics, an online user testing platform that also processes and synthesizes test data. After all participants complete the card sort, researchers can review the correlation matrix and grouping agreement scores to assess how each label relates to others and how much users agree on the grouping of each label.

This was another area where my past statistics training allowed me to explain the concepts of correlation and grouping agreement to my teammates and analyze the data closely to identify emerging mental models.

Users’ divergent mental models were most evident in how participants placed highly related labels in the same group, and yet fewer than half of the participants agreed on a specific grouping category. For example, participants placed highly related labels such as “Onboard Dining Options” and “Special Menus & Special Dietary Requirements” in distinctively different categories of “Food & Dining,” “Onboard Services & Experience,” and “Trip Planning,” reflecting different users’ thought processes regarding travel and the categories they might check first when looking for information on the test labels. (Figure 11)

Figure 11. Excerpt of highly related labels being placed in distinctively different categories by more than half of the participants, reflecting divergent mental models.

When we examined Amtrak’s current Plan mega menu, where many of the trip-related information items are housed, we noticed that the grouping logic was a mixture of function (Planning & Booking) and travel phase (“Onboard”), while the items under them do not all follow the grouping logic nor match the groupings that emerged from our open card sort. (Figure 12)

Figure 12. The existing groupings of “Planning & Booking” and “Onboard,” and the items under them, do not align with the main mental models that emerged from the open card sort.

We decided to apply the “stage of travel” mental model to reorganize the mega menu and informational items are now grouped under “Before You Go,” “At the Station,” and “Onboard the Train.” (Figure 13, outlined in blue):

Figure 13. Applying the “stage of travel” mental model to categorize informational items and surfacing Accessibility & Special Assistance to improve its findability.

During our prototype testing, we noticed that participants could quickly locate “Baggage Information” to complete their task of finding the baggage allowance after locating the entry point to the mega menu.

Another design that reflects users’ mental model is that an item’s findability depends on its personal relevance. Accessibility & Special Assistance were organized under Accessible Travel, Help & Info, or Policies & Regulations, depending on the topic’s relevance to the participant.

Amtrak currently buries “Accessible Travel Services” under “Planning & Booking” on the Plan mega menu (Figure 3). We decided to separate out Accessibility & Special Assistance into its own category to make it easier to find. (Figure 13, outlined in purple.)

Faring Well at the End of My Amtrak Journey

Amtrak has an old print ad that shows a passenger riding Amtrak from “Point A to Point Be,” which I parodied and posted at the top of this blog.

My very first UX design journey began on a rail site with the potential to improve the user experience of tens of millions of riders each year. With initial apprehension, I navigated through a series of insight discovery and testing. During the research and design process, I often felt beaten and buried by the amount of work involved to create a user-centered design. As I showcase a mid-fidelity prototype, I cannot help but feel content to bring to fruition a design proposal I believe could help some users and to see my personal growth along the way.

This case study documents my journey through learning the ABCs of a user-centered design as a “quant.” As I conclude my journey, I would like to offer my 20–20 hindsight to those in similar situations on a few things I wish I had done:

• Learn to use some popular UX tools (Figma, Miro, UX Metrics, etc.) beforehand. It is too much to learn new tools while constantly trying to deliver in a fast-paced, demanding process.
• Document everything and keep a diary to note your realizations, inspirations, reflections, and decisions, as opportunities to capture certain artifacts and thoughts are often transient.
• Read more case studies to learn how others tell their stories and how you can improve. “What is not communicated does not exist.” Similarly, a story not told well is likely to fall on deaf ears.
• Try to enjoy the messy process that is the nature of UX design, even though at times it might feel like you are working in the dark, don’t know where you are, what you are doing, and where you are heading.

Background photo created by ad agency Arnold Worldwide (Virginia) for Amtrak’s “Point A to Point Be” campaign in 2009.