How NS prevented €1.8M in revenue loss through experimentation

Insights/Action/Outcome: As NS grew its focus on experimentation, the company needed an easier way to link customer behavior, testing, and business results across its digital platforms. By using Amplitude, NS brought analytics and experimentation together in a single platform. This helped teams get reliable insights, improve customer experiences faster, and expand testing across the company. Now, NS runs about 60 experiments each year, generating around €2.4M in business wins and preventing €1.8M in potential revenue loss.

For Nederlandse Spoorwegen (NS), the largest railway operator in the Netherlands, customer experience impacts everything from operational efficiency to revenue growth. Every improvement to the ticket-buying process can make things easier for customers, boost sales, and reduce the need for support.

As a Data and Customer Experience Optimization Expert at NS, I spend a lot of time studying customer behavior, reviewing feedback, and running tests to improve how travelers buy and manage tickets online.

In the past few years, experimentation has grown from a small project into a key part of our team’s work. As this happened, we needed a better way to understand customer behavior and link test results directly to business results.

Moving from conversion optimization to CX optimization

Like many other companies, we used to rely on Google Analytics 4. It gave us access to data, but as our testing program grew, it became harder to use.

The platform was difficult for non-technical users, limiting its use across marketing and other teams. Advanced analysis often needed BigQuery and SQL, making it harder to see the impact of experiments or dig deeper into customer behavior.

At the same time, our goals were changing.

We stopped focusing solely on conversion rates and began looking at the entire customer experience. This meant measuring more than just revenue and sales. We needed to see customer effort, support needs, journey completion, and how different types of travelers behave.

Amplitude made this change much easier by combining analytics and testing in one place.

Building a more scalable experimentation culture

Now, teams across NS use Amplitude Feature and Web Experimentation to study customer behavior, review campaigns, track funnels, and improve the customer journey.

We start by looking at the data. We find where customers have trouble, come up with ideas, run tests, and then check the results to decide what to do next. Since analytics and testing are in the same place, we can go from finding insights to taking action much faster than before.

One of the biggest benefits is that we can now see the most important business metrics more clearly, such as reductions in customer contacts, revenue generation, content engagement, conversion rates, and time to order. AI-> customer contact metrics, content

Having all these metrics centralized makes it easier to spot trends, understand customers, and choose what to improve next. But it also lets us go beyond overall results and see how different customer groups experience our digital services.

Improving customer experiences through experimentation

Our customers are not all alike.

Some travelers simply want to buy tickets as quickly as possible. Others need more help, especially with ticket rules, discounts, or using the mobile app.

One of the most valuable things we’ve learned from testing is that context matters. So, we tested where travel restriction information should appear during the purchase journey. The information improved outcomes when shown earlier in the process, but introducing the same message later in the checkout flow negatively impacted conversion.

Without experimentation, we might have thought that more information always improves the experience. Instead, we learned that when and where you share information matters as much as the message—leading to more effective implementation and a better overall customer experience.

We’ve used the same approach to improve ticket-loading instructions, onboarding, QR code help, and guides for more complex ticket products.

Together, these efforts led to about 13,000 fewer customer service requests each year for e-ticketing support.

Why preventing losses can be just as valuable as creating wins

When people think about testing, they often focus on the experiments that succeed. For us, some of the most valuable tests are those that prevent us from making mistakes.

Today, about 25% of our tests are successful. Another 15% are what we call loss preventers. These tests show that a proposed change would have hurt customer behavior or revenue if we had launched it.

These findings are just as valuable as successful tests because they help us avoid expensive mistakes.

In the past two years, loss-prevention tests helped us avoid about €1.8M in possible revenue loss. At the same time, successful tests brought in about €2.4M in business value each year. This shows that testing creates value by both driving gains and preventing losses.

This mix has turned experimentation from just an optimization tool into a key business strategy that creates value by both generating wins and preventing losses.

Creating confidence through better data

As we do more testing, it’s more important than ever to trust the data behind our decisions.

NS has a complex digital setup with many departments, customer journeys, and business goals. To make decisions that affect revenue, we need to be sure our data is accurate. Amplitude has helped us build that trust by making it easier to check data quality, spot measurement problems, and confirm test results.

For us, switching to Amplitude was about more than just getting a new analytics tool. It was about building a strong base for improving customer experience at scale.

Now, every test helps us better understand our customers, remove obstacles in their journey, and make smarter choices about what to improve next.