Amplitude on Amplitude: How We Approach Learning as a Team

At Amplitude, we believe that product intelligence is a team sport—that analytics needs to be easy for the whole team to adopt. We believe this adoption not only drives a better understanding of a team’s impact but it also helps them identify and align around the right problems and solutions. Many of our customers are also focused on driving adoption, so we often get asked how we use Amplitude to drive results.

In this post, we’ll walk through how our team uses Amplitude to collaborate throughout the product development process. To explain, we’ll share how we iterated through multiple versions of a new feature called My Workspace, which allows you to find your Amplitude analyses quickly and reliably, so you can get back to work. Most importantly, we’ll share tips on how you can infuse collaboration at each step of the product process to drive results.

Stage 1: We Defined the Problem

As you’re defining the problem, it’s important to ensure the team is aligned and bought in. Everyone needs to understand the problem that they’ll ultimately be solving together.

At Amplitude, we invite the whole team to our user research sessions so that the engineers who will be building the solution can hear the customer’s pain firsthand. This collaborative approach also extends to our quantitative analysis.

Last year, during research discovery calls with new Amplitude users, we learned that most of them were having trouble finding and re-accessing their past analyses in Amplitude. Many were even developing hacks to get around this problem. We used Amplitude to verify that solving this could be a growth opportunity and found that users who hit our North Star Metric of becoming weekly learning users (WLU) more frequently revisited their own content than those who did not.

As we analyzed and sized the problem, we added each new chart to our pod’s Team Space to keep everyone up to date on progress. Every time new insights were added, a Slack notification was triggered to our team channel via the Amplitude Slack integration. Everyone on the team knew right away when there was new data, and they could stay aligned without ever leaving their workflow.

Stage 2: We Ran an A/B Test

After defining and sizing the problem, teams can think about how to design a solution with the data they’ve found, launch a new experience, and collaboratively analyze the results.

At Amplitude, we make every team member a “co-owner” of our feature release dashboards so that they can contribute analyses. This means designers often create charts to help guide toward the best solutions. For example, with our content-finding problem, we had landed on building a “personal workspace” that showed users’ recent content. Our designer used Amplitude to determine the optimal number of saved chart thumbnails to show on the page based on how often users save in Amplitude. This helped ensure we were more likely to get that thumbnail number right.

Next, when we’re ready to kick off our A/B test, we use an Amplitude Notebook, a data storytelling tool that lets you add context to charts and graphs and share your findings with others. We use the Notebook because we believe it’s important to ground experiment results in as much context as possible.

For the My Workspace Notebook, we built charts using our A/B test funnel view, Impact Analysis report, and Retention report to assess impact on our success metrics. We also added notes directly onto the charts themselves to let the reader know the takeaways they should be focusing on. Just as importantly, we included the goals of the test within the Notebook along with screenshots of the new experience. What’s critical here is that as we’re building and reviewing the analysis, everyone can see this context, so we stay on track.

Ultimately, we saw an increase in our success metrics along with strong adoption and retention, indicating we should roll out My Workspace.

Stage 3: We Shared Our Results

Now that you’ve collected and analyzed your data, you can share it with other teams and stakeholders at your company to show why you are choosing to move forward with a specific feature.

Putting all the data in a Notebook allowed us to present our findings in team meetings without having to export charts and create a presentation. Amplitude-shared links allowed us to link to the Notebook in Slack so that other teams could read about the results without the need for endless clarifying questions. When a question did arise, they could simply leave a comment on the Notebook, again triggering a Slack notification.

Bonus: We Generated a Hypothesis (Again)

Once your findings are documented and shared, use them to crowdsource new hypotheses. By aligning throughout the process, you’ll find that everyone on the team likely has high-value ideas on what to try next based on all the shared learnings.

After seeing positive results with My Workspace and hearing qualitative feedback from people who had used the feature, we wondered whether it might be an effective homepage and entry point into the Amplitude product.

To find out, we ran a Pathfinder analysis. Pathfinder shows the route people take after they perform a specific event—in our case, starting a session. We found that the majority of users navigated directly to their own content before going anywhere else.

We put the analysis in our team Notebook. That way, all of the data for the My Workspace feature was together, and the team could easily access data from the whole process in one place.

Our new hypothesis is that having My Workspace as the Amplitude homepage will make it faster for users to access their content and start working, driving up our key engagement metrics. Stay tuned for the results!