SPEAKER_00 0:00

Welcome to CaveBits, your source for the latest insights in website behavior analytics brought to you by Mouseflow. In this episode, we'll talk about building a data-informed CRO experimentation roadmap in 10 steps. So let's get started. For every conversion rate optimization or CRO professional, there's a moment of triumph when the organization they are working with recognizes the value of experimentation and decides that they need a proper experimentation program of their own. Yay! If you just celebrated this moment, congrats! Yet this is only the beginning. Now it's up to you as a CRO professional to deliver valuable learnings and exceptional results and prove that investing in experimentation and you was the right decision for the organization to make. There are so many things for you to test, so where do you start to maximize the impact? To answer that, you need a CRO roadmap, and a good one. What is a CRO roadmap? A CRO roadmap is a document that lists experiments that you plan to run within a certain time frame, with priorities, hypotheses, details, and expected outcomes defined for each experiment. Here are some things that make a good CRO testing roadmap. It's agreed upon by all stakeholders and is aligned with the business goals. The experiments described in it rely on data-informed hypotheses and are well documented, and it prioritizes the experiments consistently, based on evidence. So why do you need a CRO roadmap? Well, first, test prioritization and strategy. There's an infinite number of things you could test right now to make sure that at each moment you're testing that one thing that is likely to have the most impact, you need to prioritize. And after that test, you also need to not jump on a random testing idea that you got from it or from stakeholders, but strategically move to the next most promising test on the list. Every CRO expert says that CRO is a long strategic game. There is no place for spontaneous decisions in it. Having and sticking to a conversion rate optimization roadmap helps adhere to the chosen strategy. Second, focus and consistency. It's not uncommon for the very same stakeholders who supported having the experimentation program to occasionally undermine your efforts by introducing their own ideas, disrupting your planned process. And that's another reason why you need a CRO roadmap. If there's a document that all parties have agreed on and that is aligned with the business goals, it's much harder for the management to hijack your plans with this brilliant idea that they've just had and that they want you to test as soon as possible. Next, planning and resource allocation. It would have been much simpler if it was only you who's involved in running CRO experiments, but it's never like that. CRO always involves many parties such as analytics, design, and development. More often than not, there are also marketing people involved. In a way, they are all a part of the CRO team and need to be on the same page about who's doing what and when. And of course, there are other stakeholders that agree to having an experimentation program because they have expectations. That can include people from sales, product, and even the CEO himself. To make sure all these people dedicate their time to executing their part of the experiment when it's time to do so, you need to plan together with them in advance. Having a document to which all parties involved in experimentation can refer when planning their time helps a lot. So, here's how to create an experimentation roadmap in 10 steps. Number one, know the business goals. Before coming up with any ideas, make sure you know the business goals that you're trying to achieve. If a conversion rate optimization roadmap is not aligned with the business goals, it's doomed to fail. If you're the business owner, likely you have the goals in mind. If you're an employee or a consultant helping the business, you absolutely have to be 100% aligned with the people running the business on what they're trying to achieve. Have the business goals in writing on the first page of your CRO roadmap. Number two, jot down the strategy. Once the business goals are clear, it's time for you to figure out the strategic goals. What would you be trying to implement to achieve the business goals? For example, if the business goal is increasing average order value, which is not always about higher conversions, there are different ways to approach it. Upsell the website visitors, highlight more expensive goods, or adjust pricing. Or maybe you could use a combination of these. That's your CRO strategy that should go next to the business goals to make sure your roadmap follows it. It doesn't need to be detailed at this stage, but it should outline the areas you'll be focusing on. You're doing it to align the CRO goals with the business goals. Number three, set up the tooling. To run experiments, you'll need a tool set, and it goes beyond simply having an A-B testing tool, even though it's also a must for effective CRO. You'll also need tools that collect the customer data that helps you come up with hypotheses and understand if your experiments are actually successful or not. There are different data sources, and ideally, you'll need a tool for each. Quantitative Analytics. Use tools like Google Analytics or Adobe Analytics to analyze standard metrics such as conversion rates, bounce rates, click-through rates, and so on. Behavior Analytics. To add depth to this data, use a heat map tool and session replay tool like Mouseflow. These tools provide visual insights into how users interact with your website, helping you identify areas for improvement. User feedback. Asking your users opinions directly via surveys and feedback forms and reading the reviews on third-party websites helps to gain a deeper understanding of their needs and pain points. For that, you'll need a user feedback tool, and, well, some time to analyze it and go through the reviews. User testing. You can take a step further and conduct interviews and usability tests to better understand how users navigate your site. That helps discover areas where they struggle and evaluate the effectiveness of potential changes. Then, depending on what or where you're going to test, you may need other tools like landing page builders, email automation tools, and whatever else. Number four, run a CRO audit. Running a CRO audit of your website and campaigns helps you achieve two very important things. First, it gives you a baseline, the starting points to which you can compare your CRO results when you run tests. And second, it gives you ideas and hypotheses. As you analyze different parts of your website, the metrics, the user behavior, and so on, you'll see a lot of opportunities to improve something. Number five, write down hypotheses you get from the audit. More often than not, A-B testing hypotheses stem from thin air. Sometimes they emerge from your experience and your perceptions of best practices. Other times they may arise from directives handed down by your boss or your boss's boss. But that's why you've been running a CRO audit, to get data-driven hypotheses that are much more likely to win than random ones. Here's the formula for the perfect conversion rate optimization hypothesis compiled by one of the industry's best experts, Johnny Longdon, Group Digital Product Director at Boohoo Group PLC, who shared, if we change this thing, it will have this impact on the metric, because your theory as to why that would work. To put it simply, to formulate a strong A-B testing hypothesis, you need to focus on the why behind the potential changes. Support them with additional data. Usually, one source of data is enough to come up with an idea, but to make it into a strong hypothesis that is more likely to win, you need additional evidence. Your most valuable tools in this case are heat maps and session recordings, or user feedback or user testing. Ideally, all of them. But also, don't forget other sources of evidence such as customer support tickets, customer reviews, and etc. I'll share another quote by Johnny Longden, who says, You can take your hypotheses to another level by being far more descriptive about why you believe the change would have that impact. 7. Get stakeholder buy-in. Somewhere around this stage, it's also important to make sure you have stakeholder buy-in to run the tests that you want to run. Stakeholders usually have their own reasons and opinions. CRO experts even have a name for some of those. Getting approvals can be tedious, and people often try to minimize the amount of them by only getting the final approvals. That's a mistake though. If they don't get them, all the work they've done would be in vain. So even though it may seem counterintuitive, it's better to seek stakeholder approval as early as possible. Before jumping from theory to action, make sure you have support from all stakeholders and all the necessary resources such as time from developers, designers, marketers, and other people who are going to be involved in this one way or the other. Solid data, especially when presented visually through tools like heat maps or session replays, holds persuasive power with stakeholders. So it's important to collect it before trying to get their approval. James L. Barksdale, CEO of Netscape, shares this. If we have data, let's look at data. If all we have are opinions, let's go with mine. HIPPO, or HIPPO, is an acronym for highest paid person's opinion. The term was coined because there's an actual problem behind it that a lot of CRO professionals face. There's a tendency in organizations to lean heavily on the viewpoints of highest ranking individuals. This prioritizes subjective judgments over objective analysis or valuable insights derived from data. And the problem goes deeper than running experiments that are less likely to win. Hippo-driven decision making leaves lower-ranking employees frustrated, disempowered, and demotivated. In the absence of sufficient data, the hippo wins every time. The most effective approach to address HIPAA behavior is to prepare the necessary data beforehand. It's hard to go against hard data. So, data, especially if it comes from multiple sources, is the best defense against hippos. And that is yet another reason to have a data-informed CRO experimentation roadmap. Number eight, document the experiments. Prior to this stage, you've been investing your time in research. Now you're moving on to acting on it. Every hypothesis that you came up with is just a hypothesis. To become an experiment, it needs much more detail. Such as, what exactly are you going to change for the experiment? What metrics will you track? Where the experiment is going to take place? What audience segment are you planning on running it? What are the expected outcomes? How long should it take? And how much traffic it requires for statistical significance? What is the minimum sample size? What are the necessary resources? Who are the stakeholders and more. Number nine, prioritize the experiments. Now that you have everything documented, you are one step away from having a ready and actionable CRO roadmap. To get from a list of experiments to a testing roadmap, you need a system to prioritize the tests. How do you do it? Obviously, you want to prioritize the ones with the highest impact. But also, you likely would want to take into account the amount of resources that goes into each experiment, the confidence you have in it, and the amount of people the experiment influences. It's reach. There are a bunch of different models used for scoring. Ice score, impact, confidence ease, rice score, reach, impact, confidence, ease, and others. And number 10, plan your tests and build an experimentation calendar. At this point, you're almost good to go. The finishing touches are scheduling your experiments and building an experimentation calendar so that it's easy for you and everyone else to understand what experiment is running now and what's next in the queue. The golden rule here is to run one experiment at a time, even if you have resources to build more at once. Following this rule is important because that's the only way you'll be able to clearly see the impact each experiment will have. If you're running two tests on the same page at the same time, you don't know which one contributed to the success or failure. Done with that? Congrats! Your experimentation roadmap is ready. Use it to guide your experimentation efforts and to shield from random assignments and experiment ideas people may want you to try. So this is when you start to execute and analyze. Each experiment will bring 10 times the value if you properly analyze the results. Analyzing the results is much more than saying, this experiment is a winner, let's implement the changes. There's a million and one actionable insights waiting for you if you spend time digging into the results, segmenting them in various ways, and looking for patterns. Post-test segmentation helps you understand things like how different variations performed across various devices, which user segments were positively impacted by the change, how the CRO test effective behavior of different user segments, why did the winning variation win, and etc. This understanding is a perfect source for new hypotheses and also for pre-test division, choosing which audience segment to run the next experiments on. Using post-test segmentation to drive pre-test division was the ultimate weapon that allowed Matt Scabrook, founder and ex-director of optimization at WeTeach CRO to turn a 10k win into a 250k win. As usual, when analyzing the results, focus on the why. The very same behavior analytics tools, heat maps and session replay, can help you understand not only if your hypothesis was right or wrong, but also why it was right or wrong. So in conclusion, having a CRO roadmap solves multiple problems, from knowing what to focus on at each moment to protecting your CRO program from unexpected requests that always occur. It also helps maintain a steady test velocity so that you are always prepared and don't have huge pauses between the tests. Backing up each hypothesis in the testing roadmap with data and properly documenting every test makes the roadmap much stronger and better at both driving revenue and maintaining focus.