There continues to be a lot of buzz around marketing performance optimization and testing. Justifiably so. Technology is helpful with this, of course, but it’s non-technical expertise that really makes a difference.
I do a lot of performance testing with my clients, and I always follow scientific method. Here’s a primer on scientific method, along with a deep dive into one of its core elements: hypotheses.
Scientific method
Using scientific method offers a structured approach to performance optimization. Critically, it involves developing and testing hypotheses about what strategies or tactics will improve outcomes, like revenue, conversions, or engagement.
Scientific method:
- Provides data-driven insights to inform your decisions.
- Reduces risk by testing changes on a small scale before they are fully implemented.
- Enables continuous improvement by helping you identify not just what works, but why it works.
This approach allows you to ground you working in marketing testing in evidence, rather than assumptions.
There are eight stages to scientific method:
Researching and developing your hypothesis is a critical part of scientific method. Here are some tips and tricks for doing so successfully.
What is a hypothesis?
In this context, “hypothesis” is a fancy word for an idea of what might boost performance — but it’s also more than that.
You could say: “I want to test the color of our call-to-action buttons. Right now they are red, let’s try making them green and see if that boosts performance.”
That’s not a hypothesis. There’s no proposed explanation of why that change might be effective.
Case study: Building a hypothesis
Here’s a real-life example of how I built a hypothesis for a test years ago.
I was at my local Barnes & Noble bookstore, reading about the psychology of color in a book that I found in the discount rack.
It discussed why red was the color of stop signs — that subconsciously it could be sending a signal to stop. I realized that I had a promotional email with red buttons, because the brand colors were dark purple (it was almost black) and red.
The product was a financial advisory publication (we provided advice on what stocks to buy to enhance your financial portfolio), which is when I got to thinking about what red meant in the financial world. ‘In the red’ is a bad thing there — it means that you own more money that you have.
So, I thought, maybe the red buttons are depressing response.
But what to test against the red?
I thought of traffic lights, where red means stop and green means go. Then I thought about the financial world. Green is the color of money, which is what we were promising the stock recommendations would earn them.
The hypothesis that came from this was: “Changing the color of the CTA buttons from red to green will increase response and revenue for the reasons outlined above.”
See the difference? Tests based on hypotheses backed by sound reasoning are more likely to perform well.
Getting inspiration for hypotheses
Inspiration can come from internal or external sources. Here are a few ideas for finding inspiration for your tests.
Internal inspiration
Case study 1: Failed performance tests
Even if a test fails, you should look for learnings you can leverage in the future. For instance, years ago we pitted a control email against a version where we changed a number of different elements. Our KPI was conversions; the recipients needed to fill out a form to convert.
The control won soundly. But while the test version lagged in conversions, the click-through rate (CTR) on its top CTA button was nearly double that of the top CTA button in the control. As a result, we went on to re-test just the elements around the CTA button — including location and a message about this being an exclusive offer.
The hypothesis here: “These things appear to have increased CTR in the previous test; perhaps if we isolate them from the other elements in that test and apply them to the control, they will increase not just CTR but also conversions.”
This time the test won.
Case study 2: Campaign metrics
A lot of my client work involves multi-effort email campaigns, where we are sending a series of two or more emails to the same list over a period of time. I often get inspiration from the data, for instance…
In this example, we saw that effort 5, the last effort in the series, still generated over $18,000 — so perhaps we could garner an additional $9,000 or $10,000 by adding an effort 6. They hypothesis here “Since Effort 5 did well, we should be able to garner additional revenue by adding an effort 6.”
We also saw that effort 4 did well with $0.45 in revenue generated per email address, compared to just $0.32 for effort 3. Efforts sent earlier in the series tend to perform better. So we did a test with the hypothesis “Since earlier efforts tend to perform better, we should be able to get a lift in revenue by switching the order of efforts 3 and 4, since effort 4 generates a higher revenue-per-email-address.”
Dig deeper: Why we care about performance marketing
External inspiration
Your inbox
Your inbox can be a treasure trove of inspiration for performance tests. Do you get any testing ideas from this Walgreens email?
Here are the hypotheses that I derived from this email:
- “Including first-name personalization at the top of the email should pull more people in to read it, increase clicks. and drive more revenue.”
- “Putting the offer behind a ‘scratch off’ will increase engagements and conversion rates.”
- “Visually showing recipients their rewards status will motivate them to want to continue to earn reward points, which will increase revenue.”
Articles, blog posts, webinars, presentations and other resources
Any online resource can be a great source of inspiration; even better if it’s presenting case studies of performance tests others have done.
The last one on this list, Really Good Emails, is a swipe file. They have screenshots of more than 15,000 email messages; you can search on a number of different variables. Browsing swipe lists like this one are a great way to get inspiration for hypotheses.
Now’s the time to start performance testing, or to raise your performance testing game. I hope this primer on developing hypotheses helps.
The post Applying scientific method to performance appeared first on MarTech.
**Utilizing the Scientific Method to Enhance Performance**
In a world where continuous improvement is the key to success, whether in personal development, business, sports, or education, the scientific method offers a structured and reliable framework for achieving measurable progress. By applying this time-tested approach, individuals and organizations can identify problems, test solutions, and make data-driven decisions to enhance performance.
### What Is the Scientific Method?
The scientific method is a systematic process used to investigate phenomena, acquire new knowledge, or correct and integrate previous knowledge. It is based on empirical evidence and follows a series of steps:
1. **Observation**: Identifying a problem or area for improvement.
2. **Question**: Formulating a clear and specific question about the issue.
3. **Hypothesis**: Proposing a testable explanation or solution.
4. **Experimentation**: Designing and conducting experiments to test the hypothesis.
5. **Analysis**: Evaluating the data collected during the experiment.
6. **Conclusion**: Drawing conclusions based on the results and determining whether the hypothesis was correct.
7. **Iteration**: Refining the hypothesis or testing new ones based on the findings.
This method is not limited to scientific research; it can be applied to virtually any domain where improvement is sought.
—
### Applying the Scientific Method to Enhance Performance
#### 1. **Observation: Identifying the Problem**
The first step in enhancing performance is recognizing areas that need improvement. For example:
– In sports, an athlete might notice a decline in their endurance during competitions.
– In business, a team might observe a drop in productivity or customer satisfaction.
– In education, a student might struggle to retain information during study sessions.
By carefully observing and documenting the issue, you set the stage for targeted problem-solving.
#### 2. **Question: Defining the Challenge**
Once the problem is identified, it’s important to ask specific, measurable questions. For instance:
– “What factors are contributing to my reduced endurance during races?”
– “Why has team productivity decreased over the past quarter?”
– “What study techniques can improve my retention of information?”
A well-defined question provides focus and clarity, ensuring that subsequent steps are purposeful.
#### 3. **Hypothesis: Proposing a Solution**
The hypothesis is a proposed explanation or solution to the problem. It should be specific and testable. Examples include:
– “Incorporating interval training into my workouts will improve my endurance.”
– “Implementing a daily stand-up meeting will enhance team productivity.”
– “Using active recall techniques will improve my ability to retain information.”
The hypothesis serves as a roadmap for the experimentation phase.
#### 4. **Experimentation: Testing the Hypothesis**
This step involves designing and executing experiments to test the hypothesis. Key considerations include:
– **Controlled Variables**: Ensure that other factors remain constant to isolate the impact of the intervention.
– **Data Collection**: Gather quantitative or qualitative data to measure the outcomes.
For example:
– An athlete might track their performance over several weeks of interval training.
– A business team might implement daily stand-up meetings for a month and measure productivity metrics.
– A student might use active recall techniques for one subject and compare results with traditional study methods.
#### 5. **Analysis: Evaluating the Results**
After collecting data, analyze it to determine whether the hypothesis was supported. Look for patterns, trends, and anomalies. For instance:
– Did the athlete’s endurance improve after interval training?
– Did team productivity increase following the implementation of stand-up meetings?
– Did the student retain more information using active recall techniques?
Statistical tools and software can help analyze data more effectively, especially in complex scenarios.
#### 6. **Conclusion: Drawing Insights**
Based on the analysis, draw conclusions about the effectiveness of the intervention. If the hypothesis was correct, consider adopting the solution as a long-term strategy. If not, identify potential reasons for the failure and refine the hypothesis.
For example:
– If interval training improved endurance, the athlete might incorporate it into their regular routine.
– If stand-up meetings didn’t boost productivity, the team might explore alternative strategies, such as time-blocking or workflow automation.
– If active recall didn’t enhance retention, the student might experiment with other techniques, such as spaced repetition.
#### 7. **Iteration: Continuous Improvement**
The scientific method is an iterative process. Even if the initial hypothesis is successful, there is always room for further optimization. For example:
– The athlete might experiment with different types of interval training to maximize results.
– The business team might refine their meeting structure to make it even more effective.
– The student might combine active recall with other methods, such as mind mapping, to enhance learning.
By embracing iteration, you create a culture of continuous improvement.
—
### Benefits of Using the Scientific Method for Performance Enhancement
1. **Data-Driven Decision-Making**: The scientific method relies on evidence rather than assumptions, reducing
Recent Comments