by Looking for better ways to improve your Facebook ad performance? Have you used Facebook's Experiments?
In this article, you'll learn how to use the Facebook Experiments tool to run three different types of tests on your Facebook ads.
What Is the Facebook Experiments Tool?
Facebook Experiments is a new feature in Business Manager that allows advertisers to set up tests at no additional cost. Facebook says, “Experiments combines multiple types of tests, formerly labeled as Test and Learn and Split Testing, allowing marketers to A/B test campaigns and measure conversion and brand lift in one place.”
Advertisers can choose from three different experiments to run:
- A/B Test
- Holdout Test
- Brand Survey
Before Running Experiments
There are a few things to keep in mind before setting up your first experiment.
As with any test, you'll want to run your experiment under as close to normal conditions as possible. Pick a time of year that's as steady and close to the norm as possible. Avoid running tests on days such as Black Friday or other holidays and times where competition might spike and factors may increase drastically, hurting the accuracy of your test results.
You'll most likely want to create new Facebook campaigns to run your experiments so you can iterate based on the results you see. You don't want to waste time and effort on ads that are past their peak performance.
To reduce any chance that tests and data overlap, you should only run one experiment at a time. Start by thinking about what you want to test. What business goal do you want to achieve with your Facebook campaigns? Create a hypothesis that you can test with one of the experiment types.
Once you've created your campaigns, navigate to the Experiments tool. You can find the Experiments tab in the newest version of Business Manager located under the Analyze and Report section of the menu bar.
Note that experiments can raise your cost per 1,000 impressions (CPM) because they may split your targeted audiences into various test groups.
Once you've determined what you want to test and an optimal time to run your experiments, consider which test makes the most sense to use.
Curious About How to Use AI?
We recently launched a new show that help marketers, creators, and entrepreneurs understand the business applications of AI.
It's hosted by Michael Stelzner and explores this exciting new frontier in easy-to-understand terms.
Pull up your favorite podcast app and search for AI Explored. (Look for the cover art shown to the right.)
Or click the button below for more information.
#1: Set Up an A/B Test With Facebook Experiments
The A/B test is a great way to compare the performance of two or more versions of copy, images, audiences, or campaign objectives. Make sure you choose only one variable for your A/B test or you won't know how conclusive your test results are.
First, decide if you want to test copy, creative, audiences, or campaign objectives. Make sure each campaign has the same budget and will run for the same amount of time.
Pro Tip: Start by testing bigger differences such as which type of creative performs better (compared to something small like a different color of someone's shirt in an image). From there, you can iterate and narrow your tests as time goes on.
Create two or more campaigns that are identical, except for the one main difference you're testing. To illustrate, suppose you're testing two different pieces of creative. You create two campaigns with the Conversions objective optimizing for Purchase. Each campaign will target the same 1% lookalike audience of purchasers and have a budget of $50 per day.
Use one piece of creative in one campaign and a different piece of creative in the other campaign. In the example below, the first creative is an image of a tequila-themed water bottle at the beach during sunset, and the second creative is a video of different alcohol-themed water bottles. Notice how the ad copy is exactly the same for each campaign.
After you've created two campaigns with one difference to test, you're ready to start your A/B test. Go to the Experiments section of Business Manager and click the Get Started button under A/B Test.
On the Test Details screen, add your two campaigns to the Campaign 1 and Campaign 2 boxes, respectively. If you want to test more than two campaigns, click the Test Another Campaign option to add more.
Next, select the start and end dates for your A/B test. Note that you can't run a test for longer than 30 days.
Also name your A/B test clearly. This will save time in the future if someone else on your team is reviewing your experiment results.
Next, choose how you want Facebook to decide on a winner for your A/B test:
- Cost per Result will measure which campaign has a lower cost for the result you select from the Key Metric drop-down menu. You can choose from a variety of results including Cost per Link Click and Cost per Purchase.
- Cost per Conversion Lift will create a 10% holdout group. Holdout groups are groups of people in your targeted audience to which Facebook won't show your ads. Facebook will compare this holdout group with people who have seen your ad to measure the effect your ads had on each audience.
For this example, choose Cost per Result and leave the key metric as Cost per Result to test your campaign objective of Conversions optimizing for Purchase.
After you've decided how Facebook will select a winning campaign, check that the Estimated Test Power is above 80%. A high test power helps ensure your A/B test will be effective and lead to conclusive results.
If your estimated test power is lower than 80%, increase your budget and/or the length of your test.
When you're finished setting up your A/B test, click the Review Test Details button and double-check all of your A/B test details. If everything looks good, click Create Test to launch your first experiment.
Facebook will then take you to the Results section of the Experiments tool. Here you can see current and past tests that you've run, including the A/B test you just created.
It will take some time before your A/B test begins. When your experiment starts, you can click on your test to see results as they start to populate.
Ready to Supercharge Your Marketing Strategy?
Get expert training and an unbeatable conference experience when you attend Social Media Marketing World—from your friends at Social Media Examiner.
Broaden your reach, skyrocket your engagement, and grow your sales. Become the marketing hero your company or clients need!
🔥 As a valued reader, you can save $900 on an All-Access ticket if you act now. Sale Ends Friday! 🔥
GET THE DETAILSAfter the test is complete, Facebook will clearly label the winning campaign and show the confidence of the test results. The confidence score indicates how likely you are to get the same test results if you were to run that test again. For an A/B test, a confidence of 65% or more is considered statistically reliable.
Running A/B tests is a great way to determine what you should double-down on in your advertising strategy. Keep testing and comparing different creative, copy, audiences, and more to find out what best resonates with your ideal customers. Again, start wider with your testing and then iterate and narrow your variations over time.
#2: Set Up a Holdout Test With Facebook Experiments
If you want to test how people who don't see your Facebook ads compare to those who do, a holdout test is the perfect experiment to run.
For a holdout test, Facebook will withhold your advertising from 10% of your targeted audience and then compare the results of that audience with the rest of the people to whom they showed your campaigns. Running a holdout test lets you see how your advertising affects your conversions compared to your other marketing efforts.
To create a holdout test, go to the Experiments tool and click Get Started under the Holdout Test section.
Now select what you want to test. For a holdout test, you can choose from these options:
- Account—test all of the ad campaigns you're running
- Campaign Group—test a few of your campaigns that you group together
- Campaign—test one selected campaign
For this example, select Campaign Group so you can test how your new campaigns may positively influence your targeted audience conversion rates.
Name your campaign group clearly and select the campaigns to include in your group. Then pick your start and end dates and name your holdout test.
Scroll down and select the various conversion events you want your holdout test to measure. You can pick up to 10 conversion events for which Facebook will report conversion lifts on the Results tab.
Select conversion events that align with your business objectives. If you own an eCommerce business, you may want to select the following conversion events because they're related to the sales process:
Once you've selected your conversion events, click on the Review Test Details button.
Ensure your holdout test looks okay and then click the Create Test button to start your test.
You'll then see your holdout test on the Results tab of the Experiments tool.
After 100 of the selected conversion events have fired, you'll see the results of your holdout test in the Results section of the Experiments tool. Click on your test to view the results.
Notice that in the left sidebar, you can select which conversion event metrics will display for your holdout test.
Holdout tests help you determine how much your advertising compels people to take action compared to the holdout group.
#3: Set Up a Brand Survey With Facebook Experiments
A brand survey lets you measure the brand lift from your Facebook campaigns. This test is designed for larger ad accounts that have spent $10,000 or more in the past 90 days. Use brand surveys to determine if your ads have increased your targeted audience's brand awareness and recall.
Brand survey tests have to run for a minimum of 30 days and spend at least $30,000. These tests also leverage a holdout group to compare how effective the survey results are.
To begin a brand survey, go to the Experiments tool and click the Get Started button under the Brand Survey section.
Now decide whether you want to run the brand survey on your entire ads account, a campaign group, or a campaign. Then choose your region and vertical (industry). Make sure you choose the appropriate vertical so you can compare your brand lift percentage with the industry average.
Next, select your start and end dates and name your brand survey clearly.
Then scroll down and select the poll question you want to ask. Note that Facebook requires you to create a poll question for Standard Ad Recall first. You have the option to add two more questions.
Review the poll preview to make sure the question looks okay. Then scroll down and select additional poll questions.
When you're finished setting up your poll questions, scroll down and click on the slider under Conversion Events. This will measure brand lift on your various conversion events but won't influence your brand survey results. This gives you more data you can leverage later to see if your brand survey led to increased conversions.
When you're finished setting up your brand survey, click the Review Test Details button. If the details of the test look correct, click the Create Test button to start your brand survey.
Just like the other tests with the Facebook Experiments tool, after you create your brand survey, you can see your metrics in the Results tab. You want to see a 90% or higher confidence level for brand surveys to be deemed statistically reliable.
Brand surveys are ideal for larger brands that want to see how campaigns have affected brand awareness and recollection. Use them to determine if your ads are memorable and resonate with people.
Conclusion
Before running tests, make sure you clearly define your business objectives and what you want to achieve with testing. Use the Experiments tool to run A/B tests, holdout tests, or brand surveys. Then review the outcome of your tests and take action based on what the data shows you. Continue to improve and test over time.
Know that there rarely is one answer to everyone's advertising inputs. Get inspiration from others and feel free to test what they say, but know that not all results are easily replicated. Testing is vital so you can see what works specifically for your business and prospective customers.
What do you think? Have you tried the Facebook Experiments tool yet? What's the next test you plan to run for your advertising campaigns? Let me know in the comments below!
More articles on Facebook ads:
- Discover seven important Facebook ad metrics you need to know.
- Create three custom reports in Ads Manager to quickly analyze your Facebook ad performance.
- Explore six Facebook advertising mistakes and how to fix them.
Discover Proven Marketing Strategies and Tips
Want to go even deeper with your marketing? Check out the Social Media Marketing Podcast! Publishing weekly since 2012, the Social Media Marketing Podcast helps you navigate the constantly changing marketing jungle, with expert interviews from marketing pros.
But don’t let the name fool you. This show is about a lot more than just social media marketing. With over 600 episodes and millions of downloads each year, this show has been a trusted source for marketers for well over a decade.
