The datasine team

Facebook Ads A/B test guide

Facebook Ads A/B testing is crucial to understanding what marketing content works best. Datasine’s PPC Manager outlines how Facebook Ads A/B tests work in this simple guide.

What is A/B testing?

Facebook A/B testing is when you experiment with various elements in your advertising campaigns to decipher what drives higher performance. As you discover things that perform well (and implement them) as well as things that hinder performance (and eliminate them), you should see your campaign performance gradually increase. In Facebook Ads, A/B tests can be through the platform’s split testing functionality.


How to A/B test in Facebook Ads

The first thing you need before you begin A/B testing is a hypothesis. In the example below, we test the hypothesis: photo ads work better than graphic-only ads.


The results from our Facebook Ads A/B test

The results above clearly verify our hypothesis that ads with photos perform better than graphic-only ads.

But one test alone isn’t enough to test our hypothesis. We need to conduct several ‘photo vs graphic-only’ experiments to verify or falsify our hypothesis with certainty, because there could be many factors responsible for one ads’ success over the other. 

What to A/B test in Facebook Ads?

Ad visuals 

Most people running campaigns in Facebook Ads have done some research into or have experience of the audiences, devices and even copy that tend to perform well. And even if they don’t, Facebook Ads’ platform does an incredibly good job of optimising automatically using machine learning so long as you have a diverse number of ad sets in your campaigns. 

There is one thing however which is the most crucial to a successful Facebook Ads campaign: your ad visual. It is the element that stops your target audience in their tracks and grabs their attention. All other aspects of your ads (e.g copy and landing page) really only come into play if you have a good visual to get your target audience to look at your ad in the first place. 


But while Facebook’s platform is really good at optimising many aspects you can set at ad set level – such as audience, placements, device, etc. – it’s not as straightforward when it comes to your Facebook Ad visuals. 

Suppose you put two different images in your ad set, one close-shot and one long-shot of a person holding your product: if the long-shot does better, it doesn’t necessarily mean that it was the long-shot aspect that was the cause of success. It may be that it was actually the colour blue (featured more in the long-shot) which drove the higher performance. And, in fact, a close-shot with the emphasis of blue actually performs better than long-shots.

While you can test ad visuals using A/B testing to an extent – and you definitely should to get the most out of your Facebook Ads A/B testing – there are infinite variations to test, so this could end up being an endless experiment. Instead, focusing on the features in your ad visuals and conducting semantic content analysis can help cut back time and money spent on experimentation, while increasing ROI.


Content features our AI tool, Connect, found in this image

But, in order to scale this process, a marketer needs an AI or a machine learning model which can conduct semantic content analysis in moments.

This is where Connect can help you. The AI platform analyses your historical performance and gives you insights into what features are resonating with your audience, and which aren’t. Better still, it can use this information to rate and rank your ad visuals according to how effective they are going to be. 

So going back to our example, Connect’s AI will analyse the performance of your various ads and all the aspects within your ads and tell you that it was the colour blue that made the ad appealing. With this tool, therefore, you can supplement your investigation and creativity with data to save you time and money. 

Ad copy

Much like ad visuals, ad copy is something that requires the common sense and creativity marketers’ possess rather than something Facebook Ads’ platform can optimise with its machine learning alone. 

But, ad copy can still be optimised with Facebook’s machine learning algorithm through A/B testing. For example, testing exclamation marks, emojis, capitalisation and certain phrases can be done quite easily with dynamic ads and can show what copy is resonating most with your audience.


Other elements

Here’s some other elements to test but we recommend where possible to leverage  Facebook’s machine learning instead of manually A/B testing. For instance, let’s say you want to test different placements, it’s easier to select automatic placements with conversion optimisation and allow the platform to gradually select the highest-performing placement for your campaigns.

  • Interest-detailed targeting
  • Demographic-detailed targeting
  • Gender
  • Age
  • Geo-location
  • Landing page


Three Facebook A/B testing rules

  1. Ceteris paribus: The Latin phrase meaning “other things equal”. In our context, it means that Facebook A/B tests should only test one element at a time. So if you’re testing blue vs red, absolutely everything else (audience targeting, placements, ad copy, etc.) must be the same.
  2. Statistical significance: Just because an experiment seemingly verifies or falsifies a hypothesis, it doesn’t mean that it actually has. In order for the test to be valid, you have to have a sufficient amount of volume to your ads. There are plenty of statistical significance calculators on the web to help you with this. However, sometimes it’s not just about a simple calculation but about using common sense. For example, in our graphic-only vs photo test, there are many elements involved which cannot be controlled so it makes sense to test a diverse range of photo and graphic-only ads to get an accurate result from your experiment.
  3. Performance compass: Let performance, specifically cost-per-conversion, guide and measure your results. An increase in CTR is great but if it came at the cost of a higher CPA then, although there is some learning that could be derived, you haven’t directly found anything that drives higher performance for your campaigns.


How to set up a split test in Facebook Ads

Step 1: Create a campaign.

Step 2: Turn the split test toggle on. (Note: The campaign budget optimisation toggle has to be turned off to be able to do this.)

Step 3: Choose a variable (creative, delivery optimisation, audience, placement and more than one).

Step 4: Choose the number of ad sets (from two to five).

Step 5: Set the aspects you want to test separately to those which you do not. For example, if you wish to test ‘Creative A’ vs ‘Creative B’, then select the ad sets and edit at the settings at the same time since all these should remain constant. After that, you can put in the creatives in each ad set in turn.

For more detailed instructions for split testing on Facebook Ads, check out Facebook’s own split testing guide.


Filter by Category:
Jergan Callebaut
The AI tool marketers are using to increase Facebook campaig...

A new breed of marketing tools is here that leverages machine learning and psychology to pick high-converting creatives for your Facebook ads.

Use AI to identify high-converting marketing images

Cutting-edge AI (artificial intelligence) research into how computers see marketing images is changing the way marketers build more successful campaigns.

In Conversation with… Bernardo Nunes, Head of Science at G...

Growth Tribe’s Bernardo Nunes joins the conversation to talk artificial intelligence in marketing: how companies can leverage AI and cultivate tech talent.

Adtech vs martech: what’s the difference?

Adtech and martech are terms that are regularly conflated. Let’s put aside misconceptions to define adtech and martech – and their differences.

In Conversation with… Harry Hugo, Co-Founder of The Goat A...

Serial Entrepreneur and Co-Founder of The Goat Agency, Harry Hugo, joins the conversation to talk social media, influencer marketing and founding Goat.

Forms of personalisation in today’s digital world

Personalisation means connecting with your audience on a deeper level. Our Head of Psychology, Jergan Callebaut, looks at its many forms.

What makes us click? Increase ad performance with creative d...

Datasine’s CEO Igor Volzhanin outlines his MAD//Fest presentation, explaining why using creative data and AI leads to a steep increase in ad performance.

In Conversation with… Mohamed El Hannaoui, Growth Marketer

Experienced Growth Marketer Mohamed El Hannaoui joins the conversation to talk growth marketing in startups and how AI is going to change marketing

Your images matter: five reasons why

Got your attention with the picture above? Great! I’m sure you’ll love the rest of this article. It probably doesn’t come as a surprise to you that we live in the age of visual culture. Ten percent of all photos ever taken by humankind took place in the last 12 months.

What’s the most important part of Facebook Ads? The creati...

The data boom has driven focus on areas such as targeting, brand and reach for Facebook Ads, yet the creative remains the most impactful factor in successful Facebook Ads

Speaker spotlight: Emily Miller, Head of Marketing at Ollie ...

Datasine talks marketing, AI and personalisation with Emily Miller, Head of Marketing at Ollie Quinn, ahead of her fireside chat at #SuperchargeMarketing.

Five of the best tools for Facebook Ads in 2020

Datasine looks at the five of the best – and most affordable – tools on the market for creating and optimising Facebook Ads.

Growth hacking Facebook ads

Growth hacking Facebook ads sends engagement skyrocketing and there are many ways to optimise paid ads to drive brand awareness and sales.

You’ve overlooked creative data. But we’re pioneering a ...

Creative data has been overlooked by marketers. But at Datasine we’ve found a way to drive performance with this powerful data: Content Atomisation – supercharged by AI.

The “integrators”: Why fusing creativity and data drives...

A McKinsey report has concluded that marketers who use creativity and data together achieve a much higher performance for their brand

Speaker spotlight: Mitch Platt, UK Country Manager at Growth...

Datasine discusses growth marketing, personalisation and AI adoption with Mitch Platt, UK Country Manager at Grow Tribe, ahead of his presentation at #SuperchargeMarketing.

Speaker spotlight: Mack Grenfell, Biddable Lead at Babylon H...

Datasine talks to Mack Grenfell, Biddable Lead at Babylon Health about falling into the industry, and how AI is helping marketers optimise spend.

Semantic content analysis will disrupt marketing – for the...

Discover the benefits semantic content analysis brings, and how to utilise data, psychology and AI to predict the highest-converting content for your brand.

Running a data-driven marketing team

Data-driven marketing is a key focus for marketing teams today. I spoke with Alessio Pieroni, Head of Marketing at Mindvalley about how to practice it successfully.

Tackling data-driven disruption as a Creative

Data-driven disruption is sweeping through creative industries. We sit down with Wayne Deakin, Executive Creative Director at Huge EMEA to talk about facing it head on.

Nine Mailchimp integrations for personalisation

Check out our top nine Mailchimp integration picks for empowering marketers to increase campaign metrics and improve customer experiences.

In Conversation with… Meagan Healey, Head of Content &...

Head of Content & Product Marketing at Attest, Meagan Healey, joins the conversation to talk data-driven marketing and why aligning sales and marketing is a game changer.

Speaker spotlight: Karla Rivershaw, Head of Marketing of Tur...

Datasine discusses the tools that make marketing work, and finding the right balance when personalising communications with Karla Rivershaw, Head of Marketing at Turtl.

IWD: 20 marketing leaders in 2020 [1/2]

For International Women’s Day 2020, 20 female marketing leaders weigh in on the advice they’d give their younger selves [1/2]

Which Tokyo 2020 art poster does our AI say is the most enga...

Datasine Connect, Datasine’s AI platform, analysed the 12 official art posters for the Tokyo 2020 Olympic games to discover which was the most engaging for the UK population

3 simple tips for building better Facebook Ads

Datasine’s Performance Marketing Manager gives 3 simple steps for building better Facebook Ads, including optimising creatives and using AI

The three must-ask questions before a rebrand

Datasine’s Marketing Director, Cornel Lazar, shares the questions each brand must ask before rebranding, and Datasine’s journey to change its look and feel.

Three Mailchimp newsletter growth hacks

Email is one of the most effective marketing channels and Mailchimp has long been a favourite amongst marketers and growth hackers. Learn how to up your Mailchimp newsletter game with our growth hacks. 

Speaker spotlight: Emma Wynne, Head of Growth Marketing at P...

Datasine talks about personalising marketing communications and marketing tools with Emma Wynne, Head of Growth Marketing at Pasta Evangelists.

Can AI optimise the creative process?

While artificial intelligence (AI) has had little impact on the creative sector so far, AI has huge potential to optimise ad creatives

Marketing 20:20: meet the Speakers

Datasine sits down with the two guest speakers of Marketing 20:20, Becky Simms, Founder and CEO of Reflect Digital and Social Media Expert Kieran S Lawler

How to grow your brand using paid social

Brand Manager and Founder of, Magda Urbaniak, shares her best practices for brands paid and organic social efforts

Which images make the most successful Facebook ads?

Images are the cornerstone of successful Facebook ads. We look at how to find the best images to create more effective Facebook ads.

What is a rebrand? (And what is it not?)

I sat down with our Marketing Director to discuss what a rebrand is, what goes into a rebrand and how Datasine approached the project.

MAD//Fest Day One: the recap

We look back on our first day at MAD//Fest with presentations from Burger King, Virgin Trains and our own CEO Igor Volzhanin.

The life of a Creative Director – in the age of data and A...

Data and AI are becoming more vital to the role of the Creative Director. Datasine sits down with Alex Colley, Creative Director of ikon, to find out more about the impact tech is having.

IWD: 20 marketing leaders in 2020 [2/2]

For International Women’s Day 2020, 20 female marketing leaders weigh in on the advice they’d give their younger selves [2/2]

Mirroring at scale

How mirroring can help you level up the impact of your one-to-many communications, offering personalised content and customer experience.

Delivering truly personalised email marketing experiences

Datasine sits down with Lemlist CEO Guillaume Moubeche to talk about delivering personalised email marketing experiences and building relationships.

How AI is set to transform email marketing

Artificial intelligence (AI) is increasingly used in email marketing, learn how machine learning enhances personalised email campaigns.

A quick guide to remarketing with Facebook dynamic ads

Facebook dynamic ads, or Facebook product ads, let you remarket to your audience. Find out how to use them in this simple guide to DPA ads.

What is segmentation in marketing?

Segmentation in marketing allows marketers to place customers into groups for targeting. We take a look at the types of market segmentation.

Five things we learned at Supercharge Marketing

Datasine looks back at the five things we learned at Supercharge Marketing, from data-driven marketing to investing in learning new skills.

Make your marketing pandemic proof

In crisis times, marketing budgets get slashed. Here’s how we can support you to stretch your budget further and drive sales – even in the face of a pandemic.

What does your audience want to see from your branding?

Datasine sits down with Flavilla Fongang, Creative Brand Strategist and Founder 3 Colours Rule to talk branding, neuromarketing and the future of the industry

A/B testing is a lie. Here’s the truth.

A/B testing has long been used to find the perfect ad creative – but its effectivess is greatly exaggerated. Here's the truth of A/B testing, and how to find the best creative without it.

From binary baby to “Friends selfie”: A (brief) ...

We see thousands of images online every day. We take a trip back over the history of the digital image to see how we got to this point.

AI-powered marketing needs interpretability – and collabor...

Artificial intelligence (AI) should be taking marketing by storm, but its lack of interpretability for most marketers is holding back innovation and adoption.

How to use AI to optimise your marketing content

AI can drive marketing success by optimising audience targeting and reach but, more importantly, AI optimise your creative content through Content Atomisation.

How to send the right message at the right time, in travel m...

We speak Joe Stevens, Corporate Digital Marketing Manager at Flight Centre about using data to send the right message to the right person at the right time

Are you blinded by data?

Learn how developing an Integrator Mindset can help you to creatively personalise and build genuine relationships with your customers

How to use data to inspire customer loyalty in the travel in...

Lack of customer loyalty is a key problem in the travel industry. We take a look at some of the ways data can tackle the issue.

Creativity back on top: Why creative data will dominate this...

Data-driven marketing has swept through the industry, but the data focus has led to a decline in creativity. But you can combine data and creativity by uncovering your campaign’s creative data.