SEO Split Test Result: Will Adding Emojis to the Meta Description Help or Hurt Your SEO?

Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

Here’s what other SEO professionals have to share about this test:

Victor Valentine Romo, SEO Manager at Grey Matter:

They don't harm or benefit your SERP. It's been said that Google may completely omit the emoji from search if it makes the result look misleading/spammy or replace it completely with an equivalent word.
If I had to guess, I'd say there was no change to your organic clicks.
Or, if there were, it'd be a challenge to attribute the emoji as a CTR factor when Google ultimately holds the decision on what is displayed for end-users SERP.

Gerry White, Head of SEO at Oda: 

I would bet that the right emoticon, the right description could significantly impact the conversion rate, a small increase from 3% to 4% is a 1/3 more traffic. These small boosts are often underrated. I think doing it right is important.

Elizabeth Chen, Digital Marketing Manager at Market America, Inc.:

Google stated that adding emojis to Titles and Meta Descriptions has no direct effect on SEO. So, using emojis won't directly affect your ranking. However, can emojis boost clicks? I am curious to find out what the result is.

Do you agree or disagree? Follow us on LinkedIn to share your thoughts on the next test.

The Case Study

Over the past year, we have conducted several emoji tests. Using emojis in page titles and meta descriptions always generates a lot of discussion within the SEO community. Recently, Google’s John Mueller has spoken out about the use of emojis. In summary, Google says you can definitely use emojis on your pages, but it won't help or harm your SEO in any way. These comments inspired us to test this across multiple sites and industries as we've seen that emojis can definitely affect SEO performance.

Google once said that users are often the best judges of relevance, so if a user selects a particular search result, it is likely to be relevant, or at least more relevant than the alternatives presented. We believe that anything displayed in your SERP snippet may have a positive or negative impact on SEO performance. When you make changes to your SERP snippets, you validate those changes not only for Google but also for real users.

As mentioned, we ran several emoji tests, for example, for an online drinks retailer, adding a cocktail emoji to their recipe page titles:

This test resulted in an 11% increase in organic clicks to the tested pages. 

We also tested with special characters (emoji-like characters) in the meta description for an online music store:

This test resulted in a 3% increase in organic clicks to the tested pages. 

Before we move on to the case, let's take a look at what emojis actually are and how you can use them for SEO purposes. Emojis may look like icons or small images, but they are not. They are a sequence of characters from the UTF-8 character set, defined by the Unicode standard. Since emojis are characters, they can be used in HTML just like any other character. This means you can add emojis to elements like the meta description. 

Google (and almost every other site) supports the UTF-8 character set and can therefore display emojis and special characters in their search results. In practice, however, not all are guaranteed to appear in the search results. 

The Hypothesis

A major online retailer wanted to test if adding a smiley emoji 😀 to the meta description would have a significant impact on organic traffic. 

They hypothesized that this change would allow them to stand out more from the other search results. They added a smiley emoji to the meta descriptions of their category pages with the expectation that this would have a positive effect on the organic traffic to the tested pages.

The Test

We used SplitSignal to set up the test. Around 1,000 category pages were selected as either variant or control through stratified sampling. We kicked off the test and ran it for 21 days. We were able to determine that Googlebot visited 99% of the tested pages.

The Results

After 21 days of testing, we reviewed the results. The changes to the meta description resulted in an inconclusive effect on organic traffic at the 95% confidence interval. 

When the blue shaded area in the cumulative view performs below or above the x=0 axis, the test is statistically significant at the 95% level. Looking at the cumulative representation of the test, we can conclude that the effect did not become significant. This means that we can not be confident that the increase we are seeing is due to the change we have made. 

When a test does not reach statistical significance for our chosen confidence level, we evaluate the test and hypothesis to see if we should still continue with the change.

While we were confident that this change would most likely not have an adverse effect, we did not proceed with the change.

Note that we are not comparing the actual control group pages to our tested pages. but rather a forecast based on historical data. The model predicts the counterfactual response that would have occurred had no intervention taken place. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into the true impact of an SEO change.

Why

As mentioned at the outset, we've seen emoji-like tests that have had a significant impact. 

When we compare this test with previous tests, there is one big difference, and that is relevance. A cocktail emoji for the page titles of cocktail recipe pages or musical notes for the meta descriptions of an online music store has a sense of relevance. For this test, we believe the emoji was irrelevant to the rest of the SERP snippet or landing page.

Meta descriptions are written for users and can influence user behavior. In this case, the emoji may have made the result stand out, but it didn't prompt users to click on the search result more than usual.

With split testing, you want to ensure that the changes you make will yield positive results. We want to prevent organizations from wasting time, money, and energy on changes that ultimately make no or only a limited positive contribution. Testing allows you to move quickly and implement proven positive changes. 

Finding ways to stand out and be the most relevant answer to a search query is essential for optimal organic performance. Keep in mind that something that works for one website may not work for another. The only way to know for sure is to test what works for you!