Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
This time the majority of our followers were right. Read further to find out why.
But first, let’s see what SEO professionals had to say about the result of this test:
Nicole DeLeon, Founder at North Star Inbound:
The intent for those two queries is different. So to me, it has less to do with exact match and more to do with exactly matching intent. You can match intent exactly without exactly matching keywords and get the same result.
Dave Cousin, Head Of Organic at Oban International:
I’d need to know which market this is for and then look at SERPs to try and understand the intent a little better. If the intent behind ‘Ferries from Athens to Santorini’ is actually to compare and find the best price rather than to just get on and book immediately then the first version could actually work better at minimising short clicks and keeping users on the page.
Lidia Infante, Senior SEO Manager at Sanity.io:
I think this will have a positive impact because the intent behind the keyword is transactional, not commercial. This is better reflected by the new version of the page.
Do you agree with these statements? Follow us on LinkedIn to share your opinion on the next test.
The Case Study
In August 2021, Google announced an update on how they generate web page titles in the SERPs. While rewriting page titles is nothing new, it has become much more “aggressive” with this update. Before the update, the most common changes made by Google were minor, such as adding the brand name to the page title. Now Google uses different overrides, mostly they come from HTML text in the content of the page.
Google’s documentation states that they are making use of text that humans can visually see when they arrive at a web page. Google considers the main visual title or headline has shown on a page, content that site owners often place within <H1> tags or other heading tags, and content that’s large and prominent through the use of style treatments.
As an SEO, it is important to understand that one of the primary ways people determine which search results may be relevant to their search query is by looking at the titles of listed web pages. We believe that anything displayed in your SERP snippet can have an impact (positive or negative). In addition, previous case studies have shown that user behavior can have a significant impact on CTR and, thus, organic traffic to the website.
With all this in mind, it begs the question of whether we should take control of what appears in search results or let Google rewrite the page titles.
For a travel website, we noticed that Google was rewriting all the page titles of an important page template. This piqued our interest and inspired the following test.
The Hypothesis
For the client in question, we noticed that Google was using the <H1> heading on the page to rewrite the page title shown in the SERPs.
Please note that the website in question is not the one illustrated. All the images below are examples and not the actual website we run the test on!
We knew from search behavior analysis that the primary keyword for this page template was “ferry from Athens to Santorini” or “ferries from Jersey to France”; people didn‘t use “compare” in searches. Additionally, SERP analysis showed that the highest-ranking results all front-loaded the primary keyword, meaning the page title started with the primary keyword.
While including “compare” might make for a good CTA, we hypothesized that the page title our client wrote would better match the search intent. Moreover, when users scan the search results, it would be easier for them to see that the landing page matches their search query.
As we know, Google used the <H1> heading to rewrite the page title. We decided to match the <H1> to our client’s written page title, with the result that Google would show our clients written page title instead of the page title Google had rewritten:
This resulted in effectively testing whether our clients’ written page titles would outperform the title generated by Google from the H1 tag.
The Test
We used SplitSignal to make the <H1> change for the tested pages. In total, about 200 listing pages were selected as either variant. We started the test and ran it for 30 days. We found that Googlebot visited 92% of the pages tested.
In addition, we made a small script to keep track of whether Google would now show the desired page title. After examining all the SERPs, we were able to determine that Google was showing the correct page title, as expected.
The Results
After 30 days of testing, we reviewed the results. We saw that the traffic to the tested pages performed better than the control group, which means the test is positive and statistically significant at the 95% level. Matching the <H1> heading with the page title resulted in a 12% increase in organic clicks to the tested pages.
Note that we are not comparing the actual control group pages to our tested pages. But rather a forecast based on historical data. The model predicts the counterfactual response that would have occurred had no intervention taken place. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into the true impact of an SEO change.
Why
This case study shows that it is important not only to be aware when Google rewrites your page titles but also to test whether Google is actually showing a better performing page title.
With this <H1> heading change, we have effectively also changed the page title displayed in the SERPs. Analysis of the data shows that this test affected the click-through rate (CTR) of the pages tested. Not only did we see an increase in clicks, but we also saw an increase in impressions, suggesting that the pages tested became more relevant to the search queries for which the pages were ranked. We believe the success of this test was a combination of the H1 change and the page title displayed in the SERPs, both of which more closely matched the user’s search intent.
With split testing, you want to make sure that the changes you make will yield positive results. So as an SEO, you need to think about and experiment with different elements that make up your SERP snippet. Within search, you always compete with other search results. If your answer to a search doesn't stand out or doesn’t match expectations, a user can simply click on a different search result.
Something that works for one website may not work for another. So if you feel inspired to start split testing, check out this step-by-step guide to test and analyze SEO changes (yourself). The only way to know for sure is to test what works for you!