Is your UX design hurting your SEO & customer acquisition?
It’s commonly accepted that companies should split test their design on their marketing and user signup pages to improve conversion rates. This practice is somewhat misinformed, as it assumes that your on-page A/B tests do not affect incoming traffic at the top of the marketing funnel.
On-page UX changes actually do impact SEO and your top-of-funnel traffic. You should be testing UX changes for SEO in addition to conversion rates if you care about growth. We’ll demonstrate how we split tested a UX change for 7 Cups to improve their organic search traffic, and then we’ll share how you can apply this for your own business.
Helping 7 Cups improve SEO with UX A/B tests
7 Cups is a community of online therapists and active listeners supporting one another. They’re experienced with split testing titles and meta descriptions to influence clickthrough rates on the Google search results page, but their UX test to improve their SEO was a surprise win.
Over the past few years, they’ve grown 40X with SEO. Their customer acquisition is entirely organic search traffic coming in from Google with long-tail search queries. At RankScience, we’ve been helping 7 Cups split test their landing pages to further improve their SEO.
The winning A/B test which improved SEO was a UX experiment with their mobile app download banner:
The previous mobile download banner on all of 7 Cups’ SEO Q&A landing pages
The banner sat directly underneath the header, but above the primary landing page content and the breadcrumbs. We ran an SEO split test with the mobile download banner removed:
After 18 days, the results were conclusive: removing the banner created a 21% increase in organic search traffic. Just deleting some HTML lead to more users!
Wait, what? You can remove markup to improve SEO? The SEO stereotype is to add – not remove – countless search keywords, UI gizmos, and WordPress gadgets into your markup to draw in more search traffic.
The classic SEO joke: more is better
Turns out that this isn’t true. Google sometimes ranks your page much higher when you remove these UI gizmos and WordPress gadgets.
Here’s how we ran this SEO A/B test and how you can replicate similar experiments with your own company’s search traffic.
A/B Test Background: 7 Cups
Although 7 Cups already was performing well with organic search, they still had a lot of untapped value. Many of their long-tail pages were scattered on Page 1 of search results – some at the top, but many pages ranking in spots #5-10, where they weren’t receiving many clicks. Moving up to the middle or top of the page could increase their traffic by an order of magnitude, literally 10x their user acquisition from SEO.
The first links on the SERP get disproportionately more clicks than the last links (source: Moz)
From an SEO perspective, 7 Cups was already pretty advanced. It has over 180,000 user-generated landing pages drawing in organic search traffic. Of particular interest is their forum, where the user-generated content draws long tail traffic from users with an interest in mental health.
We wanted to optimize these forum pages to draw in more search traffic.
Running the A/B test for SEO
7 Cups used RankScience as a CDN proxy to implement a large number of A/B tests concurrently, including removing the mobile download banner.
For each test, about 500 pages were split into 2 groups, the control group and the variant group. While the control groups were unchanged, RankScience applied the tests to the variant groups.
Before implementing the tests to the variant groups, we measured each group’s search performance. After running the tests for a few weeks and collecting data, we compared each variant group’s performance against their counterpart control group.
A/B test result
Most of 7 Cups’ A/B tests failed to produce a significant uplift. Yet in the handful of tests which produced wins, the biggest win was removing the mobile download banner in the header.
The number of users clicking on 7 Cups from Google increased 21% (yes, this was statistically significant).
7 Cups then had data to support their SEO changes to roll out to the rest of their site. As expected, the new title format continued to grow over time, even months after the experiment ended.
What did we learn from this experiment?
There were a few interesting takeaways here.
First and foremost, be open-minded to testing anything, even unintuitive tests. This is especially true if your marketing funnel gets a lot of traffic or volume, as more traffic in your funnel means a smaller opportunity cost for each failed A/B test. In this batch of A/B tests, 7 Cups already had a large number of tests lined up. 7 Cups was contemplating removing the mobile download banner for UX purposes, so we (whimsically) decided to test it for SEO impact. We didn’t expect it to produce a win, let alone the largest win of the batch.
Second, we got more transparency behind Google’s algorithm. We have two hypotheses to explain 7 Cups’ improved position and increase in clicks. The first is that the header became shorter, which bumped the page’s core content ~100 pixels higher up the page. More relevant content was above the fold for the end user. The second hypothesis is that the header contained less anchor tags linking out to external sites, which Google might consider positive. Either or both of these hypotheses could be true, but we’d need further A/B testing to get more precise insights.
How is this useful for you?
Does this mean you should go to your codebase and strike out all markup for your mobile download banners? Probably not.
At a high level, this test reveals that simply adding more links and content to your landing pages could hurt your overall rankings in Google and stunt your organic search traffic. Ignore the traditional SEO checklist of best practices telling you to add more to your landing pages. More is not better. Quality over quantity.
At a tactical level, we would recommend you run your own SEO experiments to test your hypotheses before applying any of these changes to your own business. When you have data, you can be more confident in making changes to affect your marketing funnel and your top line growth.
→ Are you interested in seeing more a/b testing case studies?
RankScience is taking signups if you’d like to start split testing your SEO today! We split test SEO for everyone from startups to Fortune 500 companies.
If you’d like to receive more tips about SEO and A/B testing, please join our email list!
We are engineer-turned-SEOs, and we built RankScience to enable companies to run SEO experiments and iterate on their SEO pages faster. If there’s anything we can do to help, please email us. We’d love to chat!
