A/B Test Background: Coderwall
Coderwall is a community for developers to share development and design tips, tools, and projects. They wanted to take a data-driven approach to user acquisition via organic search.
From an SEO perspective, Coderwall was already pretty advanced, with a rich backlink profile and 130,000 pages of user-generated landing pages generating organic search traffic.
Most of Coderwall’s organic search traffic comes from their long-tail pro-tip pages. Users create pages to share programming tips with one another, such as JavaScript performance tips or Ruby on Rails tutorials. In fact, of their top 500 pages most visited by users from Google, 499 pages are pro-tips (the single exception was their homepage).
The key to Coderwall’s growth was earning more organic traffic on these pro-tip pages.
How Can We Get More Clicks via Organic Search?
On average, most of Coderwall’s pages in Google were ranking between the bottom of the first search engine results page (SERP) or the top of the second SERP.
Landing anywhere on page 1 of a Google search is obviously great for maximizing your search impressions, but being on the bottom of the first page generally earns only a small percentage of clicks. Low clicks and high impressions translate to a poor clickthrough rate when you’re at the bottom of the SERP.
The top three search results get 63% of all clicks in search engines. The bottom three results (#7-10) get only 9% in total, even though they’re still on the first page. (Note: Studies show that only 10–15% of clicks go to Adwords links, even though Google shows more ads on the SERPs these days.)
Click through ratesThere are two ways to increase organic search traffic: 1) improve CTR on your pages without improving their position on the SERP, and 2) increase your positions on the SERP. Your position on the SERP is determined by Google’s search algorithms. Many companies are afraid of rolling out changes to pages that already rank and get traffic from Google, fearing that they might displease Google’s search algorithms, hurt their position on the SERP, and break something that’s working fine.
The titles of search results are the clickable blue links, and titles that are more persuasive generate more clicks. Our hypothesis was that a simple <title> tag change could increase CTR and fatten the top of the marketing funnel, extracting more clicks from Coderwall’s existing positions in Google.
Building the A/B test for SEO
When most people think of A/B testing, they think of conversion rate optimization (CRO) and tools like Optimizely. Optimizely and other A/B testing tools for CRO are meant to fatten the middle of your marketing funnel. Since Coderwall wanted to fatten the top of their funnel instead, they needed different tools.
We built RankScience as a CDN proxy to implement A/B testing experiments for SEO for brands like Coderwall. After we helped Coderwall run a few A/B tests, the biggest win for Coderwall was a <title> tag change.
The original title format was just “{Post of title} | Coderwall”. The proposed new title format was “{Post of title} (Example) | Coderwall”.
Roughly 20,000 HTML pages were split into two groups, the control group and the variant group. The control group’s title had the original format, and the variant group’s title had the new format.
We measured the two group’s performances before implementing the new format for the variant group. After the new format was deployed to the variant group pages, we waited three weeks to gather new data.
Building the A/B test for SEO
While most of Coderwall’s A/B tests failed to produce a significant uplift, the title tag experiment was a huge win. The number of users clicking on Coderwall from Google increased 14.8% (yes, this was statistically significant).
With data at their side, Coderwall felt confident deploying this change to all of their pro-tip pages. As expected, the new title format continued to grow their search traffic over time, even months after concluding the experiment.
To date, Coderwall’s increase in traffic compounded to an aggregate 59% from this experiment alone.
What Did We Learn From This Experiment?
We learned that users who wanted Coderwall’s pro-tip content wanted tactile information that was easy to digest and immediately applicable to their own work.
These extra details exposed by “(Example)” persuaded users to click on Coderwall links instead of others on the SERP. Over time, Google noticed the CTR for Coderwall pages going up, presumed users were finding quality results for what they wanted, and rewarded Coderwall with improved position and impressions, growing 8% month over month.
How Is This Useful for You?
Does this mean you should run out and try appending “(Example)” to the title of all your pages? Probably not.
This test came about in the context of several other concurrent A/B tests. Analyzing all the test results exposed key learnings about Coderwall’s users.
To apply this to your business, you should be running your own SEO experiments to test your hypotheses and learn more about your users’ intentions. This works for Coderwall because it aligns with their user intent. The tests that will work for you will align with your own users’ intent.
Are You Interested in Seeing More A/B Testing Case Studies?
RankScience is taking signups if you’d like to start split testing your SEO today! We split test SEO for everyone from startups to Fortune 500 companies.
Get a Free SEO Consultation
Rank Science Inc.
2443 Fillmore St #380-1937
San Francisco, CA 94115
(239) 555-0108
