Increasing Subscription Payments Through A/B Testing
ROLE UX Designer
TIMEL 2 Months
TEAM Product Manager
Stakeholders
TOOLS Figma
Crazy Egg
01. CONTEXT & CHALLENGE
Users weren’t converting at the rate the marketing team expected
Fortune is a Global media company known for its magazine and website on financial and business news. Marketing stakeholders came to our design team to express concern and curiosity about the paywall. The paywall's purpose was to block readers from continuing to read articles unless they signed up for a subscription.
02. BUSINESS GOAL
Identify the fallbacks and increase subscription conversion rates by 50%
I decided to run an A/B test because based on timeline and budget, we needed answers fast without disrupting the current paywall. A/B tests are the only scientific approach to make conclusions and the key component is randomization.
03. HYPOTHESIS
If we simplify the payment language, users will be more likely to pay
In previous research, we observed through a competitive analysis, that other paywalls had simplified payment language.
so with the previous research, we hypothesized if we simplify the payment language, users will be more likely to pay for a subscription.
03. HYPOTHESIS
If we clarify the CTA button, clicks will increase
We had user interview evidence that users felt frustrated because they didn't know what they were signing up for. So, the theory was that if we clarified the CTA button on the monthly subscription, clicks would increase.
04. TESTING FRAMEWORK
Decide on testing group and allocation
To validate if these ideas could work, we decided to run an A/B test. We used CrazyEgg for testing with a 3 week timeline. We randomly assigned our customers into the control and treament groups.
05. VARIANTS
Playing it cool while driving measureable impact
With A/B tests, it's important to isolate one design at a time against the control. We don't want to receive false test results. Based on our hypothesis, we decided to design two tests. One with the payment language removed and the second with a simpler CTA button.
05. RESULTS
30% new user adoption = revenue for the business & clear direction
After three weeks, the A/B test revealed that the CTA variant was the winner. It was evident that users clicked 3X more than the control. The results increased new-user subscriptions by 30%.
Next steps: Test the placement of the subscription paywall
