Menu
24.3% Revenue uplift
with Urgency/Scarcity elements
on Cart page
with Urgency/Scarcity elements
on Cart page
Brief information about A/B test
Shopify Customer:
Can't share the name of the website because of NDA
Hypothesis:
We expect that by adding Scarcity and Urgency elements on Cart page we will increase Revenue and Ecommerce Conversion Rate.
This hypothesis was already proved in previous A/B test on Cart page, so we are going to enhance its effect by placing additional Scarcity and Urgency elements on the next step of the funnel
Main Results:
1. Revenue increase by 24.3%
2. Revenue per Visitor increase by 25.7%
3. Ecommerce Conversion Rate increase by 21.37%
4. Ecommerce Conversion Rate increase only for New Visitors by 23.13%
CRO Toolkit used:
1. Heuristic analysis
2. Behavioural psychology
3. A/B testing
Can't share the name of the website because of NDA
Hypothesis:
We expect that by adding Scarcity and Urgency elements on Cart page we will increase Revenue and Ecommerce Conversion Rate.
This hypothesis was already proved in previous A/B test on Cart page, so we are going to enhance its effect by placing additional Scarcity and Urgency elements on the next step of the funnel
Main Results:
1. Revenue increase by 24.3%
2. Revenue per Visitor increase by 25.7%
3. Ecommerce Conversion Rate increase by 21.37%
4. Ecommerce Conversion Rate increase only for New Visitors by 23.13%
CRO Toolkit used:
1. Heuristic analysis
2. Behavioural psychology
3. A/B testing
Analysis and Hypothesis
It was the 17th A/B test on this client's Shopify Ecommerce Store (can't share the name of the website because of signed NDA).
On the 15th A/B test we have increased Revenue by 24.1% by adding Scarcity and Urgency elements on Product page.
On the 15th A/B test we have increased Revenue by 24.1% by adding Scarcity and Urgency elements on Product page.
This result was 95% statistically significant and we decided to go further with this idea and additionally implement similar elements on the next step of conversion funnel - Cart page.
We decided to follow the same design styles for these elements to maintain consistency on both pages. As a result, we added to elements on Cart page
1. "Low in Stock Today" badge inside of product information
2. "Friendly Reminder: Don't miss TODAY'S SALE as you can SAVE 50% OFF entire order " text above CTA-button "Checkout"
We placed these elements on this part of the page consciously to be confident that the visitors will take attention to it. And we expected that as we have already notified users about discount and low stock of these products on product pages, it will enhance its effect and increase Revenue and Ecommerce Conversion Rate even more.
We decided to follow the same design styles for these elements to maintain consistency on both pages. As a result, we added to elements on Cart page
1. "Low in Stock Today" badge inside of product information
2. "Friendly Reminder: Don't miss TODAY'S SALE as you can SAVE 50% OFF entire order " text above CTA-button "Checkout"
We placed these elements on this part of the page consciously to be confident that the visitors will take attention to it. And we expected that as we have already notified users about discount and low stock of these products on product pages, it will enhance its effect and increase Revenue and Ecommerce Conversion Rate even more.
A/B testing
For A/B and Split testing we have used Visual Website Optimizer as it's one of the most powerful tool on the market and it can be easily integrated with Google Analytics for deep post-analysis.
In addition to it we are Certified Partners of VWO as we are actively using their platform and recommend it to all of our clients (btw, all our CRO customers have pricing discounts on VWO plans due to it)
We have used HTML/CSS and jQuery to build testing variation for this A/B test. It was done very fast, because we have a lot of experience in implementation different changes using jQuery as it's one of the main programming language that helps us to build experiments that cannot be created through basic WYSIWYG visual editor of A/B testing tools. That's why in the majority of A/B tests we don't even need any help from Development team.
Below you can see the difference between Control and Testing version of the product page in this A/B test:
Below you can see the difference between Control and Testing version of the product page in this A/B test:
As you see, there are 2 elements which are pretty prominent among other elements (that's why we decided to use red color to get user attention) and they are placed above CTA-button to ensure that when visitors will see it when scan the page from top to the bottom.
In addition to this drastic changes, we have slightly improved overall UX of the page by changing CSS styles of secondary elements (Product title, product info, secondary buttons to remove and update cart, font-size of other elements)
When testing variation was prepared, it was thoroughly tested by our team using query parameters, so we were able to target A/B test only on ourselves and to check if everything was displayed and worked as needed.
Then A/B test was integrated with Google Analytics for deep post-test analysis and was started for all Mobile visitors that reach Cart page
In addition to this drastic changes, we have slightly improved overall UX of the page by changing CSS styles of secondary elements (Product title, product info, secondary buttons to remove and update cart, font-size of other elements)
When testing variation was prepared, it was thoroughly tested by our team using query parameters, so we were able to target A/B test only on ourselves and to check if everything was displayed and worked as needed.
Then A/B test was integrated with Google Analytics for deep post-test analysis and was started for all Mobile visitors that reach Cart page
A/B testing rules that we used:
- A/B test should run at least 2 Business Cycles (for us it's usually just 1 session as we have specific drop-shipping business model with "hot" traffic from Facebook Ads)
- A/B test should run at least 1 full week (from Monday to Sunday)
- A/B test should have at least 200 conversions per testing version
- Minimal statistical significance to declare fair winner should be at least 95%
Results
This test was live from March 5, 2018 to March 12, 2018 (8 full days) and we got the following results:
1. Revenue increased by 24.3% with 95% Statistical significance
2. Revenue per Visitor increased by 25.7% with 95% statistical signfiicance
3. Ecommerce Conversion Rate increased by 21.37% with 99% Statistical significance (Among all mobile visitors)
4. Ecommerce Conversion Rate increased by 23.13% with 99% Statistical significance (Among only New Mobile visitors)
Technical details of the A/B test:
1. Revenue increased by 24.3% with 95% Statistical significance
2. Revenue per Visitor increased by 25.7% with 95% statistical signfiicance
3. Ecommerce Conversion Rate increased by 21.37% with 99% Statistical significance (Among all mobile visitors)
4. Ecommerce Conversion Rate increased by 23.13% with 99% Statistical significance (Among only New Mobile visitors)
Technical details of the A/B test:
- A/B testing tool: Visual Website Optimizer integrated with Google Analytics
- Length: 8 days
- Targeting: Only Mobile users that reached Cart page
- Sample size: ~1850 visitors per each version
- Transactions: 326 in Control and 391 in Variaton
Conversion Rate data from VWO dashboard
Ecommerce Data from Google Analytics report
Statistical Significance for Ecommerce Conversion Rate increase among all Mobile visitors
Statistical Significance for Ecommerce Conversion Rate increase among only New Mobile Visitors
Learnings and Next steps
Learnings:
Next steps:
- In our business model (Dropshipping business that is focused on one-time customers, not building retention and loyal target audience) Scarcity and Urgency elements both on Product and Cart pages can significantly influence on purchasing behaviour.
- We again observe that when we implement A/B test on Cart page, it impact not only CR on this particular step of the funnel, but also on the next steps of the funnel like in this A/B test with CR from Checkout started to Checkout finished. So it means that people on Checkout took into consideration the information about 50% Sale and Low in Stock availability that we introduced to them on the Cart page. It again proves that we have to always measure final CR in each A/B test and don't use parallel A/B testing as it can significantly skew results.
Next steps:
- Redistribute 100% of traffic to winning variation
- Create task in Trello for Dev Team to implement changes from winning variation to Live website
- Double-down on adding slicky and contextual Scarcity and Urgency elements on Product and Cart pages as I still see a lot of potential in this field
- A/B test this idea on another store with Baby products as it's completely different niches and we can't apply all ideas without A/B testing
