Menu
24.1% Revenue uplift
with Urgency/Scarcity triggers
on Product page
with Urgency/Scarcity triggers
on Product page
Brief information about A/B test
Shopify Customer:
Can't share the name of the website because of NDA
Hypothesis:
We expect that by adding scarcity and urgency triggers on above-the-fold part of Product page we will increase Revenue and Ecommerce Conversion Rate
Main Results:
1. Revenue increase by 24.1%
2. Ecommerce Conversion Rate increase by 21.27%
3. Add-to-Cart Rate increase by 11.26%
CRO Toolkit used:
1. Heuristic analysis
2. Behavioural psychology
3. A/B testing
Can't share the name of the website because of NDA
Hypothesis:
We expect that by adding scarcity and urgency triggers on above-the-fold part of Product page we will increase Revenue and Ecommerce Conversion Rate
Main Results:
1. Revenue increase by 24.1%
2. Ecommerce Conversion Rate increase by 21.27%
3. Add-to-Cart Rate increase by 11.26%
CRO Toolkit used:
1. Heuristic analysis
2. Behavioural psychology
3. A/B testing
Analysis and Hypothesis
It was the 15th A/B test on this client's Shopify Ecommerce Store (can't share the name of the website because of signed NDA).
We have already tested a lot of UX features, improved overall design of product pages, added elements that answered to main fears and concerns of potential customers. But it still didn't have any Scarcity and Urgency triggers that could stimulate visitors make purchase. That's why we started to think how we can implement it on product pages in the best possible way.
At first, we have analyzed how other stores (especially Shopify-based) are using Scarcity and Urgency triggers. After quick competitors' reviews, it was clear that almost all Shopify stores are using the same Shopify apps for that purposes.
On average it looks like this one or even more aggressive with a lot of similar pushy elements:
We have already tested a lot of UX features, improved overall design of product pages, added elements that answered to main fears and concerns of potential customers. But it still didn't have any Scarcity and Urgency triggers that could stimulate visitors make purchase. That's why we started to think how we can implement it on product pages in the best possible way.
At first, we have analyzed how other stores (especially Shopify-based) are using Scarcity and Urgency triggers. After quick competitors' reviews, it was clear that almost all Shopify stores are using the same Shopify apps for that purposes.
On average it looks like this one or even more aggressive with a lot of similar pushy elements:
It's common thing in CRO and in marketing at all that when something works well, all websites start to copy that (i.e. Booking.com social proof notification that pop up from each side of the screen, now you can see shitty version of this feature on every ecommerce store) and within a short period it becomes inefficient.
I think that the same logic applies to this function, because when visitors see that aggressive shitty countdowns, left-in-stock notifications, real-time visitors dashboards on every website with so prominent and aggressive colors, it doesn't build trust. It looks like a scam.
So based on these observations, our previous CRO experience and some inspiration from really successful Ecommerce stores who are doing it well, we decided to add just 2 unobtrusive scarcity and urgency features on testing version of product page:
1. "Today's Sale" badge on the top part of hero image above-the-fold.
2. "Availability: Low in Stock today" text below the thumbnail slider (also almost above-the-fold)
We placed these elements on this part of the page (above-the-fold) consciously to be confident that the vast majority of visitors will see it. And we expected that as we are driving traffic from Facebook ads with big discount offer, these urgency and scarcity triggers will be very relevant and will amplify each other in terms of psychological influence and we guessed that it will increase Revenue, Ecommerce Conversion Rate and Add-to-Cart Rate.
I think that the same logic applies to this function, because when visitors see that aggressive shitty countdowns, left-in-stock notifications, real-time visitors dashboards on every website with so prominent and aggressive colors, it doesn't build trust. It looks like a scam.
So based on these observations, our previous CRO experience and some inspiration from really successful Ecommerce stores who are doing it well, we decided to add just 2 unobtrusive scarcity and urgency features on testing version of product page:
1. "Today's Sale" badge on the top part of hero image above-the-fold.
2. "Availability: Low in Stock today" text below the thumbnail slider (also almost above-the-fold)
We placed these elements on this part of the page (above-the-fold) consciously to be confident that the vast majority of visitors will see it. And we expected that as we are driving traffic from Facebook ads with big discount offer, these urgency and scarcity triggers will be very relevant and will amplify each other in terms of psychological influence and we guessed that it will increase Revenue, Ecommerce Conversion Rate and Add-to-Cart Rate.
A/B testing
For A/B and Split testing we have used Visual Website Optimizer as it's one of the most powerful tool on the market and it can be easily integrated with Google Analytics for deep post-analysis.
In addition to it we are Certified Partners of VWO as we are actively using their platform and recommend it to all of our clients (btw, all our CRO customers have pricing discounts on VWO plans due to it)
We have used HTML/CSS and jQuery to build testing variation for this A/B test. It was done very fast, because we have a lot of experience in implementation different changes using jQuery as it's one of the main programming language that helps us to build experiments that cannot be created through basic WYSIWYG visual editor of A/B testing tools. That's why in the majority of A/B tests we don't even need any help from Development team.
Below you can see the difference between Control and Testing version of the product page in this A/B test:
Below you can see the difference between Control and Testing version of the product page in this A/B test:
When testing variation was prepared, it was thoroughly tested by our team using query parameters, so we were able to target A/B test only on ourselves and to check if everything was displayed and worked as needed.
Then A/B test was integrated with Google Analytics for deep post-test analysis and was started for all Mobile visitors that come to product pages.
Then A/B test was integrated with Google Analytics for deep post-test analysis and was started for all Mobile visitors that come to product pages.
A/B testing rules that we used:
- A/B test should run at least 2 Business Cycles (for us it's usually just 1 session as we have specific business model)
- A/B test should run at least 1 full week (from Monday to Sunday)
- A/B test should have at least 200 conversions per testing version
- Minimal statistical significance to declare fair winner should be at least 95%
Results
This test was live from February 22 to March 2 (9 full days) and we got the following results:
1. Revenue increased by 24.1% with 94% Statistical significance
2. Ecommerce Conversion Rate increased by 21.27% with 95% Statistical significance
3. Add-to-Cart rate increased by 11.26% with 99% Statistical significance
Technical details of the A/B test:
1. Revenue increased by 24.1% with 94% Statistical significance
2. Ecommerce Conversion Rate increased by 21.27% with 95% Statistical significance
3. Add-to-Cart rate increased by 11.26% with 99% Statistical significance
Technical details of the A/B test:
- A/B testing tool: Visual Website Optimizer integrated with Google Analytics
- Length: 9 days
- Targeting: Only Mobile users that, Only on Product pages
- Sample size: ~15000 visitors per each version
- Transactions: 216 in Control and 264 in Variaton
- Add-to-Cart conversions: 874 in Control and 980 in Variation
Ecommerce Data from Google Analytics report
Statistical Significance for Ecommerce Conversion Rate increase
Statistical Significance for Add-to-Cart Rate increase
Learnings and Next steps
Learnings:
Next steps:
- Urgency and Scarcity triggers have positive effect on our type of traffic (direct users who come from facebook ads to specific product page). We have to take into consideration that these triggers were not so pushy comparing with popular Shopify apps (like Hurrify) that looks scammy and can even decrease Conversion Rate.
- Even very small changes on the most visible part of the page (above-the-fold, image slider that gets the most engagement) can have positive impact on all steps of conversion funnel (add-to-cart rate, cart-to-checkout rate, checkout rate), so if we are going to test hypothesis that consist of adding some important elements on the page, we have to consider this place at first to increase number of people who see it on Mobile
Next steps:
- Redistribute 100% of traffic to winning variation while Dev team implements these changes to all product pages on live website
- Implement these changes on other websites of the client (need to decide if we are going to also A/B test this idea on another website)
- Think about how we can make these Scarcity and Urgency triggers more trustworthy and personalized to enhance positive effect of it.
- Think about repeating these Scarcity and Urgency triggers on Cart page to increase psychological pressure on potential customers.
