Website Widget A/B Testing by Linda Bustos

Website Widget A/B Testing by Linda Bustos

29 March 28 March ~ 10 min read 1368 views
rate it
Claspo Blog Website Widget A/B Testing by Linda Bustos

With all the design and customization options for the widgets available nowadays, it’s easy to get overwhelmed by the possibilities. While there are many tools to A/B test any aspect of your widget to prove what performs best, it’s important to test the right variables first to avoid endless rounds of trivial tests.

Because what rockets conversions for one site might fail on another, you want to determine what design and delivery decisions work best with your customer.

I'm Linda Bustos from Ecom Ideas, I’ve been an ecom consultant and blogger covering the ecommerce marketing and UX space since 2007. I want to share a practical playbook based on usability and conversion best practices that addresses the critical design decisions and widget settings first to maximize your testing data.

Testing stage 1: Find the best performing Layout

In this stage, you want to figure out the best way to serve your widget, so we’ll only be testing layout settings against each other. Remember the golden rule of A/B testing: one change at a time.

Within Claspo’s Builder, you have the choice of widget layouts, including:

  • Popup
  • Floating Box
  • Floating Bar
  • Built-in Widget
  • Content-blocking


Right off the bat, let's sideline the built-in and content-blocking layouts. They've got their uses—built-in is a good choice for blogs or landing pages, and content blockers are primarily used to verify age before accessing any website adult content. That leaves us with the pop-up, floating box, and floating bar as the contenders.

Here's my insider tip:

throw pop-up and floating box into the ring for round one. They’re the more common (and visible) patterns compared to Floating Bar. (You can always return to test Floating Bar in later stages, after you work through this playbook!)

Style your widget as you like, but make sure any colors, fonts, fields, copy and settings are consistent across all versions. You want to control your test results for a single variable – the Layout.


Then, head over to the dashboard and make a copy of your pop-up draft.


Jump into the editor and switch the layout to a floating box. It's just a click away in the top right corner.


Save your second draft, and boom, your floating box is also ready to roll.

Then go to the A/B testing tab, and:

  • Click New test and select a Widget A and B
  • Give your test a title and description
  • Publish both widgets to kick off your test


Let it run till you reach at least 90% statistical confidence level. And when you have a champ, you can re-run the winner in a new test against the Floating Bar, or move on to trigger timing.


Keep a record of your experiments and outcomes to track progress and provide visibility to team members.

Once you’ve found your ultimate Layout champion, move to stage 2: trigger timing.

Testing stage 2: Trigger timing

Form submission rate and growing your email list means nothing if your trigger causes more visitors to bounce too soon. You want to reward visitors, not scare them away.

You certainly can choose your testing options based on average time spent on the website or a specific page. But because email offers are “welcome” perks, I recommend first gauging the impact of an immediate offer against a 5 second delay. Beyond 5 seconds, your customer is likely engaged in a task – scrolling, searching or digesting your amazing merchandising and messages.

For this next test:

  • Grab the widget with the winning layout
  • Find the "Triggering section" and set the "When to display" to "Immediately"
  • Save, then clone this widget, replacing "Immediately" with "Spent on the page 5 seconds" in the copy



Then, let it rip! Once this test concludes, it’s time to test a slightly different trigger condition:

  • Take your timing champ widget and make a copy
  • In this clone, change "When to display" rules from timed trigger to "Read the page by [20%]" (or whatever makes most sense based on your scroll-depth insights – shoot for something over 80% of your visitors are likely to see)


Continuing to test various trigger conditions is a great idea, but it’s also important to determine the best offer for conversion. For this reason, we’ll take the timing test winner and move to testing your Offer.

Testing stage 3: Offer

Your incentive to join your list – be it a discount, free shipping, gift with purchase, chance to win or the trending “mystery offer” – has to impress your customers or they won’t convert. Period!

Use this round to test such offers against each other – e.g. discount A vs B, discount against free shipping, or discount against mystery offer. If you’re not able to test more than one offer or discount level due to margin constraints or other business reasons, use this round to test your headline and CTA (call to action button) copy.

Some of my faves include:

“Get X% off” vs “X% off your first order” (determine whether a shorter, action oriented headline outperforms longer, clarity-focused copy)

“Enjoy X% off” vs “X% off” (does “enjoyment” persuade more than a short and concise?)

“Sign up for $X off” vs “X% off your first order” (which call to action - sign up or purchase - persuades more?)

“Unlock X%” vs “Save X%” (sophisticated vs cheap – what motivates your customer?)

“Want X% off?” vs “Get X% off” (asking vs telling)

Your offer is the most influential component of your email pop-up, but you need to make sure you’re serving it at the right time, with the right layout. 

Follow the same A/B testing setup steps, and focus this round on finding a killer offer that can anchor the rest of your experimentation rounds. Your offer is the most influential component of your email pop-up, but you need to make sure you’re serving it at the right time, with the right layout. 

Now that you’ve nailed down the core basics, the rest of your testing program can get as creative as you want!

Testing stage 4 and beyond: Everything else

If you’ve made it this far, you’ve tested enough variables to have a solid Control version that you can test against all kinds of creative treatments, including:

🧪 Widget design (colors, fonts, graphics)

🧪 Copy and microcopy

🧪 Privacy policy visible vs no

🧪 Countdown timers

🧪 Form field shape and styling

🧪 Email hint text

🧪 With or without logo/branding

I’ve compiled some of my favorite testing variables and live merchant examples on Ecom Ideas for your inspiration.

Keep iterative testing (one variable at a time) or try “radical” redesigns for bigger lifts. Just keep in mind, when you test multiple variables together, you can’t determine which is responsible for the uplift. Testing is both art and science.

Final Thoughts

With A/B testing, you've got endless ways to optimize your email capture widget. From picking the best layout to pinpointing the perfect messaging, every step is a chance to boost your conversion rate and other key performance metrics. 

Remember, A/B testing is not just about making changes; it's about making informed decisions that drive your strategy forward. Don’t be afraid of testing out bold ideas and tailor your widgets to resonate with your audience and amplify your outcomes. 

Happy testing!

Still Not a User Of Claspo?

You Might Be Interested in
8 Spring Marketing Ideas
8 Spring Marketing Ideas

Are you looking for spring marketing ideas to catch the attention of your customers and make your brand recognizable? We have some exciting suggestions that are suitable for any business. So, let's get ready for the season of beauty and renewal! Spring Marketing Campaign: Why Is It Essential for Your Business Spring is not just the first warm days and new leaves on trees. Along with the weather, consumers' moods improve. They become more susceptible to emotional impulse spending and...

27 March 12 min read
What is Sustainable Marketing: Definition and Strategies with Examples
What is Sustainable Marketing: Definition and Strategies with Examples

Sustainable marketing is one of the key trends in the 21st century. Governments and non-profit organizations are paying more attention to environmental protection and preserving the planet’s resources, and businesses cannot stand aside. This is an excellent opportunity for businesses to contribute to solving global problems while improving their reputation and achieving other commercial goals.  Keep reading to learn what sustainable marketing is, its advantages, and ways to use it. What is Sustainable Marketing Sustainable marketing is the promotion of...

25 March 12 min read
Top 10 Easter Pop-ups from Claspo
Top 10 Easter Pop-ups from Claspo

Easter is a time of cherished traditions, prominently featuring the joy of giving gifts to loved ones. In 2024, spending for the Easter holiday is expected to reach $22.4 billion, with a third of shoppers turning to online stores. So, if you have special offers, don't tuck them away like hidden Easter eggs. Instead, showcase them front and center on your website, enticing shoppers to choose your store for their gift-hunting adventure.  How do you make your offers stand out?...

21 March 12 min read
Digital Customer Engagement: Guide with Examples
Digital Customer Engagement: Guide with Examples

Digital customer engagement is the key to online business success. The "see, buy, forget" formula is in the distant past. Today, most companies try to build long-term relationships with consumers. Why do they want to do this, and what methods do they use to achieve this goal? Keep reading to find out. What is Digital Customer Engagement The meaning of digital customer engagement is creating a strong connection between a business and its target audience. It aims to build the...

29 March 10 min read
10 Best AI Marketing Tools
10 Best AI Marketing Tools

According to IBM, 35% of big businesses have already integrated AI marketing tools into their business processes, and another 30% are experimenting with them and assessing their effectiveness. Those already leveraging these new technologies are saving at least 30% of their working time; this figure reaches an impressive 50-75% in some industries!  Now, imagine how artificial intelligence will affect your business. It will give you an impetus for scaling, reducing costs, and dealing with weak points where previously you relied...

01 April 15 min read
7 Best AI Chatbots in 2024
7 Best AI Chatbots in 2024

The best AI chatbots in the world irrevocably change the e-commerce market, increasing business efficiency and facilitating its scaling. ChatInsight statistics show that at the beginning of 2024, almost 91% of US online stores are using intelligent chatbots that can handle more than 70% of customer requests on their own. On a global scale, this means saving an incredible 2.5 billion working hours per year! So, the arguments in favor of the new technology are undeniable. All that remains is...

02 April 15 min read