What is A/B Testing?

A/B testing, also known as split testing, is a method used in marketing, product development, and other fields to compare two versions of a web page, app, email, or other content to determine which one performs better. It's a controlled experiment where two variants, A and B, are presented to different segments of your audience. By measuring the response and behavior of users exposed to each variant, you can identify which version leads to more desirable outcomes, such as higher click-through rates, conversions, or engagement.

A/B Testing in Blueshift

Control Buckets

From Blueshift's Help Center:

When a customer is created in Blueshift, a user attribute called bsft_control_bucket is assigned to the customer. Each customer, identified or anonymous, is randomly assigned a control bucket number between 1 to 100.

You can use this user attribute to split up segments into smaller groups or buckets, to create random segments, or for splitting up customers for A/B testing.

Let’s say you would like to send out an email to users within a segment but you would like to send one version to 25% of the segment and another version to the remaining 75%. You can achieve this using segments or campaign filters:

Segments

In this example, you would use 2 segments:

• One that has bsft_control_bucket less than or equal to 25

• The second has bsft_control_bucket greater than 25

• Both segments would then use the same filtering criteria such as item_code = ABC, email_active = true, etc.

You would then need to create 2 different campaigns, one that uses the segment that targets 25% of the users with email version 1 and the other uses the segment that targets 75% of the users with email version 2.

GET SCREENSHOTS

Campaigns

In this example you, would need to 1 segment and 1 campaign with 2 triggers:

• One trigger has a filter for bsft_control_bucket less than or equal to 25

• The second has a filter for bsft_control_bucket greater than 25

You would then need to use email version 1 for the trigger that targets 25% of the users and then email version 2 for the trigger that targets 75% of the users.

GET SCREENSHOTS

A/B Split Feature