A/B Testing SMS Messages: A Practical Guide

Small changes in wording, offer or timing can double the response to an SMS. A/B testing tells you which version works, instead of guessing.

ABvs

How it works

  1. Pick one thing to change.
  2. Split a random sample of your list into two groups.
  3. Send version A to one group and version B to the other, at the same time.
  4. Measure responses (codes, replies, clicks, sales).
  5. Send the winner to the rest of the list.

What to test

  • The offer: 20% off vs KES 500 off.
  • The opening: leading with the offer vs the customer’s name.
  • Urgency: “this weekend” vs “today only”.
  • Call to action: “Reply YES” vs “Call now”.
  • Timing: 10am vs 6pm — see best times to send SMS.
  • Sender ID: brand name vs branch name.

Sample size

Small tests give noisy results. As a rough guide, use at least a few hundred recipients per version, and more if response rates are low. If the difference between versions is tiny, treat it as a tie and test something bolder.

Example

  • A: “Weekend offer: 20% off all shoes at BRAND. Visit our Moi Ave branch. STOP to opt out”
  • B: “Grace, your 20% off shoes is ready — this weekend only at BRAND Moi Ave. STOP to opt out”

Give each version its own redemption code so you can count results.

Keep a testing log

Record each test, the result and what you learned. Over a few months you’ll build a playbook for your audience.

Related: measuring SMS ROI and the SMS marketing guide.

Get started

Create a Connect Media account — SMS from KES 1.0, no minimum top-up, credit that never expires, and one account for Safaricom, Airtel and Telkom. Call +254 707 339 945 or email info@connectmedia.co.ke.