Does list segmentation really help email marketing stats?
It’s common knowledge that segmenting your email marketing lists can help you get better open and click rates. The idea is that by narrowing your focus and sending messages to targeted groups within your lists, your recipients will find it more relevant, which will get you better results.
Really? We scanned our system for MailChimp users who have used our list segmentation feature. We found 1,988 users who sent 10,961 segmented campaigns to 8,762,207 recipients. We compared the email marketing results of those users’ segmented campaigns to the results of their non-segmented campaigns. We actually uncovered some counter-intuitive results in a few places.
Global Results:
When we measured stats “across all segmented campaigns” segmented campaigns definitely helped email marketing performance almost across the board, except for one metric: unsubscribes. Why would more people unsubscribe from targeted messages that are supposedly more relevant to them?
| Opens: | 14.444% better than the list average |
|---|---|
| Clicks: | 14.994% better than the list average |
| Bounces: | 0.803% better than the list average |
| Abuse Reports: | 0.010% better than the list average |
| Unsubs: | 0.632% worse than the list average |
Segment by: Merge Field
When MailChimp users segmented their lists based on some field in their recipient database (examples might include “customer_type” or “ZIP code” or “job_title”) results went way up compared to non-segmented lists. This turned out to be the most popular way to segment lists.
| Opens: | 18.852% better than the list average |
|---|---|
| Clicks: | 21.976% better than the list average |
| Bounces: | 1.429% better than the list average |
| Abuse Reports: | 0.015% better than the list average |
| Unsubs: | 0.986% worse than the list average |
| Total campaigns of this type: 6,040 | |
Segment by: Signup Date
In MailChimp, we have a feature that lets you segment your lists based on “those who signed up within the last x days” or “those who signed up since I sent my last campaign.” One common use for this segment is to send a campaign to your most recent subscribers.
| Opens: | 11.641% better than the list average |
|---|---|
| Clicks: | 10.501% better than the list average |
| Bounces: | 0.492% worse than the list average |
| Abuse Reports: | 0.002% better than the list average |
| Unsubs: | 0.126% worse than the list average |
| Total campaigns of this type: 2,761 | |
Segment by: Interest Groups
Email marketers can create signup forms with checkboxes so that subscribers can indicate their interests. For example, a music website might have an email signup form with options for favorite musical genre. When segmenting campaigns based on these interest groups, MailChimp users got slightly better results than average.
| Opens: | 1.659% better than the list average |
|---|---|
| Clicks: | 1.712% better than the list average |
| Bounces: | 0.206% better than the list average |
| Abuse Reports: | 0.002% better than the list average |
| Unsubs: | 0.071% worse than the list average |
| Total campaigns of this type: 851 | |
Segment by: AIM reports
Here’s the shocker. MailChimp offers an add-on module called AIM Reports that allows our users to track user-specific stats in their email campaigns. For example, instead of just seeing your campaign’s total number of opens and clicks, this add-on allows you to drill down and see who opened and clicked. This allows for some extremely focused segmentation, which one might think would improve one’s results dramatically. For example, one can use this to segment a list based on “those who opened a previous campaign” or “those who opened my 3 most recent campaigns” or “those who did not open my last campaign.” Extremely powerful segmentation. But it turns out almost all metrics went down when users segmented by “AIM Reports.”
| Opens: | 2.072% worse than the list average |
|---|---|
| Clicks: | 2.524% worse than the list average |
| Bounces: | 1.842% better than the list average |
| Abuse Reports: | 0.003% worse than the list average |
| Unsubs: | 0.262% better than the list average |
| Total campaigns of this type: 297 | |
Observations and Followups
For the most part, segmenting your email marketing lists can help improve your open and click rates, and will reduce the number of bounces from each campaign you send. We’re baffled as to why more people would unsubscribe from seemingly more relevant campaigns. We have some theories, such as “perhaps the segmented campaigns were sent in addition to normal batch-and-blast campaigns, which resulted in annoying duplicate messages.” We’ll save that for another study.
Most disturbing was the discovery that AIM Reports generated worse results than non-segmented campaigns. We had to look more closely at what was causing this, because the AIM Reports add-on is so awesomely cool, there’s no way it can do harm (scientifically speaking, that is). Turns out the majority of users who had AIM Reports installed were not using it to send special emails to loyal subscribers (“segment based on those who opened my recent 3 campaigns”) but were using it to send follow-up campaigns to “those who did not open my last message.” This has been documented on email marketing sites as an extremely effective tactic to generate more bookings by hotels and event organizers (Marketingsherpa: Should You Re-Send Your Email Newsletter to Non-Openers?). But when you factor in how inherently inaccurate open rate tracking is, it’s understandable that some of these followup campaigns are perceived as pesky duplicates to some recipients.
