MarketingProfs’ “Get to the Po!nt” email marketing newsletter highlighted a recent edition of “Which Test Won.” If you’re not familiar with this online resource, WhichTestWon.com is an independent publication with the goal of evangelizing best practices in marketing optimization — in particular through A/B and multivariate testing. If you are familiar with this site and the “How Good is Your Gut?” tests they run, you know the rush of correctly answering each question — finding out that your gut is pretty darn good.
In this example, an A/B test (employed by DIYthemes) invited visitors to sign up for email newsletters. Version A used the headline “Get Email Updates (it’s free!),” and used social proof messaging to encourage registration: “Join 14,752 others and get free updates.” Below this was a box for an email address and a “join” button. Version B was identical, except that it omitted the line about 14,752 subscribers.
Voters at the site overwhelmingly chose Version A: 82% to 18%. But the majority was — in this case — wrong. “Version B, without the social proof messaging, got a 122% lift in email opt-ins,” noted Anne Holland, (@AnneHolland55) publisher of WhichTestWon.com.
“Derek Halpern, Chief Persuasion Officer at DIYthemes, suggested that ‘joining 14,752 others’ just wasn’t compelling enough, or might have distracted visitors from the submit button below,” said Holland. “But we think another factor might be that the messaging didn’t clearly explain what kind of email updates the subscriber would receive — or the benefits of opting in.”
MarketingProfs went on to say that the results sparked a lively discussion in the comment section. Comments included thoughts such as:
- “Maybe the ‘joining’ messaging misled people into believing they were signing up for a discussion list as opposed to the blog’s own updates?”
- “I think with the phrase ‘joining 14,752 others’ it seems to suggest that the sole reason to join is because 14,752 have joined.”
- “My own testing has shown that you should leave the reasons for joining to the Welcome Email. Every single extra character is one more reason to delay entering your email address and hitting Submit.”
And, as the marketing resource wraps each case study, they point out what we know to be true — and benefit from with the review of such a wildly unintuitive example. Even when the answer seems obvious, and you know in your gut you’ve got it, test and test and test again.