Where Have All the Control Groups Gone?

Where Have All the Control Groups Gone? | StrongMail Skip to Main Content What We OfferProductsServicesResourcesPartnersCompany US: (800) 971-0380    UK: +44 (0) 118 903 6068    APAC: +61 7 3666 0544

home | spark community | newsletter | blogs | support | contact

Resources OverviewBlogs- Email Marketing Insights- Monetizing Social Media- Maximizing Deliverability- IT Email InfrastructureOur CustomersWhitepapersNewsletterWeb EventsBest Email Service ProviderEmail GlossaryDelivery GuidelinesAuthentication Standards Improve Your Email Marketing

Subscribe to the StrongMail Advisor today:

" alt="Join >" /> We value your privacy. We will not rent or sell your email address.

Share This Page

Blogs #blogtitle {background: url(/res/blogs/mastering_social_crm/images/headerbg.jpg) 0 0 no-repeat !important;}#maincontent #blogtitle h2 {margin-left: -103px !important;} Email Marketing InsightsKara Trivunovic
Vice President of Agency ServicesJustin Williams
Senior Marketing StrategistJason Klein
Director of Marketing Communications Where Have All the Control Groups Gone? September 11, 2012 | By Kara TrivunovicTweet

Here's an article I wrote for ClickZ:

Email marketing is often the unsung hero of marketing portfolio success. Instead of praise and worship, email marketing gets a bad rap. Questions like "Is it dead?" "Does it work?" "Will it go on?" continue, everywhere…except among email marketers. Go figure, right?

What is comical is that it's common to find that the budgets of the naysayers, the exploration of new channels, is often funded by…revenue generated from email marketing. The biggest challenge that faces most email marketers (and marketers in general) is attribution. There is very little way to attribute the proper revenue consideration to email, mobile, social, apps - the list continues to grow. So I ask, "Where have all the control groups gone?"

In an age where the mindset around email marketing is quantity over quality, the idea of a "control" group has all but disappeared - which is unfortunate, because it's one of the single most telling methodologies to leverage in attribution modeling.

Where to Begin

First, you need to understand your entire marketing mix and map it out. There are going to be elements outside of your control, such as mass advertising opportunities that cannot be tracked to the individual, but it's important to understand that they exist and play a role in the communication of a message or brand to the customer. Categorize your efforts in groups of targeted and mass media to see what dials you have the opportunity to turn.

Finding the Audience

Once you have determined where you will be messaging, you need to then focus on the targeted efforts and begin adjusting the "who." It's imperative that you keep two things in mind. First, the "select" for the test must be random and representative of the entire mix. No segmenting geographically or taking an alphabetical data set and slicing it after every 10,000 customers - it needs to be truly random. Second, it needs to be statistically viable; each cell of the test needs to meet the requirements for a statistically viable sample size - often 10 percent works; depending on the size of your data set you may even be able to get away with a smaller percent - but better to be safe than sorry.

Determine the Mix

Now that you have the "who," you need to figure out exactly what they're going to get. It's suggested to execute these types of tests over a period of time, especially with attribution. A one-time proof does not a point make. But consistently exposing the same audience to a similar or like marketing mix will truly help you determine what impact each channel has on the likelihood to convert.

Let's take an example here. Say you have a database of 1 million subscribers that have proper permission to receive marketing messaging in all of your four available targetable channels: direct mail, email, SMS, and push-app notifications. The hypothesis is that email is the strongest contributor to the portfolio. To keep it manageable, break the audience into four categories: email only, email + direct mail, email + direct mail + SMS, and all four channels. This would allow you to measure the impact of adding the additional channels to communication flow. Clearly the messaging needs to be on point and the timing considerations are important - but now you have something to measure.

Be Diligent

As you get everything ready to go, be sure each audience is held out in its assigned category for the duration of the test and that each category has its own set of tracking tags and categorizations so that you can easily determine who is part of which data set. While tests like this can get complex to manage, your diligence and attention to detail through the process will certainly simplify matters.

It's important to remain the master of the test - there are likely going to be scenarios where someone wants to "just get a message out to everyone via all channels," but these exceptions will certainly corrupt your test, and ultimately the ability to determine attribution and lift by channels.

Posted by: Kara Trivunovic at 9:56 AM Categories: email, email marketing, email strategies, control groups Permalink Comments (0) 0 Comments Leave a comment This thread has been closed from taking new comments. Email Updates

Get blog updates by email

RSS Subscribe to this blog's feed Categories > 360-degree customer view> A/B split> Baazarvoice> Baby Center> Black Friday> CAN-SPAM> CMA> CMA Next 2010> CPA> CRM Archive > Closeness, Not Costumes: Building Successful Marketing Partnerships> The Email Marketer's Letter To Santa> Addressing The Consumer Demand For Mobile Experiences> Alternatives to drowning in "big data"> Taking Your Email on the Road> 3 Key Crosschannel Email Strategies for the Holidays> Is Big Data the New Email Marketing Catchphrase?> Is Online Chatter Predicting The Success (Or Failure) Of Your Email Campaigns?> 4 ways to waste fewer impressions> Living in Your Own Private Cloud: Better for Marketers?
View All Posts Blogs > Email Marketing Insights > Mastering Social CRM > IT Email Infrastructure > Product Talk > Maximizing Deliverability Blogger Bios > Kara Trivunovic> Tim McQuillen> David Atlas> Jai Williams> Brittany Landenberger> Amanda Hinkle> Jason Klein> Tal Nathan> Justin Williams> Sean Wirt> Dan Opallo What We OfferProductsServicesResourcesPartnersCompanyContactSupport

Copyright © 2012 StrongMail Systems, Inc. All Rights Reserved.Privacy PolicyTerms & ConditionsAcceptable Use Policy

View the original article here