CBA Test: Bigger is Better

Bigger is Better
A Guide to Beating Your Control

By Brooke Grossman Coneys

There’s no resting on your laurels in direct marketing, and for good reason. Who can predict when the bottom will fall out of your beloved, dependable control that has served you faithfully for years. You've got to be ready with another winner when that time comes. Will you enjoy a long run like the Nature Conservancy’s worried ostrich? Or will you be scrambling to meet your numbers after just a few runs with a hot new package?

Not only do you have to compete with yourself, you also have to keep winning the mailbox competition for donors' dollars. The route to success is to keep testing new and innovative creative approaches, formats, incentives, offers, etc.

We would like to share with you a successful test outcome which beat the control and resulted in two alternating control packages -- which ensures continued success in meeting revenue goals (at least for the foreseeable future).

The Control

"Automatic Enrollment" has been San Francisco Public TV Station KQED's new member acquisition control package since November 1996. In its first incarnation, it was created as a standard #10 package that soundly beat the control at the time.

In almost every campaign since, (there are four per year), this control has been tested against different creative approaches, in different formats and against other formats.

The Breakthrough

Test #1 – A home run

For several clients the winning Personal Pak™ format -- a highly personalized in-line package in a snap-out closed-faced 6" x 11" envelope -- beat the standard #10 (4 1/8" x 9 1/2" window format) by a range of 20% to 40% with high ROIs.

In May 2002, we tried to get more mileage from the new control. The Automatic Enrollment #10 was tested against Automatic Enrollment #11, another break-through format delivered in a double window 4 1/2" x 10 3/8" envelope with a Temporary Membership Card showing through. Both packages had preprinted "Dear Friend" letters accompanied by a personalized reply form. And the result was...the #11 out pulled the #10 by 18% and improved ROI even more.

Test #2

Format testing continued in November 2002 with a #11 package tested against the Personal Pak™ format, the recent new control. The new #11 package has a personalized reply form.

Preliminary results show the #11 running 18% ahead of the Personal Pak™, with the ROI still climbing.

Test #3

Another test panel in November included a station logo decal in a #11 package. During the first few weeks of response, the decal panel was running about 12% ahead, but after one month, the decal and no-decal panels were even, so the decal did not make an improvement in response.

It appears that when a "freemium" is used (something given away free in the mailing), the people who appreciate it most respond the earliest. Another useful lesson: "don't count your chickens before they hatch".

Conclusions

Format, size and personalization can make a tremendous difference in response. Bigger appears to be better, especially addressed to an individual with personalized letter copy and on the response form. The #10 package was beaten by both the #11 and Personal Pak™ formats.

In a limited geographical area, prospects often need a "change of scenery," which means control packages can wear out from over use. A change in package can often regenerate lagging performance, but it still has to be a clear winner.

Index of Articles