How Small Businesses Can Use &

A few years ago a book club began making phone calls to welcome its new members, just to make sure they understood the terms and conditions they?d signed up for. This program, all by itself, raised end-of-year renewal rates by more than 10 percent.

But how could the book club?s managers possibly know that this increase was actually due to their phone calling? After all, the increase in renewals might have been due simply to a better economy that year, or to a better overall selection of books.

Actually, the answer is very basic. The book club did a traditional test-against-control exercise, in which they made welcome calls to a small group of customers chosen randomly from among the entire population of new customers, most of whom did not receive the calls. The smaller group that received calls was the ?test? group, while the broader population was the ?control? group. Then they checked the end-of-year renewal rates for each group.?

Incorporating A/B testing into your business practices, something I call ?evidence-based management,? is becoming more common because the volume of data continues to grow and competition is driving companies to scour their operations for every last nickel of profit.

Even if you don?t have the kind of business that runs banner ads, or sends out direct mail or e-mail offers to customers, you can still improve your business results by practicing evidence-based management. Here are a few good examples:

• If you operate a retail store, and you are considering some kind of coupon or special offer to boost traffic, why not test it first? Pick every tenth customer at the cash register?or every hundredth, or whatever?and make them an offer, then track the results.
• If your sales force is considering a new tactic, test it first to figure out whether it makes sense to roll out to the entire field. (But remember to try to structure the test so that it is random, unbiased, and provides enough data to be confident in.)
• If you want to know by how much customer retention and referrals would increase when customers are reminded that their warranties are about to expire, test it first.

But in order to rely on the results of such statistical tests you need to be cognizant of three important issues:?

Randomness. One of the most critical aspects of any A/B test or test-and-control experiment has to do with ensuring that the selection of your test subjects is entirely random. For the book club, the simplest way to ensure as much randomness as possible would probably have been to make welcome calls to every tenth subscriber, or to every third one, or maybe every fiftieth ^{one. Not every promotion or policy change being considered will lend itself to an nth subscriber test, but you should always remain cognizant of the potential for bias.}

Confidence. You also need to be confident that whatever variation you observe in your test group is not just a random fluctuation. Whatever ?sample? of the universe you choose for the test has to be large enough so that most random variations in individual behavior will be damped out. Choose a big enough test group so that you would expect 100 or more responses from it.?

Roll-out. More often than not, the results you get when you roll out a newly tested idea to everyone will not be quite as good as it was for the test group. Just remember that the more pleasantly surprised you are by an A/B test, the less you should expect to be able to duplicate those exact results when you roll the test out to the broader population. This doesn?t mean you shouldn?t roll the program out, but evidence-based management requires you to remain cognizant of the role that randomness will always play in your business.

Source: http://www.openforum.com/articles/how-small-businesses-can-use-big-data

minka kelly James Holmes court Rupert Sanders bachelorette penn state Ernie Els Teen Choice Awards 2012