It’s testing week here at Direct to Donor and we’re going to start with some simple principles, as is our Monday pattern.  This is the first of many testing weeks, given the importance of the topic.

Unless it doesn’t work, in which case we might never speak of this again.

This is a great segue to the first rule of testing: that which works, works.  That which doesn’t work, doesn’t work.

I know, it sounds like I’m stating the obvious, but there’s an oft-forgotten conclusion from that, which is that if you aren’t willing either to roll out with it or to scrap it, you shouldn’t test it.

Two different much-beloved CEOs have expressed to me over the years that the single best predictor of whether a mail piece would succeed or not is whether they liked the piece.  If they liked it, it wouldn’t work; if they didn’t, it would.

Part of why they were and are much beloved is that it doesn’t matter whether they liked it or not – it was all about whether a tactic worked successfully.  The mantra that goes with this is:

Or, as I would put it, it doesn’t matter if the source if the quote is a damn dirty Commie if it’s a good quote.

That said, there are some things that are beyond what your organization will accept.  For example, an environmental organization shouldn’t use paper in their mailings from non-recycled old growth forests.

If that’s the case, then, don’t test it.  A goal of testing is to find something that will be able to use in some larger capacity in the future.  If that isn’t possible, it eliminates the need for the test.

That said, the list of sacred cows should be as small as possible.  You’ll hear me say test everything and I mean it – other than things that are untenable, experimentation is the best and truest way of learning.  Donor surveys are great, but they show what people think they would do and how they would react versus what they do do; it’s important not to confuse words and deeds.

There are things that are far more important to test than others.  That’s why it’s important to start with a hypothesis and to test the fundamentals first.  I love tests that nibble around the edges as much as the next person – if you can show me how to improve a piece with a different teaser and pick up 1.4% additional in response rate, I’m game.  But the most important tests that you run will be about fundamentals – who are the people that I should be talking to and what offer am I giving them.  Having a hypothesis will help with this and the broader it goes is better.  A hypothesis like “I believe our lapsed donors will respond best to the means and message that brought them into the organization initially” works well because it lends itself to testing on various platforms with a variety of tactics and a success could mean large things for the organization.  One like “I believe a larger envelope will work better” is restricted to one piece at one time and thus has limited ripple effect.

Finally, please learn from my mistakes and don’t test something by rolling out with it.  I did this in my first year of nonprofit marketing.  We had three underperforming mail pieces and I decided to replace them with new packages I had dreamt up.  Thankfully, I was lucky – the success of one of my pieces paid for the abject failure of the other two.  If I hadn’t been lucky, I might be blogging on effective panhandling tips right now.  You don’t want to put your nonprofit in a position where hitting goals and achieving mission is based on your hunch.

This may be a bit of conventional wisdom.  However, tomorrow, there is one piece of conventional testing wisdom that needs to be taken out back and shot for the benefit of your testing program.