In my DMA Leadership Conference talk, I said that people who listened to what donors say they want in donor surveys deserve to be lied to.  That was obviously too harsh – what I should have said is that they deserved to be misled.

Because people (not just donors, but all human beings*) aren’t meaning to lie to you; they just don’t know what their true motivation is.  As we’ve seen, emotional reaction happens 6000 times faster than rational thought.  So unless someone is doing System Two thought, where they are rationally considering all alternatives, the role reason plays in this process is coming up with the best possible justification of a decision already made.

Consider a study that asked people to rank their top 16 motivations.  Sex was rated #14; wealth was dead last.  Then they looked at actual subconscious motivators of decisions.  Sex was rated #1 and wealth was rated #5.

This should be considered no surprise to people who have met, well, ya know, people.  But it was a surprise to people themselves, who think themselves chaise and uncorruptable, but in reality dream of having very special moments in Scrooge McDuck’s vault.

But that doesn’t mean all donor surveys are bad – far from it.  It just means that, in a statement that may get me arrested by the Tautology Police**:

Bad donor surveys are bad.  Good donor surveys are good.

Common traps in donor surveys:

• Talking only to current donors. You want to talk to people who stopped giving as well, to the extent that they will talk to you.  After all, you are looking for the difference between these two groups.  Trying to define who your good donors are without talking to former donors is like saying the reason that Fortune 500 companies are successful is because they have employees and offices.
• Asking donors to analyze why they did what they did. They don’t know.  So they are going to try to figure out what answer someone like them would generally say or what they think you want to hear.  Neither is helpful to you.
• Asking donors what is most important to them. Clearly, from the above, the answer is sex.  Looking only at your limited options, however, they will probably make mistakes in determining what is important to them, similar to the poor people who thought that sex and wealth (aka Genie Wishes #1 and #2) didn’t impact them.

So how do you construct a survey that gets to these important points?  You are going to set up your survey so that you can run a regression analysis.****  If you need help with how to do this, check out our post on basic regression.

You will need a dependent variable.  Ideally, this will be donation behavior because it is a clear expression of the behavior you are trying to impact.  If not, an overall satisfaction score with the organization will be generally OK, as it should correlate strongly with donation behavior.

For your independent variables, ask about aspects of your organization.  So, for example, “have you ever called X Organization about your donations?”, “did you receive a thank you note for each donation you made?”, “have you been to X Web site”, “how many days did it take for you to get your thank you note on your last gift?”, etc.

The powerful thing about regression analysis is that it will help you figure out both how people feel about their experience and how important that experience is to them?  For example, my guess is that for most organizations, the number of days it took to get a thank you will be a good predictor of retention.  Since the analysis tells you the strength of that association, you can invest the right amount of resources into that area versus new donor welcome packages or donor relations staff or database infrastructure and the like.

* Yes, non-donors are also considered human beings – just slightly lesser ones.

** Motto: Enforcing through enforcement since Socrates.***

*** Former motto: Our motto is our motto.

**** Or other modeling if you are feeling fancy.