Nothing could be less controversial to a fundraiser than the idea that it is important to thank your donors. In fact, most people in the development world could rattle off a personal story or two about an organization that failed to do just that, and the consequences when they never heard from that big donor again.

But there’s now an interesting new study that throws a wrench in that picture. Anya Samek, a professor of economics at USC, and Chuck Longfield of Blackbaud teamed up with a number of nonprofits to make thank you phone calls to new donors. You can read the long academic version here or Prof. Samek’s brief summary here.

The finding is frankly shocking: they completed about half a million thank you phone calls, tracking results over years. And they found that compared to a control group that received no thank you call, the thank you call made no difference whatsoever: No increase in giving. No greater likelihood to give again.

And to embarrass the fundraising profession even further, the study also went to the trouble of surveying fundraising professionals and the general public beforehand about what effect they thought the calls would have on retention rates.

Not only were the nonprofit professionals totally wrong about the effect of the calls, they were no better at predicting the actual effect than members of the general public.

Even the researchers seem to have been stunned by the initial findings, so they went back to their call script. Maybe, they thought, they’d have better results with a script that asked for input from the donor. Maybe with a request for feedback and an opportunity to share their opinion with the nonprofit, the donor would feel a greater connection and be more likely to give again.

The surveyed fundraisers and the general public both thought that that change would increase retention, but those calls too showed no effect whatsoever.

Is it time to panic? To tell your staff never to touch a phone again? To set fire to the call centers and put all your chips on corporate giving instead?

Not quite. While the finding that thank you calls makes no change to future giving is troubling, even disturbing, there are a number of caveats (mentioned by the authors of the study, to their credit) to mention.

First, the calls were all to first time donors, and the timing of the call lagged significantly behind when the gift was made. The authors say that most calls were made between 3-7 months after the donation was made. One might say that it’s never too late to be grateful, but at the same time it’s obvious that any effect of acknowledging the gift would decline rapidly over time.

A call within 24 hours of the gift? I might be impressed. Within a week, I’m still encouraged that the nonprofit is on top of things, and that I matter to the organization. But 6 months later? It’s increasingly obvious to many recipients at that point that a random call center volunteer or employee is just cycling through a gigantic list, and that the true purpose of the call isn’t really to thank the donor, but to remind the donor about the organization in the hopes of securing a future donation. For some people, that extended delay might even backfire—that is, it could reveal to the donor just how little their donation matters to the mission.

I still remember a former client of mine at a smaller nonprofit. The chief executive hated the phone and would do anything to avoid calling his donors. But he was forced to one day when he received an unexpected $10,000 gift. When the donor picked up the phone, the first thing he said was “I wondered if I’d be getting a call from you today!” He expected the call from the executive as a common courtesy, because he knew the executive personally, and because he knew his gift to the organization was very significant to their mission. An unexpected call six months later has just the opposite effect: it signals that the donor is not remembered, is not appreciated, and that their donation is not significant.

And this brings me to the second major issue with the study: who is making the calls. By virtue of the size of the experiment, call workers unconnected to the nonprofit in any substantive way were employed to complete the calls. It seems logical that donors would react differently and more positively to calls from nonprofit staff, board members, or beneficiaries (such as when students who have benefitted from scholarship funds call donors to a university). And that is how most nonprofit thank you calls are actually made.

If you are a hole-poker of studies—as I usually am—there are other holes to find that can help you sleep better at night as a fundraiser making thank you calls. One is that all of these calls took place within larger organizations with presumably well-honed development systems in other regards. By the time the calls happened, the donors had no doubt already received a thank you letter in the mail, perhaps several emails and newsletters, as well. In those cases, it might be hard to find evidence of the unique marginal value of any one small touch with a donor, even though together each of those touches form a whole which results in renewals and upgraded gifts.

In any case, I urge you not to stop making thank you calls, but to make them better. Make them real conversations with callers who are connected to the nonprofit. Do them as quickly as possible after a gift is made. Foster a sense of identity between the donor and the organization: show them that they matter to you, that they are truly a part of this organization, another person advancing this important mission. Use them as opportunities to learn more about the donor, and record what they tell you—the good and the bad

If you’re particularly ambitious and curious, perform your own tests to learn more about what drives results for your mission. Don’t cut thank-you calls (or any other development practice!) because of one study. Pay attention, collect results, and determine what works best for your organization and your donors.

But rest assured that as long as your donors are persons, they will be grateful for personal contact.