Note: The following is a guest Blog Post written by Michael Kaiser-Nyman.
Ever wondered if your survey results are truly representative of the group you’re trying to survey? Maybe you email all 1,000 of your customers with your survey, and 50 respond. What if young people, being more internet-savvy, respond more than old people? Or unemployed people, having more time, respond more than employed people? Statisticians call this “response bias”, and it means that you can’t take what you learned and assume it’s true for the people who didn’t complete your survey.
Pollsters have long used phone interviews because they used to have relatively little response bias: everybody has a phone, and people of different demographics are more or less just as likely to take a survey. But that’s changing: more than 1 in 4 American households don’t have a landline any more (refer to the Early Release of Estimates From the National Health Interview Survey), and, as it is illegal for pollsters to cold-call cell phones, there is now a significant response bias from telephone surveys. So what can you do?
Increasingly, pollsters are looking at multi-channel surveying: using a combination of phone and internet surveys to capture responses from people of every demographic. You can do this, too! A non-profit called the New Organizing Institute recently did a study of multi-channel surveying (refer to http://neworganizing.com/2011/01/13/want-a-higher-action-rate-pick-up-the-phone/).
They emailed a survey to about 1,000 people who had attended a recent training. Then, they randomly chose half of them to get a phone call as well. Here’s the takeaway: “Those who we called filled out the survey at a rate of 16% while those who we didn’t call filled out the survey at a rate of 5%.” That’s a pretty big improvement! And while they don’t have data on whether the people who got the call were more representative of the overall group being surveyed, it’s seems pretty likely that they were.
So the next time you want to improve the response rate and reduce the response bias of your online surveys, consider making some phone calls after your emails go out!
Michael Kaiser-Nyman is the CEO and founder of Impact Dialing, an easy to use hosted predictive dialer for small businesses, call centers, political campaigns, and pollsters.