Vol XVIII Nov `05

"Writing good, effective survey questions is more difficult than it looks," said Parmelee, who will be presenting a primer on survey questionnaire development at the AMDA Foundation Research Network Fall Conference in November. "When we write our survey questions, we sometimes assume that others are as familiar with the issues as we are. As a result, we sometimes ask questions in a way that is not clear to respondents; and we don't get usable answers," she added.

The best way to ensure that questions are strong, objective, and understandable is to pre-test them thoroughly, suggested Parmelee. She suggests doing this with both individuals who are representative of the survey audience and people who are unfamiliar with the subject matter. "Start by sitting down with someone and conducting a cognitive interview. Go through the survey question-by-question. Have the person tell you what he or she thinks you mean, what you want, where they had problems, and so on," she offered. These individuals can be pretty much anyone, she explained, such as a spouse, colleague, neighbor, or friend.

In addition to the face-to-face pre-test, Parmelee also recommends giving the survey to several individuals in the way the researchers will be administering the final survey, e.g., online. These people should be representative of the survey audience. "Take your survey to these people and let them run you over the coals," Parmelee offered.

Parmelee suggests a couple of "red flags" that suggest a question or questions is "bad" and needs to be eliminated or reworked. For one, "you get a lot of middling responses--'don't know,' 'not applicable,' and so on." An abundance of such responses "can tell you that respondents don't understand the question."

A vast majority of the same answers to a question is another indication that there is a problem. For example, "asking people to agree or disagree with the concept that falls prevention is a priority for nursing facilities is a waste of space. No one is going to disagree with this. Make sure that your questions are set up to tell you something you didn't know already," Parmelee observed.

Watch for more of Parmelee's comments in the next issue of this e-newsletter. And don't miss her presentation at the Fall Research Network Conference in Chicago this November.