Developing a questionnaire seems easy, right? Decide what you need to know, write a bunch of questions and get people to answer them. What you need to know, is that the responses people give are affected by the way you word each item or question. Many problems arise and much information is rendered useless because of inattention to the way items are worded during questionnaire design.

Item writing is affected by item wording (the term item is used because not all items on a questionnaire are questions). Questionnaire designers must consider the following: Does the item elicit information related to the topic of interest? Is wording clear and unambiguous? Will the item mean the same thing to everyone?

Does the item demand knowledge and information the respondent has (or not)? Does the item contain personal or potentially threatening issues the respondent might resist? Are items loaded with social (un)desirability? Often the approach to items and questions will differ depending on how the survey is administered.

Survey means ‘to look at in a comprehensive way’. There are numerous ways to look at issues in the world of survey research, including in-person and telephone interviews, mailed and online questionnaires.

In-Person Interviews

Getting information in person may be the most personal approach and most effective way of gaining trust and cooperation from the respondent. It is easier to react to puzzled facial expressions, answer questions, probe for clarification or redirect responses. Face to face contact is particularly useful to detect respondent discomfort when discussing sensitive issues or attempts to respond in a socially desirable way.

The in-person interview is usually more costly than any other method. Interviewers must be trained and flown to geographic areas or found and trained within the area of study. It may be unrealistic to send interviewers into areas of high density housing or high crime which may result in an important demographic left out of the study.

In 2008 I wrote a series on “conducting surveys”.  This series was written from the point of view of an absolute neophyte in the survey research industry.  Everything that I have learned since then has been from Organizational Psychologists and experts in related fields; experts such as Dr. Jan Stringer, Dr. Ken West, and Dr. Terrie Nolinske. 

Today I am happy to announce that we are starting a new article series, this time written by Dr. Nolinske.  We will be featuring a new article every Friday.  Dr. Nolinske’s articles will describe the benefits and pitfalls of conducting your customer or employee survey.  I am confident that you will find her postings extremely entertaining and very informative.

We welcome your comments and questions and hope you enjoy the article series.  Suggestions for future series’ are also welcome.   Dr. Nolinske’s first article is below.

Colleagues in the exhibition department at a local museum met to review the list of traveling exhibitions available for rent in the next three years. After a rousing discussion, they decided to seek input from museum members, finding out what traveling exhibition topics were of interest to them. Staff put together a short checklist of known exhibitions, asking members to identify those they would pay a separate admission fee to come see, and added two open-ended questions. Once complete, the exhibits director gave the questionnaire to the membership department for distribution to members in an upcoming member survey.

As the membership coordinator looked at the questions, she thought, as long as we’re asking for member feedback, let me add several questions on issues of interest to the membership. She printed a draft and left for a meeting. As a manager of education was making copies of a program handout, he noticed the questionnaire in the copier. What a great opportunity to get member input on educational program planning, he thought! So, he generated seven questions and added them to the questionnaire.

Needless to say, when member feedback was collated and reviewed, the exhibits director was not only surprised by the inclusion of information related to membership and education, but disappointed by the lackluster responses to the questionnaire; few meaningful responses were obtained to any of the questions.

I live in an area of the country that rarely sees snow. As I am writing this it is the early part of winter and the holidays are only a few weeks behind us. For quite some time before Christmas, a chief meteorologist on a major network kept saying “There is absolutely no way we will have a white Christmas.” The odds were in his favor. There had been no white Christmas in the area in 83 years. Since the National Weather Service began keeping records in 1898, there had never been measurable snow here on Christmas Eve—flurries that amounted to a trace of snow—yes, but never any measurable snow. The National Weather Service defines a white Christmas as any December 25th with at least an inch of snow on the ground. It had been seven years since we had even seen a snowflake on Christmas Eve and prior to that, brief flurries had occurred only two other times on December 24th. So, when the chief meteorologist said that a white Christmas was impossible, I believed him.

We were both wrong.