Survey Question Miswording

Survey Question Miswording

What you don’t know WILL hurt you!

Use NBRI’s knowledge and experience to put power into your research!

In surveys, language is everything. Composing survey questions with a universal meaning that is clear to respondents is of paramount importance. This ensures that respondents interpret the questions correctly, which speaks to the integrity of the data you receive from your project. It also ensures that respondent ‘fall-out’ is minimized.

In order to understand how survey questions should be phrased, we’ll explore some misworded questions below. By understanding the common pitfalls associated with the wording of survey questions, you can be better prepared to recognize bad questions and ensure they are removed from your question set. The types of questions that do not belong on your survey are:

Polarizing. When strong language (e.g. always, never, etc.) is used in a survey, respondents may be biased to answer in the converse and/or it may restrict the potential for respondents to use the entire range of the survey rating scale. For example the question: “I can always get in touch with an account representative in a timely manner,” will produce few—if any—respondents indicating a strong level of agreement. Furthermore, it will lead respondents to answer more negatively if their experience isn’t always consistent. This type of question produces compression in the data set, meaning scores will be tightly clustered together. As a result, it can be difficult to accurately distinguish between groups and areas.

Double-Barreled. Survey questions should focus on one construct, or topic, at a time. Questions that fail do to this are called “double-barreled” meaning they ask about multiple constructs in one question. For example the question: “I have good opportunities for pay raises and promotions,” will provide answers that lack clarity. Perhaps there are good opportunities for promotions, but these promotions come without pay increases. It is imperative to focus on one construct at a time in order to understand and accurately interpret the data gathered from a survey question. While it may be tempting to take ‘shortcuts’ in survey research by combining questions, it will undermine the potential to accurately understand results.

Ambiguous. Survey questions should be clearly and sharply worded. This means that respondents should be provided with definitions if their might be some uncertainty surrounding the individuals that constitute “supervisors” or “management” for example. The question should also ensure that targeted action can be undertaken in response to the question at hand. For example, the question: “Communication at work is good,” does not specifically address the source or direction of communication. As a result, if the question is less than 100% positive, it is impossible to know what elements of communication should be improved upon.

Wordy. A good survey question is one that is direct and to the point. Superfluous descriptions, excessive prepositional phrases, and convoluted ways of shaping inquiries should be avoided. For example, the question: “Thinking over my entire experience with the company, I would say the company has a good focus on customers,” could be phrased much more simply and elegantly as, “Customer service is good.” Wordy questions can confuse respondents, reduce the quality of answers received, and reduce response rates.

Mini-Messages. While it may be tempting to use questions to communicate core values to employees or customers, it can alienate respondents. The survey is an opportunity to gauge customer, employee, or market perceptions about the company, not to tell respondents what you hope their perceptions ought to be. For example: “I exemplify the core value of Responsiveness by addressing customer requests within 24 hours.” A survey should not be used to fulfil any hidden agendas. It should be used to get a pulse of the organization, its employees, and actual or potential customers.

Negatively Phrased. A series of negatively phrased questions has the opposite effect of leading questions, as these questions can implant undesirable thoughts and perceptions about an organization. For example, the question: “Workloads at our Company are excessive,” can cause respondents to reconsider their opinions about workloads and begin to see negativity where there was previously none. For this reason, negative language should be minimized or avoided.

In sum, questions that should be avoided are: polarizing, double-barreled, ambiguous, wordy, mini-messages, and negatively phrased. Our review of poorly worded questions has surfaced some themes in what constitutes good questions. These commonalities include the reliance on neutral language, sharply defined groups and categories, and the economical use of language. There should be a feeling of transparency around the question set.

Having well worded survey questions is only one part of the equation; it is essential questions are assessed with a balanced survey rating scale. A balanced survey rating scale contains equal points of positive and negative sentiment. In other words, the response categories should project neutrality and minimize the potential to bias or to lead respondents. Appropriately worded questions and a balanced survey rating scale are two essential components of any good survey.

Survey questions may appear straightforward, but as you can see, the process of arriving at ‘simple’ questions can be rather complex. By relying on NBRI’s expertise in the design of survey questions, the process can be painless. We’re here to simplify the process for you.