Survey Question Design

Survey Question Design

The Good, The Bad, and The Ugly

One of the first things to consider with a survey is what you want to do with the data. The content of the survey itself will then emerge from the research objectives. Thus, the first step of the question database development begins with internal questions of: “What do I need to know?” and “How will I take action on this data once I receive it?” Once the answer to these items are crystallized, we will know how best to proceed with the survey.

Many survey providers focus on selling standardized questionnaires that address specific organizational constructs. While these types of questionnaires do have utility, they may not address the most pressing or important issues in an organization. If your company is unique, your question set should be as well. NBRI takes a bespoke approach to developing a question set. We first focus on understanding your research objectives and topics of interest before drilling down into specific questions. We then offer a ‘buffet’ of questions for your selection. Questions that are less relevant or unimportant to your organization are not included.

By mixing and matching your question set, you have a great deal of flexibility in your survey program. However, this does not come at the expense of validity. Since NBRI’s questions have been written by psychologists and tested on millions of people, they are valid and reliable questions. Furthermore, reliance on these standardized questions gives you the ability to compare your scores against others who have used the exact same questions. In this way, you can form a relative understanding of your performance through a process called benchmarking.

We know that your question set should be flexible to meet your needs, and we also know that flexibility does not have to be sacrificed for validity. What are some other things that we should consider? Consideration to these five guiding principles of survey research will go a long way to ensuring the development of an effective question set:

Keep action at the forefront of your mind. Many well-intentioned people initiate a survey program to enact change, but begin to lose sight of this objective as they become entrenched in question development. Don’t let yourself become distracted. As you work through the question set, continually ask yourself how you would react if you encountered low scores for each specific question you include on your survey.

Ask about only those things you can address. If you do not have the resources (i.e. the manpower, financial means, or technological capabilities) to address certain issues, do not include these issues on your survey. By including an item on a survey, you raise awareness of that item along with the hope that something will be done to address that issue, should it be an area of low performance. For example, if career development opportunities are severely limited in a company, asking questions about the opportunities that exist there can be damaging. In general, we recommend focusing only on those items that you have the ability to do something about. By narrowing the scope to actionable pieces, you will gather a greater amount of important information.

Ask the appropriate amount of questions. A survey should be comprehensive, yet economical. Ultimately, the amount of questions you ask will depend on the population being surveyed. If respondents are gifting their time to complete the survey, we should be considerate of that and keep the survey relatively brief. Generally speaking, employee surveys will be longer than customer and market research surveys, because there are more facets of the organization to address. Since every question takes up ‘space’ on the survey, we must ensure we ask valuable questions.

Focus on a mix of big picture and specific items. By asking broad, big picture, items you will be able to form an accurate assessment of the totality of the employee, customer, or market experience. While this will give you a good indicator of the status of the organization, it will not reveal the specific areas that need to be targeted to make improvements in the organization. This is why we need to drill down more narrowly into select topics and ask specific questions. Ultimately, with ClearPath Analytics, we are able to understand how specific and actionable items impact big picture data.

Don’t forget the demographics. Demographics allow you to cluster your data into reports that permit greater specificity and enhanced understanding. If you already have demographic data on file for your population, NBRI won’t have to ask these questions in deployment. If you don’t have demographic data, this is something you need to consider incorporating into your survey. Some examples of demographic information include:

Employee characteristics such as tenure, age, gender, performance appraisal rating, pay grade, managerial status, department, and employment status (i.e. part-time, full-time).

Customer characteristics such as zip code, age group or gender, customer income, the average spend of a customer, or the frequency by which they use their service or goods.

By following these guiding principles of survey research, you will ensure a successful survey experience. If you choose to work with NBRI, you will work with an organizational psychologist who has guided others through this process countless times. Select your questions—and your research partner—wisely!

Read More

Survey Question Miswording