Can the integrity of a customer survey be impacted by self-administration?

Yes. Let’s take some lessons from a case study. An organization in the travel industry had outstanding reviews from almost all of its clients on its self-administered customer surveys. Customer surveys were passed out by the tour director on the second to last day of the tour and were collected by the tour director the following morning.

Customer reviews were really good. In fact, the company was able to boast they received a 99% recommendation rate from their guests. All in all, things looked positive for the company. However, they had noticed a trend toward gradual decreases in returning clients and, even more puzzling, it seemed that few people were actually being referred to the travel company. Somehow things didn’t add up. Why weren’t people referring their travel company?

One of the new company directors made a controversial decision to outsource their customer survey to a third party. Much to the company’s surprise, the information gathered by the third party didn’t even seem to be from the same company. Only seventy percent of customers said they might consider recommending the travel company to others. Hardly a glowing endorsement. Furthermore (and despite the substantial cash outlay for the tour packages offered by the company) some clients were abandoning ship and leaving early—they weren’t even around to answer the customer survey!  Finally, it surfaced that tour directors became scapegoats when clients were dissatisfied with company offerings, even if the points of dissatisfaction were beyond the tour director’s control. Knowing this, it wouldn’t be surprising if several dozen surveys from unhappy clients went missing.

This case study demonstrates poor survey administration practices that likely led to skewed response rates as well as affirmation and socially desirable response biases. Ultimately, this created an inaccurate picture of the organizational state. Unfortunately, this sort of occurrence is not uncommon.