Be a Survey BOSS

We’ve all seen surveys in our inbox or have been inundated with pop-ups asking us to take part in a survey. If you’re in the communications field, chances are you’ve been the one sending surveys too. It’s important to have well-crafted questions. When I was responsible for sending e-newsletters, I used to send reader surveys, and the quality of questions always surprised me.  They ranged from good to incomprehensible with multiple steps and questions. This post offers advice on writing quality survey questions.

It may seem obvious, but it is worth repeating that surveys need to be valid and reliable. Validity in surveys means you measure what you intend to measure (does it fit your research goals). Reliability means questions should elicit the same type of response from similar respondents. To achieve a valid and reliable survey, questions need to be relevant and accurate.

Floyd Fowler, author of Improving Survey Questionnaires: Design and Evaluation, on writing survey questions: “The goal is to have differences in answers reflect differences in where people stand on the issues, rather than differences on their interpretations of the questions.”

Fortunately, there’s an acronym to aid would be survey authors. Giuseppe Iarossi in The Power of Survey Design outlines the four criteria for effective survey questions: Brief, Objective, Simple, and Specific (BOSS).

Brief

Questions shouldn’t include any superfluous words. Aim for questions that are twenty words or fewer in length and contain fewer than three commas. Complex sentence structure detracts from a survey’s readability. It’s important to keep in mind that brevity also applies to contextual simplicity. Ask one question at a time. Iarossi uses the example of asking someone, “What interest rate are you paying on your loan?” The question implies that a person has a loan, which may or may not be the case. It should be prefaced with the question: “Do you have a loan?”

Objective

When writing survey questions, one needs to focus on using neutral wording. Leading questions are questions that guide respondents toward an answer. For example, MW-COMM Blog is a valuable resource, agree or disagree? An objective version of the question would be Have you used MW-COMM as a resource in your job?

Loaded questions use emotionally charged words or stereotypes. For example, Don’t you see a problem in allowing children to consume sugary beverages? To make this question objective, it could be written Is it a problem for children to consume sugary beverages?

Simple

Survey questions need to be written or phrased using plain language. Be direct. Use words like “think” instead of “consider” or “say” rather than “state.” Similarly, avoid technical terms and jargon.

Specific

Be specific, ask precise questions. Vague questions yield vague results. Worse vague questions frustrate respondents, and frustrated respondents abandon surveys. Ambiguous words should be avoided. Words like occasionally, rarely, good, and bad are ambiguous because different people will interpret them differently. For example, occasionally for one person might mean twice a day, and for another, it might mean twice a week. Also, avoid abbreviations because you don’t want to assume the audience knows what they stand for.

The next time you’re tasked with writing survey questions, try to visualize yourself as the respondent. And remember you’re the BOSS of the survey. Always be brief, objective, simple, and specific.