Survey bias is something researchers and survey experts have long loathed. Too much of it can literally make the results of your surveys insignificant and irrelevant.
The thing about survey bias that most businesses forget is that it’s not just about the way you phrase a question, but also the types of questions you choose and the order of your questions.
Let’s examine each of these in more detail.
One of the most common ways to insert bias into your survey is by phrasing questions in such a way that they elicit either a positive or negative reaction from the respondent. The key is to keep the phrasing as neutral as possible. What makes this challenging is that often the survey designer is unaware of their own bias or lack of knowledge on the survey topic. They accidentally phrase questions that don’t take into account things like respondent sensitivity and public opinion. Or worse yet, due to their ignorance on a topic, they may even confuse respondents with their phrasing. As a general rule, you’ll want questions to be brief, single focused, jargon-free and without emotive language. For questions about a popular topic, read as much as you can about the issue before designing your survey, and even consider getting a industry expert to weigh in on the phrasing.
Types of Questions
At QuickTapSurvey, we offer an extensive range of questions including multiple choice, ranking, sliding scale, open-ended and true/false. Depending on what question type you choose, you may get a very different response. It’s important that the survey designer is aware of the strengths and weaknesses associated with each question type before creating a survey. For example, let’s say you’re a pie shop owner and you’re trying to figure out what your most popular sweet and savory pies are. In a customer satisfaction survey, you have a multiple choice question asking Which of our pies do you like best? where you list all of your pies. The data you collect from this question will tell you what your most popular pies are, but not according to each sweet and savory category. Respondent might choose different pies if you used a different question type and split the question into two. A better option would be to use two ranking questions, Please rank our savory pies according to your preference and Please rank our sweet pies according to your preference. This will give you more granular and specific data.
Order of Questions
The order in which you place your questions, including what you say in the intro and end screen, can significantly impact the nature of the responses you get. Not only will asking controversial questions first kill your response rate, but it can cause the respondent to be more timid and less forthcoming with information as the survey proceeds. Sensitive demographic questions like age or questions that put a greater cognitive burden on respondents should also be asked near the end. Remember, you want your respondents to be excited about your business and brand as they begin the survey. Don’t turn them off before they give you the information you want.
It’s important to note that survey bias doesn’t exclusively show itself in questions. There are other ways that survey bias can negatively affect your results including styling, methodology, and data interpretation. Stay tuned for another blog post where we’ll discuss those issues.