Today I want to elaborate a little on one of the topics brought up in a previous article. That topic is on Opinion Bias. From a previous article we wrote:
“Imagine if we had to go to our boss and give a level of certainty against choosing choice A out of 4 different possibilities. With more data backing up the choice we are able to be more and more confident that we are presenting the right choice. With a smaller sample size we expose ourselves to opinion bias. This occurs when the sample size disproportionately represents our target audience. If our target audience is 50% male and 50% female, then polling 4 women and 16 males will give us opinion bias. When you run your surveys or polls try your hardest to get your sample size to match as closely to your target audience as possible You may even want to include questions about the respondents demographic details so that you can use that later on to filter results.”
It can be so easy to introduce Opinion Bias without even really meaning to. In this article I want to give examples of Opinion Bias that we may not have meant to introduce.
Opinion Bias via Distribution Channels
I think this is the most common form of receiving opinion bias. When you send a survey or poll to users who have signed up for your newsletter, they are users who WANT to hear from you. Sometimes your most valuable data can come from users who maybe don't want to hear from you or who have had a bad experience. Reaching out to these individuals can seem unorthodox – but if done properly it can be very valuable. Send these users a letter or an email. Inform them that they are not on a mailing list and that this is a one time reach out to solicit feedback on what you could have done better. You may be surprised to find out how many individuals are willing to give you feedback even though they are not signed up for your newsletter or mailings.
Opinion Bias via Location or Technology
If you are going to spend all of the effort to run a survey – ensure that you keep it open an appropriate amount of time. Let's assume the example of a carpenter who is out working every day. Maybe it is his/her practice to focus on office tasks (such as keeping up the business, doing bids, etc…) every friday. If you run your survey for only a few short days you will miss out his/her opinions simply because you didn't give enough time to respond. If your survey requires technical steps – you will most likely loose several potential candidates. If they have to install something, register or otherwise do any extra steps a lot of potential respondents will drop out. Keep the technical barrier to a minimum if not completely out of the way.
Opinion Bias via Holidays
be aware of any upcoming holidays that may affect a specific country, religion or area. Often times people piggy back long vacations on to these holidays to maximize their time. If you run a survey just before or just after you will most likely loose out on the candidates who are thinking about leaving or who have just returned and are catching back up with life.
Reducing Opinion Bias in your Surveys is hard. If you thoughtfully plan out your survey, including when to release it, how long to keep it open and who to release it to you will help reduce the opinion bias in your survey and increase the quality of your collected data.