Friday, January 18, 2013

What thoughts come to mind when you hear “open-ended questions?”

Online surveys seem to have changed the expectations of open-ended questions over the last few years.  Let’s not forget that open-ended questions solicit open-ended responses (if any response in some cases), and quantifying the feedback is not in our best interest of time or analysis.  Open-ended questions in a paper or online environment don’t allow for instant follow-up (although I’m sure live surveys are in operation now).  Open-ended questions are the start of an exploratory process.
I like the idea of developing and using open-ended questions as if it’s a “micro focus group.”  Focus groups are great for developing hypotheses, but not good for making conclusions and they fall into the category of exploratory research.
From Marketing Research Methodological Foundations, Tenth Edition on Focus Groups value and challenges:
  • Generate hypotheses that can be further tested quantitatively
  • Generate information helpful in structuring consumer questionnaires
  • These data are small samples, not representative, the data are qualitative, gathered in an artificial environment.  Findings are suited to “suggesting” (e.g., lines of follow-up research), than predicting (e.g., sales or market share).
My advice:  Don’t forget that just because an open-ended question is included in a quantitative study doesn’t mean it provides quantitative insight.  Treat open-ended questions like micro focus groups; an opportunity to uncover possible follow-up research studies or questions.