How To Analyze Open Ended Questions

by
Philip Cleave
on
November 25, 2022
Researcher thinking about how to analyse open ended questions

When it comes to your research and what you’re trying to find out, the research process behind it will generally fall into two camps quantitative or qualitative.

While quantitative research is more focused on the numbers, such as the volume of your audience that holds a particular view, qualitative research would be more concerned about identifying exactly what those people felt. Subsequently, this will influence the survey questions that you use with quantitative data typically collected by asking closed ended questions, and the qualitative approach more focused on open ended questions. But you can use a mix of the two in your questionnaire.

For the purposes of this blog, we will be focusing on open ended questions in surveys and how best to analyze them. But before we look at the analysis side of open ended questions, it helps to explore in more detail what they are and how they differ from closed ended questions.

Open ended questions definition

Open ended questions are the type of question that provides an open-text format for respondents to answer it in their own words.

Rather than limiting them to answer from a set number of options, open ended questions enable respondents to provide more detail about their thoughts and feelings.

Open ended questions are ideal for qualitative research, as they allow researchers to probe much deeper into a respondent’s answers. This can enable them to gain valuable information about a subject that they might not otherwise pick up.

Difference between open and closed questions

The main difference between closed and open ended questions can be seen in terms of the flexibility respondents are offered for answering them.

With closed ended questions, respondents can only answer ‘yes’ or ‘no’ or choose their answer from a pre-set number of multiple-choice formats and options. This makes the data quicker and easier for researchers to review and analyze.

In contrast, with open ended questions respondents are given a free text box and the freedom to answer the question in their own words. While the answers to open ended questions can be slower and less simple to analyze, they do offer researchers more detailed and valuable information. This is because the answers provide more insight into why respondents have answered in the way that they have and their feelings behind this. In addition, open ended questions can often deliver insights that you didn’t expect, which can be beneficial in providing you with ideas that you may never have otherwise considered.

It’s worth noting that there are now some analysis tools available, which make it simpler and quicker to uncover the valuable insight provided by open ended questions, which we will look at next.

How to analyze open ended questions in a survey

When it comes to your survey, you’ll want to get as much value from your results as you can. However, respondents who don’t meet your target criteria, only answer a portion of your survey, or provide nonsensical feedback can easily harm the quality and reliability of your data.

Therefore, before analyzing open ended questions, using analysis tools to review your data, it helps to remove any unhelpful data from your survey results first.

Once you’ve done this, you’re ready to start looking at analysis tools.

Text analysis

In terms of tools to help analyze and uncover the most valuable respondent sentiment in open ended questions, the Text Analysis tool, is probably one of the most effective around.

Text analysis is essentially a machine learning technique, which can be used to automatically extract valuable data insights from unstructured text data. Text analysis is usually available with a range of options that you can select to run the tool and comb through your open text responses, looking for correlations and keywords.

Options to run text analysis

When it comes to analyzing open ended questions with the text analysis tool, you can either set up and apply categories, or quickly view responses as a Word Cloud or Word List.

With any research there’s usually a number of different outcomes you think may happen. And therefore, if you’re running a survey, there’s likely to be some themes, patterns and correlations in the responses that you’ll be hoping you might see as supporting evidence for one of these outcomes. This is where the categories option and the tagging of text within responses can be useful.

For example, if you were running a product testing survey for a new website and wanted to know how positively or unfavorably a sample of your customer base viewed it in terms of its usability, functionality, engagement and more, the categories option could help you with this.

In such a scenario, you might choose to set up two categories, one for positive phrases and the other for negative phrases and tag key words and phrases you wanted the text analysis tool to search for under each category. So, in the case of positive phrases, you might want to scan for words and phrases such as, ‘easy’, ‘easy to use’, ‘user friendly’, ‘simple to navigate’, ‘great user interface’, ‘great UX’ and more. You could also apply different color codes for positive and negative words, to make the data simpler to review at the analysis stage.

Having run the text analysis over your completed responses, the tool will count the total number of tagged words and phrases to each question. It will then display these as a pie chart, so it’s easy to see how favorably or unfavorably respondents view different aspects of your website. This would help you to identify which areas customers were happy with and which areas you needed to improve.

If you don’t fancy using the categories feature, there are some further options you can use with the text analysis tool including the Word Cloud and Word List features.

By selecting the Word Cloud option, it will provide you with an image of the most common words used in the responses to your questions. So, if you were asking respondents a question about how they perceived the service of your support team, it could generate a Word Cloud image similar to the one below.

Alternatively, you could select the Word List option, which would give you a quick breakdown of these words and how many times they were mentioned, from the most to the least used.

To learn more about text analysis and how you can set it up, you might want to take a look at the guide we’ve put together to help with this.

Final thoughts

Whether you’re new to open questions or have been already using them for some time, we hope this overview of the question type and advice on how to analyze open ended questions has been useful to you.

Open ended questions are hugely beneficial in providing you with a deeper insight into the thoughts and feelings of your respondents, that would be difficult to achieve with any other question type. And fortunately, with tools such as text analysis you can now get to crux of the detail you need quicker and more easily than ever before.

Drive great insights with the right survey tools

When you’ve got great questions you want to ask, and you have the best survey tools supporting you, you’ll be able to reach out and get to the insights your need faster.

Get started and create your first survey

If you would like more information then please get in touch.