Analyze Sentiments in Survey Responses

The Survey Analysis workspace allows you to analyze the sentiment of survey responses and automatically code them based on their emotional tone. MAXQDA evaluates whether responses are negative, neutral, or positive, helping you quickly identify patterns in how respondents feel about your survey topics.

Analyze Sentiments of Survey Responses

MAXQDA can automatically evaluate whether the content of your responses is negative, neutral, or positive. Once analyzed, you can view sentiment scores directly in the table and optionally autocode responses based on their sentiment levels.

To analyze the sentiment of your survey responses, click Analyze Sentiments in the workspace's main menu:

1. Click Analyze Sentiments in the workspace's main menu.
2. In the dialog that appears, you can configure the following options:

Select language – This selection ensures that MAXQDA uses the appropriate lexicon for sentiment evaluation.

Ignore hashtags – If this option is enabled, hashtags such as #bestever are not included when evaluating sentiment.

Apply stop word list – If this option is enabled, all words from the selected stop word list are ignored when evaluating sentiment. Hashtags that correspond to a stop word are also ignored.

3. After clicking OK, the sentiment analysis runs and four columns are added to the table view:
   • Sentiment: Contains the sentiment rating on a five-level scale from "negative" to "positive." Responses that do not contain words with a sentiment value are marked as "No sentiment."
   • (Positive) Words: Number of words evaluated as positive.
   • (Negative) Words: Number of words evaluated as negative.
   • Difference: Difference between positive and negative words. If the number is negative, the negative words predominate. Emojis and symbols count as words.

Autocode Survey Responses with Sentiments

You can save the sentiment assignments for individual responses as codes in MAXQDA's "Code System." This allows you to work with sentiment categories just like any other codes in your project.

1. Click Autocode Responses with Sentiments in the workspace's main menu.
2. In the dialog that appears, select the sentiment levels you want to autocode.

After clicking OK, codes are created for the selected sentiments. If they already exist in the code system, the existing codes will be used for coding. The text of the responses will be autocoded with the selected codes.

How does MAXQDA evaluate the sentiments?

MAXQDA uses a lexicon to evaluate sentiments, which contains a sentiment score for each word in the lexicon. This value is negative for words with negative connotations, close to zero for neutral words, and positive for words with positive connotations.

When analyzing a response, MAXQDA checks each word whether it is contained in the lexicon and assigns the sentiment score to this word (hashtags and stop word list words are ignored if desired). If the word is not in the lexicon, MAXQDA looks up the word in a lemma list. If a lemma is found for the word and this lemma is contained in the sentiment lexicon, the sentiment score of the lemma is used for the word.

In addition, two rules are applied to optimize the evaluation of sentiment:

1. In case of negation, the scores of the following 3 words are reversed, for example, the statement "I was not very happy" is classified as negative.
2. In case of modal verbs, such as "can", "should", etc., the sentiment scores of the following words are reduced.

The mean value is calculated from the sentiment scores of the evaluated words in a response. If the mean value is negative, the response is evaluated as "negative" or "slightly negative", if the mean value is positive, the response is evaluated as "positive" or "slightly positive". If the mean value is equal or close to zero, the response text is classified as "neutral". Responses that don't contain any word with a sentiment score are classified as "no sentiment".

In the literature on automatic sentiment analysis, values of 60 to 70% agreement compared to a human control assignment are usually reported. It has to be taken into account that human assessments of the same text can also differ. The quality of a sentiment analysis depends on several factors, especially the fit of the lexicon for a given topic. Please also note that a sentiment analysis has little meaning for responses in which little to no evaluations are expressed.

English lexicon

The lexicon for English is based on the resource SentiWordNet 3.0, which is licensed under an Attribution-ShareAlike 4.0 Unported (CC BY-SA 4.0) license (for more info on SentiWordNet, see Baccianella, Esuli & Sebastiani, 2010). The SentiWordNet 3.0 lexicon has been optimized for use in MAXQDA, including

• removal of special characters and duplicates
• addition of typical social media abbreviations (e.g. LOL - laughing out loud), emojis, and superlatives
• manual adjustment of scores for individual words to scores of their synonyms

Bibliography

Baccianella, S.; Esuli, A.; Sebastiani, F. (2010). SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. Proceedings of LREC. 10.
http://nmis.isti.cnr.it/sebastiani/Publications/LREC10.pdf