This is a guest article written by Matthew Loxton, a Principal Healthcare Analyst and Professional MAXQDA Trainer.
In this blog post, I will expand the scope to the left of the survey analysis process, and draw your attention to a source of data that can enrich the coding and code memos in a way that is efficient, effective, and uses existing data to enhance the coding and analysis of survey responses.
Using Survey Questionnaire Handbooks to speed up the development of Codes, and to enhance analysis of survey data with informative Code Memos
Although it has always been possible to collect participant feedback via open-ended survey questions, the advent of online surveys has made them ubiquitous. However, although online questionnaires typically have several open-ended questions, almost all the analysis work has typically gravitated to the closed answers that are more easily quantifiable.
For example, I am sent a weekly survey of patients at a U.S. ambulatory care clinic, and each week, and at the end of every month, quarter, and year, the Likert scale questions are used to generate trend graphs, pie charts, and bar charts. In comparison, over several years of the survey being sent out, the open-ended responses have never resulted in more than an occasional “why did this complaint happen” query from the hospital management team.
Part of this disparity between reporting on closed and open responses is due to the difficulty in analyzing open-ended questions. However, as MAXQDA Professional Trainer Graham Hughes shows in his recent webinar tutorial on Analyzing Answers to Open-Ended Questions in Surveys, MAXQDA has very powerful methods to import, analyze, and report on open-ended survey responses. Likewise, the MAXQDA team created an online tutorial for Working with Surveys in MAXQDA, which shows methods to streamline the process of importing and analyzing survey data. A link to the videos can be found at the end of this article.
In many cases, a survey questionnaire handbook will have been developed in parallel with the questionnaire. The handbook documents the research approach, including the constructs that are to be measured, the population and sampling methodology to be used, as well as details of the question items, their scales and interpretation, and the logic and context for each question.
Some of the metadata can be used to bolster analysis in MAXQDA by using the questionnaire item titles as codes, and question item details as part of the code memos. Having these details can greatly enhance the thickness of the coding and depth of analysis in MAXQDA.
A questionnaire handbook may contain the following sections:
- Overview: A high-level description of the scope, goals, and stakeholders.
- Background: A description of the problems and context of why the research question is relevant and important.
- Measurement Approach: A description of how the sampling is envisaged, and what process, tools, and populations will be used.
- Working Group: A list of who was involved in the survey design.
- Goals: A description of the research questions the survey aims to answer, or what problems it seeks to resolve or explain.
- Metric Definition Key: A multi-level detailed explanation of the measurement structure.
Table 1: Benefit Construct Structure
Likewise, in this handbook, each metric is listed again as a question item with a preamble that will be used to populate the online survey. For example, in Table 2, “Self-Advocacy” is broken out into facets (metrics) as previously, but now shows the question that will be presented, along with an introductory preamble.
Table 2: Effectiveness
Constructing the Import Sheet
The preamble and questions can be easily combined in an Excel sheet by concatenating the cell for the question preamble and each of the question items, as well as any other details that will inform the qualitative analysis. In Table 3, I have used the question title for the code column and combined the question components as memos.
Table 3: Excel Import Sheet
Analysis with MAXQDA 2018
Importing the resulting code structure is done by clicking on the “Codes” tab in MAXQDA 2018, and then on “Import Codes and Memos from Excel Spreadsheet”, as shown in Figure 1.
Figure 1: Code Import from Excel
The result of importing the code system from Excel is shown in Figure 2. The codes for each question are now in MAXQDA, and the mouse hovering over the memo for the “Clin-Plan” code shows that the question details have been successfully imported.
Figure 2: Imported Code System
Following the establishment of the code system, survey results can be imported by using the MAXQDA Import function. By clicking on the “Import” tab, and “Documents from Excel Spreadsheet” (Figure 3), we can add the respondent feedback and auto-code with the imported questionnaire item titles as a single step.
Figure 3: Import Documents from Excel
After the import is done, we can see that the codes have been associated with the related documents (Figure 4).
Figure 4: Imported and Auto-Coded Documents
The use of the questionnaire codebook as the source of MAXQDA codes and memos enables us to reuse existing data, stay aligned with the original survey terminology, and assist analysis by having a ready reference in the code memo to remind us of the code meaning and intent without having to retype or create memos manually.
In this example, I have only pulled in the preamble and question items as memos, but the same technique can be used to add scoring interpretation or any other pertinent details to assist in analysis.
Want to learn more about Survey Analysis with MAXQDA? Check out these 2 videos on the MAXQDA YouTube channel or download the chapter from the online Manual:
About the Author
Matthew Loxton is a Principal Analyst at Whitney, Bradley, and Brown Inc. focused on healthcare improvement, serves on the board of directors of the Blue Faery Liver Cancer Association, and holds a master’s degree in KM from the University of Canberra. Matthew is the founder of the Monitoring & Evaluation, Quality Assurance, and Process Improvement (MEQAPI) organization, and regularly blogs for Physician’s Weekly. Matthew is active on social media related to healthcare improvement and hosts the weekly #MEQAPI chat. You can also read another guest post by Mathew Loxton on “Mixed Methods Research: Surmounting Obstacles with MAXQDA and R” here in the MAXQDA Research Blog.