Guest post by Lukas Müller.
As political science researchers, we are interested in how political causes can lead to political effects. How these causes and effects can be measured has, however, long eluded political scientists and continues to be contested. While quantitative researchers have found a solution to this issue by creating statistical data and looking for correlations between variables, qualitative researchers have often struggled to convincingly attribute causal relationships to their research, particularly if their research employs a case study design. It has often been called into question how much causality can be attributed to a single case or even a comparison of two cases. The type of research I’m conducting in Indonesia and Nigeria aims to bridge the gap between methods sensitive to the facts of single cases and those enabling comparison between larger sets, through the establishment of clear causal connections in related political settings.
Process Tracing: the method with the tools of a crime scene investigation
Fortunately, progress is consistently being made on methodology in qualitative research. One significant innovation in qualitative political science research is that of process tracing, which essentially translates the tools and methods of a crime scene investigation to the objects of political analysis. As a first step, hypotheses are established, and a concrete case is broken down into the relevant actors, actions, and the mechanisms by which actors engage in these actions. These hypotheses are then tested against the available evidence and a case is built to confirm or disconfirm the explanation provided for a certain case. A good process tracing design should read like a court case: a crime is presented, and the results are outlined. A hypothesis is put forward and a suspect (or multiple suspects) is identified. The case is then proven by using a variety of evidence and eliminating alternative explanations. Alibis are disproven and, in the end, there is hopefully little doubt that the explanation put forward is the correct one.
Lukas and a collaborator at the Friedrich Ebert Foundation, Abuja
In my research, the causal relationship of interest is that between capacity building in regional organizations by external actors and the constraints encountered by those organizations. In other words, it is about finding out how capacity building projects stemming from the European Union (EU) and other partners do or do not solve the capacity gaps faced by similar organizations in the Global South that are operating under political, financial, and various other constraints. One key question here is how those constraints experienced and voiced by those regional organizations translate into the development programs executed by the EU. Given that the decision over these programs is a dynamic process involving various actors from regional organizations in the Global South and the EU, there are processes of communication and negotiation underlying the design of these programs. To convincingly trace the process of how these programs are created, it is important to prove the hypothesis with statements from actors on both sides of the relationship.
This is where I have chosen to use MAXQDA 2018 to carry out my methodological treatment of interview data covering the design and objectives of these projects. Looking at a dataset consisting of 140 interviews with both internal and external actors in both the Economic Community of West African States (ECOWAS) as well as the EU, I have been using MAXQDA’s Code Matrix Browser function to pre-select the interviews (and therefore the data sources) to quickly select the interview partners who spoke most about the institutional capacity issues in question.
Code Matrix Browser: using MAXQDA to sort through (extremely) large interview data sets
Figure 1: Analyzing co-occurrences of information with the Code Matrix Browser
Based on this data, I can very quickly select a subset of documents to do a more in-depth analysis of the interviews containing relevant information on the institutional capacity constraint. This enables me to not only deal with a large dataset (transcriptions of 50 hours of audio recordings plus several hundred pages of conversation notes) but also to get a preliminary view of which capacity constraints are mentioned more or less frequently. In a previous study I conducted before using MAXQDA, coding of causal processes within regional organizations was done by hand, which was a very labor-intensive process and left me with the nagging suspicion that I had missed certain pieces of information or failed to connect them in the most meaningful way.
Using MAXQDA enables me to see the most relevant facts on a case at a glance and correlate them with other pieces of information to immediately highlight how they relate to one another. This technique makes it possible to see whether there are any co-occurrences of information on institutional capacity constraints and problems involving capacity building projects and partners. The following figure highlights a Code Matrix Browser output showing the same mentions of institutional capacity constraints and issues with partners within capacity building projects.
Figure 2: Multidimensional analysis with the Code Matrix Browser
This output also gives me an impression of which documents are relevant to analyzing the occurrence of relevant mentions across a large set of documents. In addition, it also shows me which documents mention both capacity issues inside the organizations and issues within capacity building projects. These documents are particularly important for teasing out the potential for misunderstandings in the design of capacity building projects as these interview partners have addressed both organizational issues as well as issues with partner projects simultaneously, showing sensitivity for both sides of the bilateral relationship under analysis.
Complex Coding Query: building a case for causality with co-occurring codes
Using the Complex Coding Query, we can now analyze these documents to establish causality between the identified institutional capacity constraints and the design of capacity building projects. This is done by testing for the successive mentioning of said capacity constraints and information on relevant projects within the same interviews. The Complex Coding Query has allowed me to search for segments assigned to one or various codes followed by another single code. In the example of Figure 3, these codes are all relevant codes subsumed under the code “institutional capacity constraints” followed by all codes subsumed under the code “capacity building projects”.
Figure 3: Analyzing capacity building projects with the Complex Coding Query
As you can see in Figure 3, searching for the co-occurrence of codes within documents narrows down the relevant documents and sections even further. MAXQDA allows you to set up a maximum distance between the first selected codes (A) and the second selected code (B). In Figure 3, that distance is set to 10 paragraphs, which makes the query show ten segments. Figure 4 shows the same query with a reduced distance. The selected segments now have to occur within the same paragraph in order to appear in the query.
As you can see in Figure 4, the query still comes up with 5 segments. Establishing the co-occurrence of these codes within the same documents helps in process tracing by building a case for the causality between the identified institutional capacity constraints and the design of capacity building projects. By having these co-occurrences exist in close proximity within the same document, our case that we are in fact dealing with a causality in design, not simply random co-occurrence, is strengthened.
Figure 4: Complex Coding Query with a max. distance set to 0
Going the distance: how to conclusively establish causality in political processes
The analysis process shown above is obviously just the first step towards establishing causality between the identified codes and the empirical data attached to them. Being in the field so far has shown me that using software and doing fieldwork simultaneously can bring surprising benefits. Being in my second stint in the field in Indonesia currently, I have derived several new research hypotheses not just based on theory but also on my preliminary analyses of my interviews conducted in Nigeria.
Leaving my first fieldwork location in Abuja and moving to Jakarta has been an interesting transition in many ways. Even though there are certain structural similarities between the two organizations I am analyzing, and my project utilizes a rational choice framework, culture looms large in the comparison of the two organizations. Certain organizational features and activities are framed differently by the policy-makers involved, creating a potential for misunderstanding, particularly when conducting qualitative research. Having a good sense of the main features of my first case has helped me to avoid getting sidetracked during my second fieldwork stint. It has also made my work more focused on certain key aspects and enabled me to take a comparative angle.
I will address questions of comparison and case-sensitivity further in my future fieldwork diary entries. I will also outline the next step in my process tracing analysis methodology, where I move from this pre-selection of data towards the actual mapping of causalities using the MAXMaps function. Using the sections highlighted through the Complex Coding Query above, I can now look at the concrete segments previously coded as relating to institutional capacity constraints and capacity building projects. Looking at the detailed statements made there allows me to build theoretical models of causality and tie project design choices (financial, structural, and content-related) to the capacity constraints identified.
ECOWAS building
Interview participant with poster
Editor’s Note
Lukas Müller is a recipient of MAXQDA’s #ResearchforChange Grant. He is a PhD student in Political Science at the Albert-Ludwigs-Universität Freiburg (University of Freiburg), Germany. His research project titled, “Whose Policy? The Policy Processes of the ECOWAS Commission and the ASEAN Secretariat and the Impact of External Capacity Building” is a comparative case study analysis across two continents. Müller has already completed the first half of his fieldwork in Abuja, Nigeria, and is now in Jakarta, Indonesia. Stay tuned for two more fieldwork diary entries by Lukas Müller coming soon!
