The Problem of Intercoder Agreement in Qualitative Research

When assigning codes to qualitative data, it is recommended that certain criteria be set. You assume, for example, that coding is not arbitrary or random, but rather that a certain level of reliability will be reached. The MAXQDA “Intercoder agreement” function makes it possible to compare two people coding the same document independently of each other. In qualitative research, the purpose of comparing independent coders is to discuss the differences, figure out why they occurred, and learn from the differences in order to improve coding agreement in the future. In other words, the actual percentage of agreement is not the most important aspect of the tool. This percentage is, however, provided by MAXQDA.

It is always the goal of qualitative analysts to achieve as high a level of agreement as possible between independent coders. It is not, however, focused on getting to a standard coefficient that is statistically necessary as in quantitative research. Rather, the focus is on a practical improvement of the coding quality. For this reason, one doesn’t focus on the coefficient or percentage of agreement. Instead, qualitative researchers want to address and edit the coded segments that do not agree, so that they can move on with more accurately coded material.

The Intercoder Agreement tool is set up to compare and contrast a single document coded by two independent coders, highlighting the areas where the two researchers’ coded segments align and do not align. It is then possible for these two coders (or a larger research team) to decide which coding variation in each situation is most accurate.

Was this article helpful?