Blog

Today’s blog entry is the second article about the evaluation currently underway of an online community of practice – MHPSS.net – whose research team is utilizing MAXQDA for their coding and analyzing needs (go to previous article). The MHPSS.net platform is an online community of practice developed out of the vision to connect practitioners, policy makers, and researchers around the world working in the field of Mental Health and Psychosocial Support in and following situations of emergency and adversity. This article describes how the team used the “intercoder agreement” function of MAXQDA 12 during their initial coding process.

Using “Intercoder Agreement” to Improve the Code System

After completing transcription of the interviews, the research team began the coding process. Two researchers were specifically dedicated to coding, under the supervision of a third researcher. First, a unified list of codes was developed based on research questions and the themes that were arising from the initial review of interviews. The codes fell into areas of interest such as “purpose of mhpss.net” or “site functions most useful.” Code definitions were created for each code, using the code memo feature.

Next, two researchers coded the same five interviews and utilized the inter-coder agreement function on these interviews. The researchers were allowed to discuss the coding process while coding these interviews, to clarify any confusions or misunderstandings. When utilizing the inter-coder agreement, the code existence and segment agreement were calculated. While ‘code existence’ provided an overview of comparison between the two researchers’ coding, the ‘segment agreement’ was used to do an in-depth review of the coding. It was realized that some codes were underutilized, while other codes’ meaning was understood differently. Based on these results, the researchers discussed their individual coding styles and decided how to ameliorate differences. Code definitions were refined further, as a result, and new codes were added where necessary.

After this point, the two researchers divided up the rest of the interviews’ coding amongst them. Both of them worked on different project files, and at the end, the two projects were merged together.

Coding and Analysis

The improved code system formed the basis for a more structured coding process. As shown in the photo below, areas of interest were color coded into groups for ease of identification. Certain sections of texts fell into multiple areas of interest and therefore were given multiple codes.

Once all interviews were fully coded, analysis began. Overall, positive and negative perceptions around site functions of the MHPSS network emerged as themes. Perceptions and experiences shared by members, hosts, and advisory board members, along with suggestions, were analyzed. In addition, we also found that there is a strong perception by members, hosts and advisory board members alike that the MHPSS Network has yielded positive individual and organizational level outcomes through increased knowledge from resources and connections made available by the network’s online learning platform.

The evaluation validated the importance of the MHPSS Network for the MHPSS sector and MHPSS actors around the globe. Despite challenges faced and the need for ongoing work to improve its success, the network presents as an influential platform to construct and share knowledge around MHPSS.

About the author

This blog post was a collective effort by the MHPSS.net Independent Evaluation Team, represented by Zeinab Hijazi.

Zeinab has over 9 years of experience supporting MHPSS programs in the Middle East and North Africa region, providing guidance and oversight in the development, monitoring, evaluation and running of culturally appropriate mental health and psychosocial activities, including provision of mental health care, psychosocial support, and protection activities targeting various vulnerable groups, including children, among internally displaced and vulnerable host populations.