Conducting systematic reviews in psychology: issues, challenges, and opportunities - Mohd Mahudin, Nor Diana (2009)
This paper discusses the issues and challenges associated with conducting systematic reviews using the existing guidelines (e.g. the Cochrane and Campbell systematic reviews). While these guidelines are appropriate for reviews involving predefined study criteria with fixed procedures and techniques or concerning assessment of effect sizes in established interventions; a more flexible approach is recommended when conducting reviews that deal with evidence from non-randomized, qualitative, or mixed-methods studies. Based from cumulative experience gained from conducting systematic reviews of two ongoing projects, the authors identified seven broad challenges. These challenges were found mostly in 1) defining the research scope and formulating the research question(s); 2) determining the search strategy and adhering to the disciplinary syntax; 3) accessing relevant literature; 4) filtering, appraising, and prioritizing the evidence from different study designs- particularly those related to primary qualitative and mixed-methods studies; 5) combining meta-analysis and narrative synthesis in evidence synthesis; 6) managing time and resources constraints; and 7) enhancing the transparency of decision making process. This paper maintains that while the essence and principles of the existing systematic reviews guidelines should be adhered to, researchers need to adapt them in a more complex nature when dealing with evidence from non-experimental studies or qualitative research. It is suggested that such systematic review process may, among other things, entail development of search protocol using stepwise refinement, integration of both narrative synthesis and meta-analysis for synthesizing the evidence, employment of multi-disciplinary team, and approach to ensure transparency and rigour.