Background: Given the underinvestment in global mental health to-date, it is important to consider how best to maximize the impact of existing investments. Theory of Change (ToC) is increasingly attracting the interest of funders seeking to evaluate their own impact. This is one of four papers investigating Grand Challenges Canada’s (GCC’s) first global mental health research funding portfolio (2012–2016) using a ToC-driven approach.
Methods: A portfolio-level ToC map was developed through a collaborative process involving GCC grantees and other key stakeholders. Proposed ToC indicators were harmonised with GCC’s pre-existing Results-based Management and Accountability Framework to produce a “Core Metrics Framework” of 23 indicators linked to 17 outcomes of the ToC map. For each indicator relevant to their project, the grantee was asked to set a target prior to the start of implementation, then report results at six-month intervals. We used the latest available dataset from all 56 projects in GCC’s global mental health funding portfolio to produce a descriptive analysis of projects’ characteristics and outcomes related to delivery.
Results: 12,999 people were trained to provide services, the majority of whom were lay or other non-specialist health workers. Most projects exceeded their training targets for capacity-building, except for those training lay health workers. Of the 321,933 people screened by GCC-funded projects, 162,915 received treatment. Most projects focused on more than one disorder and exceeded all their targets for screening, diagnosis and treatment. Fewer people than intended were screened for common mental disorders and epilepsy (60% and 54%, respectively), but many more were diagnosed and treated than originally proposed (148% and 174%, respectively). In contrast, the three projects that focused on perinatal depression exceeded screening and diagnosis targets, but only treated 43% of their intended target.
Conclusions: Under- or over-achievement of targets may reflect operational challenges such as high staff turnover, or challenges in setting appropriate targets, for example due to insufficient epidemiological evidence. Differences in delivery outcomes when disaggregated by disorder suggest that these challenges are not universal. We caution implementers, funders and evaluators from taking a one-size-fits all approach and make several recommendations for how to facilitate more in-depth, multi-method evaluation of impact using portfolio-level ToC.
Bibliographical noteFunding Information: The collection and analysis of data for this research were funded by GCC through the MHIN innovator support platform. GCC participated in the design of the portfolio-level Theory of Change and collected the quantitative Core Metrics data that were forwarded to MHIN for data cleaning and analysis. However, the funder played no role in the analysis or interpretation of data, the writing of the manuscript or the decision to submit for publication.
The authors have previously received GCC funding, through the MHIN innovator support platform (GKR, GLE, GME, JE, LL, MDS, MR, OQ, SU, TE) and/or through a GCC grant for a project included in this dataset (JM, OQ).
Open Access: This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher Copyright: © The Author(s) 2021.
Citation: Esponda, G.M., Ryan, G.K., Estrin, G.L. et al. Lessons from a theory of change-driven evaluation of a global mental health funding portfolio. Int J Ment Health Syst 15, 18 (2021).
- Global mental health
- Theory of change