What Constitutes Capacity Development?

The concept of capacity development has a long history, having emerged in the literature in its current format in the late 1980s, but with origins in the 1950s, when development finance institutions started providing technical assistance for institution building, equipping local public institutions with financial resources and infrastructure to administer public investment (Vallejo and Wehn 2016). Over the decades, terminology evolved from institutional building to a focus on institutional development, capacity building, and capacity development, reflecting a greater emphasis in development aid toward a more people-centric and demand-driven approach. If, for example, institutional development emphasizes “creating or reforming organizations and institutions…to improve public sector effectiveness,” capacity building emphasizes “building individuals’ abilities and organizations’ capabilities to undertake different tasks in various development sectors” (Kacou et al. 2022, 222). Capacity development shifted this focus from building to strengthening individual and organizational skills and knowledge to acknowledge that these interventions aim to enhance existing capacities, not necessarily build them from zero (see Brinkerhoff and Morgan [2010] for a discussion).

Despite a strong consensus on its crucial role in development and its frequent use, capacity development has no standard definition, nor are the types of interventions that constitute it standard. In part, this is because of the multidimensional, multistakeholder, multilevel nature of capacity development, ranging from individual-level change in skills and knowledge to organization-level and even systemic-level changes in norms, processes, policies, or structures (Christoplos et al. 2014; Kacou et al. 2022; Ubels et al. 2010). In addition, because capacity development targets systemic and participatory goals, it becomes increasingly relational and dynamic and requires encompassing both tangible and intangible aspects of capacity (Ubels et al. 2010).

Seeking to encompass such variation and complexity under a single overarching conceptual umbrella can lead to weak theoretical underpinnings and risk conceptual ambiguity. In response, academics and practitioners have developed several conceptual frameworks and approaches that highlight different aspects of capacity development or operationalize it with a focus on distinct dimensions, with different degrees of overlap and complementarity. For example, as discussed in the Independent Evaluation Group (IEG) Evaluation Insight Note on capacity development, the World Bank does not have an overall framework, but it often productively relied on the Institutional Change Assessment Method to assess institutional capacity development along three dimensions (see, for example, Otoo et al. 2009; World Bank 2013, 2018, 2021): strengthening ownership by interest groups, reforming policy instruments, and enhancing organizational arrangements (World Bank 2023). Ubels et al. (2010) discuss two influential practitioner frameworks. The first is Kaplan (1999) and the Community Development Resource Association’s six elements of organizational capacity, which emphasize the complexity, systemic nature, and visible and invisible dimensions of capacity, while also distinguishing between loosely hierarchical organizational subsystems. The second is the European Centre for Development Policy Management’s 5Cs framework (Baser and Morgan 2008), which identifies five core capabilities applicable to any level of a human system as key: the capability to act and self-organize, to generate development results, to relate, to adapt and self-renew, and to achieve coherence.

In addition, as noted in the introduction, development finance institutions use many different terms to refer to capacity development and capacity building (refer to box 1.1 for definitions from select institutions). What constitutes a capacity development intervention also varies across these institutions (refer to box 1.2 for examples of capacity-building activities). Some are internal to the institutions and focus on improving their understanding, knowledge, and business; others target the institutions’ external clients or audiences (for example, public institutions, private firms, academia, and the public). In this paper, we focus on client-facing capacity development.

Within the World Bank, advisory products or services for clients are known as advisory services and analytics. They aim to support clients and help them design or implement more effective policies; strengthen their institutions or organizations; and inform their development strategies, operations, or financing (World Bank, n.d.). Deliverables related to advisory services and analytics include reports on key economic, social, and sectoral issues; project-related advice and technical assistance; policy notes and presentations; impact evaluations; and knowledge-sharing workshops, conferences, or training programs (World Bank, n.d.).

The International Finance Corporation (IFC) defines advisory services as engagements in which “IFC provides a client with advice (for example, consulting services, technical assistance) on a specific subject” (IFC 2023, 1). IFC’s clients are either private sector companies or governmental or nongovernmental institutions or organizations, such as ministries, municipalities, central banks, and business associations, among others.

IFC advisory services projects have a wide scope and fall into three main categories (IFC 2023; refer to box 1.3 for examples of each):1

• Enabling the environment for private sector activities, which includes a particular sector’s legal or policy framework, building capacity of public institutions serving the private sector, or facilitating private sector entities’ access to information or services
• Firm-level advisory services, which include support to private sector companies to enable them to enhance their operational capacity and efficiency, improve financial sustainability, manage risks better, and improve their development impact
• Transaction advisory services, which include advice to public sector entities on public-private partnerships and support to private sector companies regarding partnerships, mergers, and acquisitions

How Do Development Finance Institutions Evaluate Work in Client-Facing Capacity Development?

To accommodate the variety and scope of capacity development activities, major bilateral, multilateral, and international organizations have developed several frameworks and methodologies for evaluating capacity-building activities (ECG 2012). Most of these frameworks are rooted in criteria established by the Organisation for Economic Co-operation and Development’s Development Assistance Committee: relevance, coherence, effectiveness, efficiency, impact, and sustainability. These criteria are also used for assessing capacity-building work and the performance of technical assistance operations. For example, the Asian Development Bank and the International Monetary Fund (IMF) measure the performance of their technical assistance operations based on assessments of relevance, effectiveness, and efficiency, each with a corresponding rating scale.

Approaches of this type have several advantages: a standardized and widely accepted methodology, a structured results chain, and comparability across different capacity-building initiatives. Nevertheless, the methodology used in such approaches also presents several downsides—it can be rigid and is not always readily adaptable to specific contexts, and it tends to work well when assessing the delivery of outputs but is more challenging when assessing the contribution to outcomes. It also has weaknesses in capturing intangible changes such as behavioral shifts or alterations in institutional culture, which are critical for capacity development. For example, a 2018 IMF review found that the organization needed to shift focus to results in the field rather than outputs (IMF 2018). A 2022 evaluation by IMF’s Independent Evaluation Office found that none of the systems then in place could provide a full picture of the contributions, outcomes, and impact of the IMF’s capacity development (IEO 2022). Similarly, the European Investment Bank’s 2023 evaluation of its technical assistance found that the bank’s systems for monitoring and evaluating advisory activities could not capture fully the impact of advisory support on securing investment. This was partly due to the typical time lag between delivery of outputs and realization of results and conceptual issues in establishing attribution and causality (EIB Group 2023).

Alternative approaches for measuring the effectiveness of capacity development interventions have been developed over the years. One of these approaches is the Inter-American Development Bank’s Development Effectiveness Matrix for knowledge and capacity work (IDB 2008). Whereas the Inter-American Development Bank follows the Organisation for Economic Co-operation and Development criteria for measuring the performance of its lending projects and country programs, it takes a different approach to evaluating the performance of its knowledge and capacity-building products. Its common performance metric uses four mission-specific dimensions: relevance, policy development, knowledge management, and customer service. However, depending on the knowledge product type (client-centered, outreach-centered, policy development, knowledge management, or strategic development), a relative weight is assigned for each mission-specific dimension. For example, if a knowledge and capacity-building product is derived from short-term, client-driven needs, then performance is measured by delivery speed and customer satisfaction, and more weight is assigned to the relevant mission dimension, customer service. In contrast, long-term and client-driven needs are primarily assessed based on some combination of the policy development dimension, the quality of the analytical work performed, and the degree to which it helps shape the formulation and implementation of the policy and build the required consensus for policy decision-making.

As noted earlier, the Evaluation Cooperation Group has not developed Good Practice Standards for advisory services. Nevertheless, many development finance institutions have frameworks in place to evaluate capacity development activities, but systematically assessing the performance and impact of technical assistance interventions can pose a challenge in capturing both changes in outcomes and the interventions that drive these changes. First, understanding what results to measure and how to measure them—that is, conceptualizing, operationalizing, and measuring outcomes—is seldom straightforward. Outcomes of capacity- and institution-building activities are complex and varied, making them hard to quantify or capture under simple metrics that can be applied across diverse potential changes. The longer time frames allotted or expected for observing impacts introduce further complexity, as trajectories of change for capacity development interventions are seldom linear, and reaching improvements depends on a complex combination of conditions.

Such conceptualization is necessary because indicators of results tracked for advisory services and analytics are not always comprehensive. For example, at the World Bank, self-assessment comprises the ratings and information provided by project team leaders in the projects’ activity completion summaries to indicate the achievement of outcomes. Indicators of results are tracked for programmatic advisory services and analytics and some donor-funded activities. Such tracking is optional, however, for all tasks related to other advisory services and analytics. The World Bank does not evaluate each project involving advisory services and analytics, and the results of such services and analytics are not reported at the corporate level (World Bank 2016, 2019). Independent evaluation at the World Bank does not evaluate or validate advisory services and analytics unless they are part of a country, sector, or thematic evaluation.

Second, assessing impacts requires establishing a robust link between any changes in outcomes and the capacity development interventions themselves. A large number of capacity development interventions, with activities ranging from small and rapid to very large, programmatic, multicountry, or long term, may be ongoing and targeting similar development outcomes. A single advisory services intervention is unlikely to trigger or significantly contribute to change on its own. Targeting the right package and sequence of interventions at the right level of analysis is critical yet also challenging to assess because of this heterogeneity in interventions.

In the rest of this paper, we provide an overview of the key elements and processes underlying IFC’s self-evaluation and IEG’s independent validation. We then discuss methodological limitations to systematic learning and accountability from stand-alone advisory services and analytics and reflect on the practical approaches IEG has developed to address these limitations.