Professionalizing Evaluation
As the long-running debate continues in this year of evaluation, we ask if the time has come to professionalize.
As the long-running debate continues in this year of evaluation, we ask if the time has come to professionalize.
By: Caroline Heider
As the long-running debate continues in this year of evaluation, we ask if the time has come to professionalize.
About 25 years ago, when I became a full-time evaluator, evaluation journals like the American Journal of Evaluation frequently featured articles about the need to professionalize evaluation, and regularly discussed the merits of various accreditation systems and approaches to credentialing.
Now, all those years later, the debate continues. Some countries, most notably Canada, have moved on and have set up such systems. Elsewhere, professional associations at national, regional, and global levels have agreed on evaluation competencies as a foundation for professional standards.
The question is: are we now ready for the next step towards professionalization?
Looking at other professions it is quite easy to understand why professionalization is important. Would you want to attend a doctor and follow his advice if you were not confident he had the right qualifications? Would you hire a lawyer to defend you if you were unsure about her qualifications, and professional standing? In fact, absent the required accreditation, neither the doctor nor the lawyer could lawfully practice in most countries.
The rationale for professionalization is fairly simple: it legitimizes practitioners and aims to ensure that those using a professional service or taking professional advice can be sure of a certain minimum standard (or at least have defined recourse if the service is deficient). This provides a level of protection from potentially significant damages that may be associated with bad advice and malpractice.
The same level of risk is not as apparent for the consumers of evaluation. In fact, a perennial concern among practitioners is that our advice is often ignored, and does not, often enough, lead to desired learning and change. This observation and experience might possibly tempt us to underrate the risk associated with poor evaluation practice, and poor evaluations.
But, we should never under-estimate the consequences of poor quality evaluation. I know from experience and through colleagues - both in the evaluation field and on the operational side - that the risks and consequences are very real. The risks associated with poor evaluation quality, whether because of inadequate design, processes, or reporting, have first level effects on the evaluation office itself (efforts to rectify an evaluation gone awry can be an incredible drain on resources), and on the program or institution whose interventions have been evaluated, where ill-informed evaluations can damage reputation and lead to wrong decisions.
To ensure we "get it right" we have systems in place for quality assurance (including external evaluations of our work, client surveys, and a robust results framework), and we place a premium on the professionalism of our evaluators - which brings me back to the question of professionalization.
Like others, the World Bank Group adopted professional competencies for evaluators and created a work stream to help bring professionals together from evaluation and related fields to support professional development and networking. A great step in the right direction.
But professionalization will take more. The established professions, like the ones I mentioned earlier, have some features in common that provide food-for-thought:
Most of these professions are practiced within a country and are governed by national requirements. If, say a doctor, wants to move from one country to another, very often he or she will need new and different accreditation, including additional studies or tests to qualify.
The auditing profession operates globally and is a good comparator for evaluation. In that case, the profession established the International Organization of Supreme Audit Institutions (INTOSAI) more than 50 years ago.
However, there are important differences between the two professions. For example, unlike auditors, evaluators come from many different disciplines, subscribe to diverse methods, and can embrace different value systems. That diversity is both positive and essential to effective evaluation, but it makes it more complicated to codify professional standards, let alone introduce full professionalization.
Clearly there's more thinking and work to do. And I will be giving this subject more thought over the coming weeks as I prepare to present at a conference in Germany on the future of training and further education in evaluation. Why not help me out and let me know what you think should be the next steps?
Comments
Add new comment