The Government of Canada’s newly announced Policy on Results replaces the former Policy on Evaluation. One new and welcome feature is that it declares in its Directive on Results that heads of evaluation are responsible for “ensuring that departmental evaluators have opportunities to develop their competencies and to earn evaluation-related designations or certifications from recognized professional associations and certifying bodies” (paragraph 4.4.12).
This new policy is the first time that that the professionalization of evaluation through the use of competencies and certification has been mentioned in the federal evaluation context. While there has been recognition in past policies for heads of evaluation to possess relevant competencies, this expansion to all federal evaluators and reference to evaluation-specific designations or certifications represent important steps forward in the professionalization of the federal evaluation function.
It is a milestone. But how do you develop your evaluation skills and seek certification?
Golf Anyone?
Practicing evaluation is a lot like golf. First of all, you need to train to improve your skills, getting out on the course at least once or twice a week. Each time you swing that club, you learn something new and the more you swing, the better you get. Evaluation is the same. You have to do a lot of evaluations to really understand what is involved.
Golf requires many tools—driving irons, long irons, short irons, putters. They are all important to the quality of your game and that game is comprised of many different processes. Evaluation also requires a whole kit bag, from qualitative to quantitative methods, from formative to summative approaches, you need to know what pieces to use and how to fit them together to create a competent and useful evaluation.
So what do you do when you need to improve your game? In golf, you can choose to play with a friend who has better skills than you have. In evaluation, you can find a professional colleague to act as a mentor, guiding you through the many aspects of practice that comprise the whole.
Or you could take lessons from a pro! This could be the fastest and most reliable way to improve your skills. In evaluation, the competencies identified by the Canadian Evaluation Society (CES) can be used as a guide to identify areas where professional development is needed. Once you have identified your skill gaps, you can find the appropriate training.
Probably, not too many of us will become golf pros, but we can still reduce our handicap. Evaluators are luckier. They can be recognized by their peers for having attained the skills needed to be deemed a competent evaluator. The Canadian Evaluation Society’s Credentialed Evaluator (CE) designation indicates that those skills have been acquired (see evaluationcanada.ca/ce).
A Case in Point
Here’s how one federal government group is striving to improve its evaluation capacity. The Public Health Agency of Canada (PHAC) and Health Canada (HC) are served by one evaluation team, approximately thirty evaluators in the Office of Audit and Evaluation. This team has always had a strong focus on continuous professional development. For example, attendance at CES workshops and conferences, coaching and mentoring by senior members, and “lunch and learn” events have contributed to the development of team members’ skills. All of these strategies served the team well, but there still was a desire for more formal recognition of the team’s competence.
An important realization occurred in July 2015 when the evaluation team was combined with the audit team in one corporate office. It became evident that the federal audit function was more mature than evaluation in terms of its approach to professionalization. Audit designations were recognized in job descriptions and their pursuit and maintenance were supported by federal policy. This was not yet true for evaluation.
As a result, a business case was put forward to senior management to have the same recognition and support for evaluation designations that was already available for auditors. The concept was approved in March, 2016. Immediately, a study group was created to support interested team members’ pursuit of the CE designation. Speakers were invited to discuss such topics as the credentialing process, what Credentialing Board members were looking for in a CE application, and what CEs themselves had to say about their experience. The group continues to meet to support evaluators as they work through their application portfolios and the team is excited about the number of designations that are underway or have already been attained.
Now What?
In 2012/13, under the 2009 federal policy, there were 459 full time equivalents in the Federal Government involved in the completion of 123 evaluations at a total cost of $56.2 million including salary, O&M and professional services. With the strong focus on results and additional flexibilities in the new policy, and the important role that evaluation can play in giving decision makers access to the best possible information necessary, the potential consequences could be significant as all federal agencies aim to improve their evaluation capacity.
But this is only the beginning of the story. As evaluators work towards greater competence, they will discover, just as other professionals have, that the emphasis then shifts to continuing education. In the case of the CE, maintenance requirements demand the completion and reporting of 40 hours of professional development every three years. Early feedback suggests that evaluators actually welcome this requirement because they can legitimately pursue their own continued improvement, actively seeking out appropriate training. Hopefully, as these activities drive home the professionalization of Canadian federal evaluators, it will contribute to the increased recognition of evaluation as an important profession in government and this in turn will lead to better outcomes for federal program participants.
Harry Cummings, PhD, RPP, Shelley Borys, PhD, CE, Gail Vallance Barrington, PhD, CE
