Canadian Government Executive - Volume 24 - Issue 01

22 / Canadian Government Executive // January/February 2018 Program Evaluation Who Cares? Principled Evaluation: By Wayne MacDonald MA, CEP, Emilie Peter, PhD, Natalie Kishchuk PhD, CE, FCES Compared to audit professionals and their preoccupation with “independence”, evalu- ation specialists may seem unconcerned about ethical principles. Their fixation is on systematic processes, methods and techniques. Principles underlying evalu- ations rarely come up in evaluation plan- ning or implementation, and they seldom figure in evaluation reports or follow-ups. Give a workshop to evaluators on “big data” or “data visualization” and you can fill an auditorium. Have a session on evalua- tion principles and ethics and the room is mainly empty. So, what’s going on? Perhaps the absence of discussion leads evaluators to conclude that ethical challenges do not exist. They then infer that they and their field must be, by their very nature, prin- cipled and ethical. Suspecting that it’s not that simple, we feel it’s time to ask: What dialogue on evaluation principles and eth- ics ever occurs? What ethical challenges are evaluators encountering? What are they doing about them? As are any professionals, evaluators are first and foremost defined by the central val- ues they hold. If their central tenets go unno- ticed, then their practice fades into a parade alongside every other activity in the scientif- icmanagement field. Indeed, no profession is immune from this risk. For example, a study of almost 1000 physicians in several coun- tries found that although they were aware that their profession had a code of ethics, al- most to a person they admitted to not refer- ring to it and not using it. The time is right for the Canadian evaluation community to have a robust and vital discourse on the ‘ties that bind’ us together as professionals, and that set the stage for principled practice. In the absence of such, a key cornerstone of evalua- tion excellence may begin to erode. To understand how evaluators imple- ment ethics in their day-to-day practices, surveys were conducted of Canadian Evaluation Society (CES) members in 2010 (response rate 26 per cent, n = 455) and 2015 (response rate 12 per cent, n = 218). The 2015 study included qualitative case studies of eight evaluations conducted be- tween 2013 and 2016. See below for a sum- mary of the results from these studies. Ethics is a serious issue for Canadian evaluators. They associate ethics with ideas about integrity, quality and impar- tiality, values that are important to them as professionals. Start a dialogue about principles and ethics with evaluators and it quickly descends down a long chute of abstract concepts – accountability, compe- tence, confidentiality, impartiality, inde- pendence, integrity, neutrality, objectivity, subjectivity, and the list goes on. Comparing the 2010 and 2015 surveys sug- gests that evaluators are becoming more aware of and can identify more challenges in their work. As the 2010 and 2015 studies attest, the power of those who commission evaluations and their influence over evalu- ators’ treatment of ethics during the evalu- ation process is substantial. Evaluators are frequently called upon to deal with direct pressures from stakeholders to modify the evaluation process or findings. This can lead to potential bias in the evaluation, especial- ly in terms of impartiality. It also increases evaluators’ anxiety. Evaluation clients, those who commission evaluations, and evalua- tors themselves hold important roles and responsibilities with regard to ensuring principled, ethical practice. At the same time, evaluators seem shy to speak out on this topic. This has much to do with sensitivities related to conflicts of in- terest that are perceived to affect employ- ment circumstances and opportunities. While evaluators do welcome occasions to provide feedback, there is a profound sen- sitivity around full ethical disclosure. The 2010 survey revealed that three- quarters or 77 per cent of Canadian evalu- ators encountered at least one perceived ethical challenge in their work, but only 57 per cent were willing to provide a de- scription or example. For them, the setting for such a discussion needs to be right, and the space safe. Both the 2010 and 2015 studies reveal that colleagues (professional evaluators or otherwise) are an important source of reflection and advice, i.e., a trust- ed viewpoint in a safe space. As participants in the 2015 study observed, discussion about ethical challenges in evalu- ators’ daily practice is not systematic. It rais- es the question of the extent to which evalu- ators can and feel comfortable raising these ethical issues with the client before, during, and or after the evaluation process. The 2010 survey also revealed that there is room for a more active role from the Ca- nadian Evaluation Society at 62 per cent. A vacuum currently exists and is something only the national association can fill. This may include professional development – workshops, formal training; regular surveil- lance among members; greater emphasis on ethics in the Credentialed Evaluator des- ignation; more space in professional publi- cations (e.g. Canadian Journal of Program Evaluation), and more dialogue at chapter level events and national conferences. The Canadian Evaluation Society is in- deed now taking on this role, starting with a review of its now 20-year old Ethics Prin- ciples of Competence, Integrity and Ac- countability. Since that document was pro- The time is right for the Canadian evaluation community to have a robust and vital discourse on the ‘ties that bind’ us together as professionals, and that set the stage for principled practice. In the absence of such, a key cornerstone of evaluation excellence may begin to erode.