Canadian Government Executive - Volume 24 - Issue 03

May/June 2018 // Canadian Government Executive / 33 Evaluation from a health institution in Haiti, Pernelle Smits identified factors influencing pro- pensity among the group of participants. Individuals characterized with a higher propensity for evaluation had past posi- tive experiences linked with evaluation activities, professional responsibilities in- volving a supporting role for evaluation, and identified themselves with individual and organizational performance. They were also giving a particular importance to quality of work and had a strong orien- tation toward processes and the possibil- ity to act upon them, as well as a longer- term perspective on results. From another standpoint, a study in two public sector organizations in the United Kingdom identified barriers to evaluation associated with the organizational level. The researchers identified obstacles such as a lack of mutual trust between stake- holders and the evaluator, diverging in- terests among stakeholders, and a percep- tion by actors at lower hierarchical levels that evaluation was an imposed and rigid process. Their results demonstrated that some characteristics of the organizational environment may contribute to inhibiting evaluation, such as prevailing institutional values regarding evaluation and the orga- nizational and relational environment in which evaluation is implemented. At the evaluation level, the roleof the eval- uator, the choice of an evaluation approach, the design of data collection instruments and the quality of information produced by the evaluation may influence stakeholders’ propensity or resistance. In a study report- ing on the evaluation of a training program within an organization with established evaluation practices, a team of evaluators noted changes following the adoption of a new national vocational training policy. They observed how this external factor generated uncertainty regarding the future of the program and created fear regarding the potential consequences of the evalua- tion process in case of negative findings. As a result, it contributed to the development of resistance to the evaluation process. In this context, drafting a contract between the evaluators and program stakehold- ers contributed to reducing participants’ anxiety. Under this contractual agreement, both parties committed themselves to self- examination and mutual consultation upon manifestations of resistance. They also agreed on creating spaces for open discus- sion on the objectives of the evaluation, and exchange of ideas and concerns regarding the evaluation, its results and controversial issues. The main lesson learned in this case concerned the relevance of supplementing the standard evaluation contract with a for- malized agreement between stakeholders and the evaluator covering technical and ethical considerations. This recommenda- tion also applies in situations where evalu- ation is an established practice in the orga- nization. The occurrence of an unforeseen event (change in the policy environment) contributed to changes in stakeholders’ perceptions. The agreement provided a for- mal space for specifying collaboration rules related to problem resolution, communica- tion lines and the nature of interactions in the context of the evaluation process. Evaluation: It’s the Trust Thing The diversity of contexts in which evalua- tion is implemented prevents from adopt- ing a “one-size-fits-all” solution to support stakeholders’ propensity for evaluation. Nevertheless, some general principles is- sued from research combining social and organizational psychology as well as eval- uation can contribute to decision-making in these situations. First, situations in which individuals feel that evaluation rep- resents a threat to their power, control, au- tonomy and self-image generate negative feelings about evaluation and increase the risks of developing resistance. Second, previous experiences with evaluation or similar processes have an influence on perceptions toward evaluation. Accord- ingly, positive experiences with evalu- ation will contribute to future positive anticipation about the role of evaluation. Third, conflict and competition related to individual and organizational goals may influence stakeholders’ attitudes and be- haviours toward evaluation. Facilitating the development of a trusting relation- ship (among stakeholders and with the evaluator) and identifying common goals to which evaluation is contributing may provide conditions for a stronger support to the evaluation process. To support stakeholders’ buy-in, differ- ent measures can be implemented before (or at the planning stage), during and after the evaluation process. At the beginning of an evaluation, tools such as evaluability assessments may help identify upstream relevant factors that could influence stake- holders’ resistance or support to the evalu- ation process. Taking time to understand manifestations of resistance (at any time during the evaluation) to discover their underlying causes and then identifying appropriate ways to deal with them is a key aspect to a successful management of the evaluation process. From another standpoint, increased stakeholders’ participation in planning and technical decision-making regarding evaluation can contribute to generating positive feelings toward evaluation as participants acquire a greater knowledge of evaluation’s activities and purposes. It is also a good idea to begin with involv- ing the most supportive or enthusiastic stakeholders in the evaluation process. They may act as change agents and con- tribute to creating a virtuous circle around evaluation. Participation is also relevant in the final phases of the evaluation process when presenting and diffusing results. Generating spaces for discussions and identifying learning opportunities on the basis of the evaluation’s results creates a stronger appropriation and increases the likeliness that key stakeholders will sup- port the implementation of the recom- mendations from the evaluation. M arie -H élène L’H eureux is a doctoral candidate in Political Science at Laval University (Québec City, Canada) and a member of the Canadian Evaluation Society (CES). She is also a research as- sistant at PerfEval, a research labora- tory on public policy performance and evaluation (www.perfeval.net) . S teve J acob is a member of the Ca- nadian Evaluation Society (CES), a full professor in the Department of Political Science at Laval University (Québec City, Canada), and the director of PerfEval, a research laboratory on public policy performance and evalua- tion (www.perfeval.net) . P ernelle S mits is past president of SQEP and acted as a board member of IOCE and RFE. She is a professor at the Department of Management at Laval University, teaches evaluation in China, Africa, among others, and supports the emergence and reinforcement of national evaluation systems.