Evaluation may provide information relevant to broader public processes such as program improvement, decision-making or accountability. Nonetheless, evaluation’s positive contribution to these goals may be affected by resistance from stakeholders associated with the program being evaluated.
Resistance to evaluation may take various forms at different stages of an evaluation initiative. Prior to beginning the evaluation, resistance may result from anxiety about the process or an anticipation of the effects of evaluation. For example, program managers may be afraid that the evaluation will involve a judgment on the quality of their work. Resistance could manifest itself as a set of arguments around the program specific characteristics, contesting the relevance of conducting an evaluation.
During evaluation, resistance may affect – passively or actively – the conduct of the evaluative process. Resistant stakeholders may try to generate barriers to the implementation of evaluation activities, for instance by refusing to participate, distorting information or altering the evaluation procedures. Resistance may arise, among other sources, from first line intervening actors who consider the exercise a waste of time affecting the fulfillment of their mission. It may also arise from program managers who question the timing of the evaluation in relation to their project management agenda. At the end of the evaluation initiative, resistance may involve challenging the legitimacy of the evaluation results and discrediting the evaluation by attacking the methodology or skills of the evaluator. Again here, an anticipation of the effects of evaluation may cause resistance. For example, a director may want to preserve the notoriety of its organization and perceive that evaluation results will feed critics outside the organization or potentially involve cut in program funding. Municipal actors may fear that evaluation results affect their city’s reputation. Indeed, evaluation – like other organizational processes – is at the centre of different interests and information needs that may contradict or complement each other.
A good understanding of the dynamics associated with resistance and propensity to evaluation is important. It can help decision-makers and managers, as sponsors of evaluation processes, to develop evaluation procedures and strategies minimizing resistance and maximizing propensity – that is to say, a positive inclination toward evaluation – from stakeholders.
Building on empirical examples, we discuss prominent factors influencing stakeholders’ perceptions as well as suggestions to prevent resistance and encourage the development of a positive orientation toward evaluation.
Finding the Path of Least Resistance in Evaluation
Factors at the individual and contextual levels, as well as the characteristics of the evaluation process, may have an impact on stakeholders’ perceptions regarding evaluation.
Personal characteristics such as a fear of criticism or a negative experience in previous evaluation processes may influence individual feelings and expectations with regard to the evaluation process. For instance, in a study involving professionals from a health institution in Haiti, Pernelle Smits identified factors influencing propensity among the group of participants. Individuals characterized with a higher propensity for evaluation had past positive experiences linked with evaluation activities, professional responsibilities involving a supporting role for evaluation, and identified themselves with individual and organizational performance. They were also giving a particular importance to quality of work and had a strong orientation toward processes and the possibility to act upon them, as well as a longer-term perspective on results.
From another standpoint, a study in two public sector organizations in the United Kingdom identified barriers to evaluation associated with the organizational level. The researchers identified obstacles such as a lack of mutual trust between stakeholders and the evaluator, diverging interests among stakeholders, and a perception by actors at lower hierarchical levels that evaluation was an imposed and rigid process. Their results demonstrated that some characteristics of the organizational environment may contribute to inhibiting evaluation, such as prevailing institutional values regarding evaluation and the organizational and relational environment in which evaluation is implemented.
At the evaluation level, the role of the evaluator, the choice of an evaluation approach, the design of data collection instruments and the quality of information produced by the evaluation may influence stakeholders’ propensity or resistance. In a study reporting on the evaluation of a training program within an organization with established evaluation practices, a team of evaluators noted changes following the adoption of a new national vocational training policy. They observed how this external factor generated uncertainty regarding the future of the program and created fear regarding the potential consequences of the evaluation process in case of negative findings. As a result, it contributed to the development of resistance to the evaluation process. In this context, drafting a contract between the evaluators and program stakeholders contributed to reducing participants’ anxiety. Under this contractual agreement, both parties committed themselves to self-examination and mutual consultation upon manifestations of resistance. They also agreed on creating spaces for open discussion on the objectives of the evaluation, and exchange of ideas and concerns regarding the evaluation, its results and controversial issues.
The main lesson learned in this case concerned the relevance of supplementing the standard evaluation contract with a formalized agreement between stakeholders and the evaluator covering technical and ethical considerations. This recommendation also applies in situations where evaluation is an established practice in the organization. The occurrence of an unforeseen event (change in the policy environment) contributed to changes in stakeholders’ perceptions. The agreement provided a formal space for specifying collaboration rules related to problem resolution, communication lines and the nature of interactions in the context of the evaluation process.
Evaluation: It’s the Trust Thing
The diversity of contexts in which evaluation is implemented prevents from adopting a “one-size-fits-all” solution to support stakeholders’ propensity for evaluation. Nevertheless, some general principles issued from research combining social and organizational psychology as well as evaluation can contribute to decision-making in these situations. First, situations in which individuals feel that evaluation represents a threat to their power, control, autonomy and self-image generate negative feelings about evaluation and increase the risks of developing resistance. Second, previous experiences with evaluation or similar processes have an influence on perceptions toward evaluation. Accordingly, positive experiences with evaluation will contribute to future positive anticipation about the role of evaluation. Third, conflict and competition related to individual and organizational goals may influence stakeholders’ attitudes and behaviours toward evaluation. Facilitating the development of a trusting relationship (among stakeholders and with the evaluator) and identifying common goals to which evaluation is contributing may provide conditions for a stronger support to the evaluation process.
To support stakeholders’ buy-in, different measures can be implemented before (or at the planning stage), during and after the evaluation process. At the beginning of an evaluation, tools such as evaluability assessments may help identify upstream relevant factors that could influence stakeholders’ resistance or support to the evaluation process. Taking time to understand manifestations of resistance (at any time during the evaluation) to discover their underlying causes and then identifying appropriate ways to deal with them is a key aspect to a successful management of the evaluation process.
From another standpoint, increased stakeholders’ participation in planning and technical decision-making regarding evaluation can contribute to generating positive feelings toward evaluation as participants acquire a greater knowledge of evaluation’s activities and purposes. It is also a good idea to begin with involving the most supportive or enthusiastic stakeholders in the evaluation process. They may act as change agents and contribute to creating a virtuous circle around evaluation. Participation is also relevant in the final phases of the evaluation process when presenting and diffusing results. Generating spaces for discussions and identifying learning opportunities on the basis of the evaluation’s results creates a stronger appropriation and increases the likeliness that key stakeholders will support the implementation of the recommendations from the evaluation.
Marie-Hélène L’Heureux is a doctoral candidate in Political Science at Laval University (Québec City, Canada) and a member of the Canadian Evaluation Society (CES). She is also a research assistant at PerfEval, a research laboratory on public policy performance and evaluation (www.perfeval.net).
Steve Jacob is a member of the Canadian Evaluation Society (CES), a full professor in the Department of Political Science at Laval University (Québec City, Canada), and the director of PerfEval, a research laboratory on public policy performance and evaluation (www.perfeval.net).
Pernelle Smits is past president of SQEP and acted as a board member of IOCE and RFE. She is a professor at the Department of Management at Laval University, teaches evaluation in China, Africa, among others, and supports the emergence and reinforcement of national evaluation systems.
