Reaching organizational goals through improved evaluation capacity – Canadian Government Executive

NEWS

SEARCH

Performance Measurement
January 30, 2015

Reaching organizational goals through improved evaluation capacity

Over the last four decades, public sector organizations have institutionalized program and policy evaluation practices to varying degrees. As a management tool, evaluation has the potential to provide timely, useful information to decision-makers concerned with resource optimization and program outcomes.

However, piecemeal approaches to evaluation are commonplace in various public administrations, including the federal government. Such approaches tend to produce operational recommendations that hinder the ability of evaluators to support management and contribute meaningfully to organizational learning, accountability and improvement.

Building capacity
Organizations interested in developing a more strategic approach to program evaluation typically focus their efforts on staff training and budgetary allocations. Although these measures are an important starting point, the institutionalization of evaluation requires additional effort.

Researchers have turned their attention to this issue over the last decade and have dubbed the process “organizational evaluation capacity building.” Evaluation capacity can take on different forms, depending on an organization’s context; our own research has focused, therefore, on the development of an empirically-based description of organizational evaluation capacity in the federal government.

Our organizational evaluation capacity framework includes six main dimensions. The first three focus on the organization’s ability to produce high-quality evaluation studies, while the others describe the organization’s ability to use evaluation for organizational learning and accountability purposes.

Human Resources, the first dimension, refers to the technical and interpersonal competencies of evaluators as well as evaluation leadership within the organization. The second dimension, Organizational Resources, focuses on the systems and structures available to evaluators to carry out their duties, such as the financial resources allocated to evaluation studies and the availability of quality performance measurement data. Evaluation Planning and Activities, the third dimension, describes how evaluation is planned and conducted in the organization. This dimension includes, for example, the use of consultants and other external resources to complement internal capabilities and skill sets.

Evaluation Literacy is the fourth dimension and the first to describe organizational capacity to use evaluation. It refers to the widespread knowledge of evaluation and its associated concepts, such as results-based management, across the organization. Organizational Decision Making characterizes the fifth dimension and focuses on the integration of evaluation findings in broader organizational processes and decision-making, such as program resourcing or deficit reduction activities. Finally, Learning Benefits describes how evaluation is used for learning and improvement in the organization, which is the ultimate objective of evaluation itself and evaluation capacity building more specifically.

In addition to the six dimensions and their associated sub-dimensions, the framework also outlines four levels of evaluation capacity: low, developing, intermediate and exemplary capacity. Each of the sub-dimensions, therefore, is described qualitatively at each of those four levels in order to profile and highlight the incremental steps required to move from one level of capacity to the next.

Managers interested in assessing their own organization’s evaluation capacity may use this organizational profile framework to situate their organization across the four levels for all six dimensions. Alternatively, they may also be interested in using an organizational self-assessment instrument, which operationalizes the original framework and facilitates the assessment process.

Although the instrument is meant to be used by federal government departments and agencies, it is possible to adapt the tool to other organizational contexts. We have already adapted the instrument to the not-for-profit sector and the Quebec provincial government, and we are continuing this work to render the tool suitable for use in the Ontario public health sector. Fundamentally, the six dimensions apply to most public sector settings and it is possible to use the instrument in all of these, albeit in a slightly modified form that accounts for local contexts and terminology.

Organizational improvement
Our research shows that organizations that have a good sense of their evaluation capacity are better able to devise strategies to address any major gaps and continue on their path towards institutionalization.

We also know, based on our own sectoral comparative analyses, that organizations with higher evaluation capacity get the greatest “bang for their evaluation buck”; not only are their studies of greater quality and more timely, they also better anticipate senior decisio- makers’ information needs. This, in turn, leads to greater evaluation use and organizational learning.

The link to improved program design and administration, and to a more effective and efficient use of organizational resources, makes this endeavour wholly appealing, not only to evaluators, but to managers across all levels of public sector organizations.

About this author

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Performance Measurement
 
With businesses and government agencies increasingly operating in a highly information-based...
 
Risk is always present in any undertaking, no matter the size...
 
In this episode, J. Richard Jones share information about the Canadian...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
For several months now, the Liberal government ministers have been using...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Here’s some bad news for organizations contemplating projects that have something...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
The latest Auditor General’s report on Shared Services Canada (SSC) and...
 
Public sector innovation, once dismissed as a contradiction in terms, is...
 
The necessity for government executives to manage change, and achieve results,...
 
Every organization does it, but often with ambivalence: the performance measurement...
 
There is an apparent disconnect in performance measurement. Today, agencies measure...
 
The government of Canada’s transformation agenda continues to drive change across...
 
Cost savings in government programs and services used to be achieved...
 
As we move into 2015 – with a looming federal election...
 
Over the last four decades, public sector organizations have institutionalized program...
 
When you purchase clothes, do you like off-the-rack, or do you...
 
All governments, including Canada’s, strive to manage taxpayers’ dollars more efficiently....
 
Getting a clear definition of policy processes and functions can be...
 
While government agencies do not have “bottom lines” in the way...
 
Most of you already know that project management is a means...
 
The taxpayers are wary. Politicians worry about retaining their jobs. Everyone...
 
There has never been a time since the introduction of program...
 
The management of resources has always been taken for granted in...
 
In the last two decades, managing for results has gained momentum...
 
For the first time, the Consortium of Universities for Evaluation Education...
 
The government of New Zealand, and the State Services Commission in...
 
While it has been around for over 130 years, the concept...
 
Performance measurement and public reporting are intended to achieve two complementary...
 
Canada has over 400 administrative tribunals that are probably the least...
 
Last summer, the Vice Chief of the Defence Staff (VCDS) signed...
 
The core mandate of the Consortium of Universities for Evaluation Education...
 
the author fails to provide a complete picture (the good and...
 
Why talk about ethics at all right now? After all, Canada’s...
 
we now have what one federal ADM described to me as...
 
The reduction or mitigation of risks or harms is a major...
 
What are Canadian universities doing to maintain their relevance and innovative...
 
Managing for results has gained significant momentum in the public sector...
 
“Is performance management still relevant?” was the thought-provoking question posed by...
 
Within the last year, the Treasury Board of Canada Secretariat (TBS)...
 
Results matter in the public sector, particularly in these challenging times...
 
Today’s public sector environment is one in which many managers and...
 
Just as with public financial reporting, performance reporting is here to...
 
Being a high-performing organization is like being a high-performing individual: it...
 
With the current reality of deficit reduction, cost cutting and overall...
 
This article is New and Improved! After an exhaustive review cycle...
 
Canada was at the table when the “Commonwealth of Nations” was...
 
If you believe that performance measurement is a vital part of...
 
The last great international effort to refashion the public sector took...
 
Beginning on April 1, 2013, and over a five-year cycle, the...
 
Governments are increasingly using benchmarking to improve organizational performance; to determine...
 
Senior public sector managers, deputy heads of departments and parliamentarians need...
 
Canada is a hockey-loving nation and now is the time of...
 
Too few public sector organizations have truly embedded performance thinking into...
 
CGE Vol.13 No.3 March 2007 Governing is never easy. And in...
 
CGE Vol.13 No.2 February 2007 Improving organizational health and employee engagement...
 
CGE Vol.13 No.1 January 2007 The proposed Federal Accountability Act is...
 
CGE Vol.14 No.2 February 2008 The passage of the Federal Accountability...
 
CGE Vol. 14 No.4 April 2008 The Tellier-Mazankowski advisory committee on...
 
Performance management leads to either disdain or glee in public sector...
 
Les administrations publiques utilisent de plus en plus l’analyse comparative pour...
 
The redesign of corporate tax administration for Ontario is a major...
 
Coincidental with the push for more attention to be paid to...
 
The BC Public Service has had an intense focus on improving...
 
La reconception de l’administration de l’imposition des sociétés en Ontario est...
 
Dans son budget de lutte contre la récession (et d’économies dans...
 
The Harper government’s recession fighting (and government saving) budget heralded an...
 
Many governments have bought into the notion of the Balanced Scorecard,...
 
David P. Norton est le cocréateur, avec Robert Kaplan, du Tableau...
 
Together with Robert Kaplan, David P. Norton is the creator of...
 
Il y a dix ans, la Société canadienne du sang (SCS)...
 
Début septembre, la fonction publique de l’Ontario (FPO) célébrait le 10e...
 
Bon nombre d’administrations publiques ont adopté le tableau de bord prospectif,...
 
Today’s public sector managers encounter increasingly complex corporate issues that call...
 
Performance management seems to be everywhere these days, with much of...
 
The two key tools used by the Canadian government to measure...
 
The principle is unassailable: Canadians have the right to know whether...
 
L’administration publique s’efforce d’obtenir des résultats tangibles en mettant en œuvre...
 
Une étude récente commandée par l’OCDE conclut que, malgré le fait...
 
A recent study commissioned by the OECD concludes that while the...
With businesses and government agencies increasingly operating in a highly information-based...

Member Login

Forgot Password?

Join Us

Password Reset
Please enter your e-mail address. You will receive a new password via e-mail.