Canadian Government Executive - Volume 22 - Issue 3

25 / 32

March 2016 // Canadian Government Executive / 25
Program Evaluation
The first phase of the risk-based process
focused on defining the universe of enti-
ties that could be evaluated. Defining our
preliminary evaluation universe only re-
quired us to review the departmental pro-
gram inventory. But we wanted to expand
this universe of possibilities and we could
only do it by learning more about our cli-
ents’ needs.
This was made possible during phase
two, where we jointly held consultations
with several levels of management within
the Department, from program managers
to the Deputy Minister, and various coun-
terparts in other departments. These con-
sultations enabled us to better understand
their top priorities, concerns, challenges,
areas of risks, as well as their appetite in
terms of future audits and evaluations re-
ports.
The valuable information obtained
through the consultations was applied
in phase three: the risk assessment, dur-
ing which we created an expanded list of
potential evaluable entities. In order to
measure the level of risk of each entity, we
developed a total of six risk criteria: Ma-
teriality, Previous Evaluations and Audits,
Public Sensitivity, Recent Changes, Com-
plexity, and Senior Management Interest.
Each evaluable entity was defined against
all of these criteria.
The fourth and last phase was the iden-
tification of potential evaluation projects.
In this phase, we took a number of essen-
tial factors in consideration: the program
inventory, the highest priorities and con-
cerns of senior management, the highest
level of risk and the Policy requirements.
The outcome was a list of potential eval-
uation engagements, in the form of a five-
year calendar. We also added a rationale
and a strategic evaluation approach for
each project, to justify the reason behind
each choice. We then followed up with se-
nior management to present and validate
the calendar of evaluation projects. This
resulted in our new evaluation engage-
ments.
The joint participation of the Evaluation
and the Audit teams in this process also
resulted in another product: the very first
combined Risk-Based Audit and Evalua-
tion Plan in the Department.
Success and Lessons
learned
What contributed to the real success of
this collaborative approach were the joint
consultations held with multiple levels of
management. By only contacting clients
once, we were able to significantly reduce
the burden on them while still obtaining
strategic and valuable information. Also,
by taking into account the clients’ true
needs, challenges and risks, we ended up
creating a list of evaluation engagements
that are better aligned with current priori-
ties, and thus, that have a higher chance of
being more useful to decision-makers. As
for the lessons learned, moving forward,
extra time should be allocated right after
the consultations so that the evaluation
team can better define the preliminary
objectives and scope of its next evaluation
engagements.
Marie-Josée Courchesne is the
Manager of Evaluation Services at
Infrastructure Canada. She also
completed a Master’s Degree in Public
Administration specialized in Program
Evaluation at the École Nationale
d’Administration Publique (ENAP) in
Gatineau.
Linda Vertefeuille is currently
working as an Evaluation Analyst
at Infrastructure Canada. She has
recently obtained her Master’s Degree
in Public Administration specialized
in Program Evaluation at the École
Nationale d’Administration Publique
(ENAP) in Gatineau. vertefeuillelind@
. on a more traditional
planning approach
based on the
development of a
program inventory,
our evaluation team
embarked upon a
special journey with
the audit team, by
participating in a joint
four-phased risk-based
process.