Canadian Government Executive - Volume 22 - Issue 6

27 / 32

June 2016 // Canadian Government Executive / 27
Program Evaluation
ator sports many hats in these relation-
ships, s/he wears only one at a time. For
example, in meetings with organizational
leadership, the evaluator is the expert,
hired by the organization to complete
a project, and must convey progress on
bringing that project to its successful com-
pletion. But when the evaluator interacts
with the organization’s evaluation cham-
pion, the former becomes an enabler, em-
powering the champion to make sense of
the evaluation process and resultant data,
and to sustain evaluative efforts after proj-
ect completion.
I have been often asked why is there
the need for an evaluation management
framework to build these types of relation-
ships. My response is that the framework
allows all involved parties to see where
they are on the evaluation roadmap. Once
roles and responsibilities are identified,
each party within the company better
understands the evaluation process and
where they fit. The feeling of comfort and
reassurance that this provides cannot be
underestimated. The framework’s orienta-
tion function may seem elementary, but in
organizations with no evaluation culture,
it is immensely beneficial in building such
capacity of the individuals involved.
Lessons learned
The multi-step framework and its use as a
road map, especially in stakeholder analy-
07090_CCPV_ad_FINAL_b.indd 1 2016-05-03 10:20 AM
sis and identification of PIUs to clearly
delineate roles and responsibilities, also
helps in building of relationships within
the organization. The collective effort
made to understand the historical context
of the organization and program is invalu-
able. It is important for external evalua-
tors to know what efforts have been made
before to bring about program improve-
ment, howwell they worked, and who was
involved in the process.
Most people are naturally reluctant to
venture down a road they have already
unsuccessfully travelled. They are espe-
cially suspicious about being led by some-
one external to their company. The nature
and strength of relationships forged via
the framework also provides confidence
to the evaluator in adherence to ethical
and professional obligations without fear
of reprisal during times of organizational
pushback. The fostering of relationships
within a context of professional conduct
helps to ensure relevance of evaluation
processes and utilization of results with-
in the organization on an ongoing basis.
Here, the external evaluator is viewed not
only as the “expert” but also the “enabler.”
Though I have had much success using
this PIU-focused evaluation framework,
not all evaluations have run smoothly.
Recently, my team and I completed a com-
prehensive evaluation of the national cer-
tification program for a group of financial
professionals. The framework identified
the course for the evaluation, clarified
people’s roles, identified PIUs and mar-
shalled the efforts of an internal evalu-
ation champion. When it came time to
make our recommendations, which would
have improved and altered the course of
the program, we received a great deal of
organizational pushback. But the organi-
zation’s evaluation champion echoed our
opinions. Further, the capacity we had
been building with the executive steering
committee allowed them to better under-
stand the evaluative processes and result-
ing recommendations.
Because we fostered strong relation-
ships within the organization, our sug-
gestions were not viewed as threatening.
In the end, our recommendations were
implemented, some program operations
were altered and an improved program
emerged. I do not believe that any of this
could have realized if not for the applica-
tion of the multi-step evaluation frame-
work and its focus on PIUs. It provides a
clear path, where few alternatives exist,
for evaluation management and capacity
building within a private sector organiza-
tion that lack an evaluation culture.
A. Sidiq Ali , PhD CE, is Managing
Director of Research & Evaluation
Consulting, Inc. His blog is www.rae-