Performance Measurement
September 25, 2014

Student research: Impressive and making a difference

For the first time, the Consortium of Universities for Evaluation Education (CUEE) sponsored a student track of research presentations at the Canadian Evaluation Society’s national conference in June. Under the guidance of the conference co-chairs, Dr. Isabelle Bourgeois and Jane Whynot, students were able to submit proposals, which were then turned over to the CUEE to adjudicate. The CUEE selected 12 projects to present over two days in four panels.

Students from across the 12 member universities were encouraged to submit proposals under one of three themes: innovations in research on evaluation; findings from evaluation projects; or key learning points from their evaluation projects. Papers in both English and French were selected for each of these themes, and each panel session was supported by academic discussants from member universities.

With respect to the first theme, innovations in research on evaluation, Frances Gagnon, University of Ottawa, discussed the importance of “construct validity,” noting that it is critical especially when the credibility and utility of evaluation findings are in question. She provided a useful approach to ensuring construct validation throughout various project stages.

Marie-Pier Marchand, Université du Québec, who is working on the roles that judgments play in program evaluations, argued that program stakeholders play a vital role in the process of producing credible judgment, but that this role is poorly captured or documented. She argued from a democratic perspective that stakeholders have important messages to contribute to the evaluation process that increase the usability of reports.

Raimi Osseni, Ècole nationale d’administration publique, presented a development evaluation model for understanding assessment at the country level based on the work of Mario Segone (2009). He maintained that beneficiary countries, not donors, should direct important aspects of their evaluations.

With respect to the second theme, findings from evaluation projects, several papers were presented. First, Barbara Szijarto and Kate Svensson, University of Ottawa, presented results from a knowledge mobilization project, arguing that although participation in program design and implementation encourages knowledge exchange, care is needed to maintain clear roles for the evaluators and other stakeholders involved.

Tanya Halsall, University of Ottawa, spoke about challenges and strategies for success of a leadership program for Aboriginal youth, underlining the efficacy of “sport for development” programs.

Isabelle-Ann Leclair Mallette, Université de Sherbrooke, drew on Huey Chen’s work in action research to describe evaluation findings from a child protection program in Montreal. With over 10 years of data using 37 groups, she reported that program effectiveness is linked to the importance of factors such as client dosage (extent of exposure) to the program activities, stakeholder participation, and design fidelity when the program is implemented.

Sophie-Claire Valiquette-Tessier, Nathalie Freynet, and Marie-Pier Vandette, University of Ottawa, reported results from the evaluation of the needs and services of a Quebec program for single mothers. They reported that the needs and input of single mothers has been under-valued in evaluations.

Carol Miller and Ann Lipman, Carleton University, reported their findings from a formative evaluation of the Kitigan-Zibi First Nation funding agreement with Aboriginal Affairs and Northern Development Canada. They concluded that the agreement did not facilitate local autonomy and severely restricted achieving local objectives.

With respect to the third theme, key learning points from evaluation projects, Jennifer Rae, University of Ottawa, spoke about the Youth Futures Program aimed at improving enrolment and participation in post-secondary education by those students from disadvantaged family backgrounds. She discussed the importance of building capacity in community-based programs, how to generate interest in evaluation with diverse stakeholder groups, and how to make their reports useful.

Hind Al Hudib and Nathalie Gilbert, University of Ottawa, presented their experience as graduate research assistants on a multi-university research team. They reflected on their experiences performing collaborative research by examining work management, tools to facilitate the collaboration process, and lessons learned as graduate students.

As academics who have committed to increasing opportunities for graduate students to learn about evaluation, CUEE members were uniformly supportive of continuing this student-focused strand at future Canadian Evaluation Society conferences.

The CUEE wishes to thank the CES and PPX for their financial support for the students. The PPX was instrumental in covering the costs of student presenters, and the CES generously covered conference fees for several attending students. Most important, we wish to thank the students for generously sharing their work, and generating thoughtful and stimulating conversations.

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Performance Measurement
 
Leadership, service delivery, change management, performance measurement, big data, transparency, digital...
 
Recently, there has been considerable interest in what is being referred...
 
Over the years, governments have been adopting Lean management principles to...
 
As the resettlement of Syrian Refugees exemplifies, government programs, currently running...
 
With businesses and government agencies increasingly operating in a highly information-based...
 
Risk is always present in any undertaking, no matter the size...
 
The Government of Canada’s newly announced Policy on Results replaces the...
 
In this episode, J. Richard Jones share information about the Canadian...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
For several months now, the Liberal government ministers have been using...
 
Global market trends are accelerating to increase the pressure on commercial...
 
As a consultant, I work primarily with private sector organizations including...
 
Here’s some bad news for organizations contemplating projects that have something...
 
If the purpose of the Government of Canada’s performance measurement system...
 
Performance auditing can lead to more efficient, effective, and economical program...
 
The latest Auditor General’s report on Shared Services Canada (SSC) and...
 
Public sector innovation, once dismissed as a contradiction in terms, is...
 
The necessity for government executives to manage change, and achieve results,...
 
Every organization does it, but often with ambivalence: the performance measurement...
 
There is an apparent disconnect in performance measurement. Today, agencies measure...
 
The government of Canada’s transformation agenda continues to drive change across...
 
Cost savings in government programs and services used to be achieved...
 
As we move into 2015 – with a looming federal election...
 
Over the last four decades, public sector organizations have institutionalized program...
 
When you purchase clothes, do you like off-the-rack, or do you...
 
All governments, including Canada’s, strive to manage taxpayers’ dollars more efficiently....
 
Getting a clear definition of policy processes and functions can be...
 
While government agencies do not have “bottom lines” in the way...
 
Most of you already know that project management is a means...
 
The taxpayers are wary. Politicians worry about retaining their jobs. Everyone...
 
There has never been a time since the introduction of program...
 
The management of resources has always been taken for granted in...
 
In the last two decades, managing for results has gained momentum...
 
For the first time, the Consortium of Universities for Evaluation Education...
 
The government of New Zealand, and the State Services Commission in...
 
While it has been around for over 130 years, the concept...
 
Performance measurement and public reporting are intended to achieve two complementary...
 
Canada has over 400 administrative tribunals that are probably the least...
 
Last summer, the Vice Chief of the Defence Staff (VCDS) signed...
 
The core mandate of the Consortium of Universities for Evaluation Education...
 
the author fails to provide a complete picture (the good and...
 
Why talk about ethics at all right now? After all, Canada’s...
 
we now have what one federal ADM described to me as...
 
The reduction or mitigation of risks or harms is a major...
 
What are Canadian universities doing to maintain their relevance and innovative...
 
Managing for results has gained significant momentum in the public sector...
 
“Is performance management still relevant?” was the thought-provoking question posed by...
 
Within the last year, the Treasury Board of Canada Secretariat (TBS)...
 
Results matter in the public sector, particularly in these challenging times...
 
Today’s public sector environment is one in which many managers and...
 
Just as with public financial reporting, performance reporting is here to...
 
Being a high-performing organization is like being a high-performing individual: it...
 
With the current reality of deficit reduction, cost cutting and overall...
 
This article is New and Improved! After an exhaustive review cycle...
 
Canada was at the table when the “Commonwealth of Nations” was...
 
If you believe that performance measurement is a vital part of...
 
The last great international effort to refashion the public sector took...
 
Beginning on April 1, 2013, and over a five-year cycle, the...
 
Governments are increasingly using benchmarking to improve organizational performance; to determine...
 
Senior public sector managers, deputy heads of departments and parliamentarians need...
 
Canada is a hockey-loving nation and now is the time of...
 
Too few public sector organizations have truly embedded performance thinking into...
 
CGE Vol.13 No.1 January 2007 The proposed Federal Accountability Act is...
 
CGE Vol.14 No.2 February 2008 The passage of the Federal Accountability...
 
CGE Vol.13 No.3 March 2007 Governing is never easy. And in...
 
CGE Vol.13 No.2 February 2007 Improving organizational health and employee engagement...
 
CGE Vol. 14 No.4 April 2008 The Tellier-Mazankowski advisory committee on...
 
L’administration publique s’efforce d’obtenir des résultats tangibles en mettant en œuvre...
 
Une étude récente commandée par l’OCDE conclut que, malgré le fait...
 
A recent study commissioned by the OECD concludes that while the...
 
The principle is unassailable: Canadians have the right to know whether...
 
La reconception de l’administration de l’imposition des sociétés en Ontario est...
 
Dans son budget de lutte contre la récession (et d’économies dans...
 
The Harper government’s recession fighting (and government saving) budget heralded an...
 
Many governments have bought into the notion of the Balanced Scorecard,...
 
Bon nombre d’administrations publiques ont adopté le tableau de bord prospectif,...
 
David P. Norton est le cocréateur, avec Robert Kaplan, du Tableau...
 
Together with Robert Kaplan, David P. Norton is the creator of...
 
Performance management leads to either disdain or glee in public sector...
 
Il y a dix ans, la Société canadienne du sang (SCS)...
 
Les administrations publiques utilisent de plus en plus l’analyse comparative pour...
 
Début septembre, la fonction publique de l’Ontario (FPO) célébrait le 10e...
 
The redesign of corporate tax administration for Ontario is a major...
 
Coincidental with the push for more attention to be paid to...
 
The BC Public Service has had an intense focus on improving...
 
Today’s public sector managers encounter increasingly complex corporate issues that call...
 
Performance management seems to be everywhere these days, with much of...
 
The two key tools used by the Canadian government to measure...
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
Leadership, service delivery, change management, performance measurement, big data, transparency, digital...