Principled Evaluation: Who Cares? - Canadian Government Executive

NEWS

SEARCH

ManagementPerformanceProgram Evaluation
January 18, 2018

Principled Evaluation: Who Cares?

The time is right for the Canadian evaluation community to have a robust and vital discourse on the ‘ties that bind’ us together as professionals, and that set the stage for principled practice. In the absence of such, a key cornerstone of evaluation excellence may begin to erode.

Compared to audit professionals and their preoccupation with “independence”, evaluation specialists may seem unconcerned about ethical principles. Their fixation is on systematic processes, methods and techniques. Principles underlying evaluations rarely come up in evaluation planning or implementation, and they seldom figure in evaluation reports or follow-ups.

Give a workshop to evaluators on “big data” or “data visualization” and you can fill an auditorium. Have a session on evaluation principles and ethics and the room is mainly empty. So, what’s going on? Perhaps the absence of discussion leads evaluators to conclude that ethical challenges do not exist. They then infer that they and their field must be, by their very nature, principled and ethical. Suspecting that it’s not that simple, we feel it’s time to ask: What dialogue on evaluation principles and ethics ever occurs? What ethical challenges are evaluators encountering? What are they doing about them?

As are any professionals, evaluators are first and foremost defined by the central values they hold. If their central tenets go unnoticed, then their practice fades into a parade alongside every other activity in the scientific management field. Indeed, no profession is immune from this risk. For example, a study of almost 1000 physicians in several countries found that although they were aware that their profession had a code of ethics, almost to a person they admitted to not referring to it and not using it. The time is right for the Canadian evaluation community to have a robust and vital discourse on the ‘ties that bind’ us together as professionals, and that set the stage for principled practice. In the absence of such, a key cornerstone of evaluation excellence may begin to erode.

To understand how evaluators implement ethics in their day-to-day practices, surveys were conducted of Canadian Evaluation Society (CES) members in 2010 (response rate 26 per cent, n = 455) and 2015 (response rate 12 per cent, n = 218). The 2015 study included qualitative case studies of eight evaluations conducted between 2013 and 2016.  See below for a summary of the results from these studies.

Ethics is a serious issue for Canadian evaluators. They associate ethics with ideas about integrity, quality and impartiality, values that are important to them as professionals. Start a dialogue about principles and ethics with evaluators and it quickly descends down a long chute of abstract concepts – accountability, competence, confidentiality, impartiality, independence, integrity, neutrality, objectivity, subjectivity, and the list goes on.

Comparing the 2010 and 2015 surveys suggests that evaluators are becoming more aware of and can identify more challenges in their work. As the 2010 and 2015 studies attest, the power of those who commission evaluations and their influence over evaluators’ treatment of ethics during the evaluation process is substantial. Evaluators are frequently called upon to deal with direct pressures from stakeholders to modify the evaluation process or findings. This can lead to potential bias in the evaluation, especially in terms of impartiality. It also increases evaluators’ anxiety. Evaluation clients, those who commission evaluations, and evaluators themselves hold important roles and responsibilities with regard to ensuring principled, ethical practice.

At the same time, evaluators seem shy to speak out on this topic. This has much to do with sensitivities related to conflicts of interest that are perceived to affect employment circumstances and opportunities. While evaluators do welcome occasions to provide feedback, there is a profound sensitivity around full ethical disclosure.

The 2010 survey revealed that three-quarters or 77 per cent of Canadian evaluators encountered at least one perceived ethical challenge in their work, but only 57 per cent were willing to provide a description or example. For them, the setting for such a discussion needs to be right, and the space safe. Both the 2010 and 2015 studies reveal that colleagues (professional evaluators or otherwise) are an important source of reflection and advice, i.e., a trusted viewpoint in a safe space.

As participants in the 2015 study observed, discussion about ethical challenges in evaluators’ daily practice is not systematic.  It raises the question of the extent to which evaluators can and feel comfortable raising these ethical issues with the client before, during, and or after the evaluation process.

The 2010 survey also revealed that there is room for a more active role from the Canadian Evaluation Society at 62 per cent. A vacuum currently exists and is something only the national association can fill. This may include professional development – workshops, formal training; regular surveillance among members; greater emphasis on ethics in the Credentialed Evaluator designation; more space in professional publications (e.g. Canadian Journal of Program Evaluation), and more dialogue at chapter level events and national conferences.

The Canadian Evaluation Society is indeed now taking on this role, starting with a review of its now 20-year old Ethics Principles of Competence, Integrity and Accountability. Since that document was produced, there have been major changes in the environment affecting Canadian evaluators’ work, including the introduction of national and provincial privacy legislation; the publication of the Government of Canada’s Tri-Council Policy Statement on research with Human Participants and its exclusion of program evaluation from ethical reviews; the publication of the CES Competencies for Canadian Evaluators, and its soon-to-be-released with a renewed focus on ethics.

Other national associations, notably the American Evaluation Association and the Australasian Evaluation Society, are also undertaking ethics renewal initiatives. As Harry Cummings, CES President noted, “Evaluators’ work, roles and fundamental obligations to their clients, program beneficiaries and society are only becoming more complex, and ethical challenges more subtle and intersectional. CES needs to step up and assume leadership in supporting and protecting Canadian evaluators and evaluation stakeholders in ethical practice.”

The Canadian Evaluation Society, National Capital Chapter (NCC) will be holding its Annual Learning Event in Ottawa on February 13, 2018. The theme of this event is Principled Evaluation: Should I give a s*#t? This title is admittedly provocative but one that will delve into the questions and concerns raised in this article.

To learn more or to register for this event, go here.

 

Wayne MacDonald MA, CEP – Director, Canadian Evaluation Society – National Capital Chapter and President, Infinity Consulting and Legal Services (ICLS). Wayne.macdonald@rogers.com

Emilie Peter, PhD – is a recipient of the Joseph-Armand-Bombardier Canada Graduate Scholarship (SSHRC, 2013) and recently completed her PhD in Public Administration at the École nationale d’administration publique (ÉNAP). Emilie.peter@enap.ca

Natalie Kishchuk PhD, CE, FCES – Vice-President, Canadian Evaluation Society and independent consultant located in Montréal, Québec. vicepresident@evaluationcanada.ca

 

 

About this author

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Management
 
‘Elephant in the room’ is a metaphorical idiom for an obvious...
 
Compared to audit professionals and their preoccupation with “independence”, evaluation specialists...
 
Having a clear definition of success is one of the key...
 
Time is of the essence for Alice’s White Rabbit. In business,...
 
Professional development is tremendously important to members of Canada’s public sector....
 
The International Civil Service Effectiveness (InCiSE) Index project, a collaboration between...
 
Yogi Berra, Maurice (The Rocket) Richard, Bill Russell, Michael Jordan, and...
 
Three years before his death in 2011, Jack Layton released a...
 
In today’s workplace, individuals increasingly face dynamic and difficult challenges that...
 
We are happy to share with you the September 2017 issue...
 
The health care system in Canada, known as Medicare, is publicly...
 
Please to view this Content. (Not a member? Join Today! )...
 
Water is essential to sustain all forms of life and more...
 
We are thrilled to offer the July/August issue of Canadian Government...
 
We are happy to share with you the May/June issue of...
 
The International Monetary Fund (IMF) in issuing its annual review of...
 
In about two weeks, APEX will be holding its 2017 Symposium...
 
Please to view this Content. (Not a member? Join Today! )...
 
Cost estimation is becoming an extremely important skill within government due...
 
Please to view this Content. (Not a member? Join Today! )...
 
Risk is always present in any undertaking, no matter the size...
 
Last year, procurement Minister Jody Foote was prompted by the swirling...
 
Bill Gates once said that “Technology is just a tool. In...
 
Laskey Hart guest hosts CGE Radio this week with a summary...
 
With baby boomers continuing to hang onto their office desks and...
 
In every industry, the competition for the talent pool is on...
 
The Conference Board of Canada recently published a report titled Leveraging...
 
Please to view this Content. (Not a member? Join Today! )...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
Ontario’s provincial government needs to act fast in building a coordinated...
 
The bootleg fentanyl overdose crisis that is sweeping across Western provinces...
 
The association representing more than 42,000 physicians and medical students in...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
There’s a fine line between confidence and naivety – a line...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Digitizing certain aspects of the hiring process appeals to both employers...
 
As populations grow and age, the demand for services increases. As...
 
Please to view this Content. (Not a member? Join Today! )...
 
On June 7, over 40 senior executives from within the public...
 
Please to view this Content. (Not a member? Join Today! )...
 
By Nicole Verkindt Despite growing up in a small town in...
 
The actions and decisions of public servants have consequences for the...
 
By Jason Zhang  The best way to cost effectively manage the retention...
 
Please to view this Content. (Not a member? Join Today! )...
 
He is often referred to as the federal government’s first “chief...
 
By Michael Murphy Not all assets can and should be equally...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Here’s some bad news for organizations contemplating projects that have something...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Nearly half of Canadian organizations are falling behind on implementation of...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
The idea of converged infrastructure and converged systems in the datacentre...
 
Please to view this Content. (Not a member? Join Today! )...
 
That the new Liberal Government has embraced ‘open government’ is hardly...
 
Meet Bob Heart.  He is an outstanding employee who works hard...
 
A new study released yesterday, Securing the C-Suite, Cybersecurity Perspectives from...
 
The bloodletting at once-mighty Canadian mobile tech giant, BlackBerry continues as...
 
There is no shortage of examples of businesses that effectively used...
 
Please to view this Content. (Not a member? Join Today! )...
 
Rogers Media announced yesterday cost-cutting moves that will see some 200...
 
Microsoft, on Tuesday, ended support for older versions of Internet Explorer...
 
Written by Tim Wacker Almost a quarter century ago, when most...
 
Last week the Wall Street Journal, citing anonymous sources, reported that...
 
Written By Jason McNaught The Public Service Alliance of Canada was...
 
A leader’s credibility begins with personal success. It ends with helping...
 
Written by Donald Farmer Too often, we base Business Intelligence today...
 
Written by  Gail Vallance Barrington Anyone who has commissioned a program...
 
Please to view this Content. (Not a member? Join Today! )...
 
Skills International, a global talent and career management firm, has launched...
 
When it comes to training, statistics show small businesses are more...
 
Today, TD Canada Trust released the results of a survey studying...
 
Management has its upsides, but no one said it would be...
 
Independence has long been regarded as a cornerstone of the auditing...
 
Congratulations. You’ve achieved professional recognition for your performance and the public...
 
In 2011, PwC began a conversation with global defence leaders on...
 
Canada is a diverse nation, in language, culture, geography, and, ultimately,...
 
Public sector organizations are under increasing pressure to identify all risks...
 
The government of Canada has implemented several measures over the past...
 
The economy is sluggish…oil prices are down…stock markets are up but...
 
Whether at the territorial, provincial or federal government level, internal audit...
 
On a cold, grey Sunday afternoon last November, about 2,500 residents...
 
An organization’s reputation can take years to build but it can...
 
Change definitely happens, and it is happening faster and faster as...
 
On October 30, Ontario began second reading of the Public Sector...
 
commentEmail””:””ipfrazer@ipfconsulting.com””...
 
The success of organizations, and the Canadian economy, is directly tied...
 
In the well-known children’s story, an Emperor falls victim to the...
 
The Nova Scotia Office of the Ombudsman is a small operation...
 
I think the ombudsman needs to be independent, because without independence...
 
Today’s business environment changes rapidly to adjust to evolving conditions and...
 
There’s a cloud looming over public sector employers in Canada. Baby...
 
“”corporate governance””...
 
The best internal auditors actually are really good managers first. I...
 
In the middle of an empty field on the edge of...
 
The recent controversy about the actions of some staff members in...
 
Most professionals don’t need more than a sentence at a cocktail...
 
Recent research by the Institute of Internal Auditors Canada aims to...
 
Today’s tasty hamburger bun is a marvel of industrial cuisine and...
 
In this last week before Christmas and the holidays, this is...
 
As part of the Public Policy Forum’s university lecture series, Mel...
 
Respecting, processing and cross-pollinating thoughts from diverse cultures will advance team...
 
When is it that a politician becomes part of the governing...
 
It is something that few senior managers spend much time thinking...
 
It’s been a busy couple of weeks on the information, privacy...
 
In 1996, a new budget watchdog, the Parliamentary Budget Officer, was...
 
The publication of Donald Savoie’s latest book, Whatever Happened to the...
 
The Ontario government is moving forward with the creation of a...
 
He blew up the factory! Jack Welch, as CEO of General...
 
There is no doubt that the challenges faced by governments are...
 
When printing press technology was diffused through Europe, adult literature preceded...
 
The Office of the Public Sector Integrity Commissioner of Canada (PSIC)...
 
After the Auditor General’s (AG) report was released on April 30,...
 
Entering 2013, Canadian employers and unions, public and private sector alike,...
 
In the U.K. system, Permanent Secretaries are what we call Deputy...
 
As both a management consultant and a Boston Marathoner who was...
 
I believe that the greatest challenge and opportunity for management is...
 
The Asian Development Bank estimates that the global demand for infrastructure...
 
n Saskatchewan, a drug shortage team in the Ministry of Health...
 
Three decades ago, the federal public service was proportionately larger, generally...
 
We are living in a period of rapid change and limited...
 
US public sector employees don’t trust their management to do the...
 
The news of Mark Carney’s nomination as the new Governor of...
 
Following Singapore’s independence in 1965, the controversial leadership of Lee Kuan...
 
Over the past few years, the preparation and delivery of the...
 
In healthcare, cost-cutting can result in cutting what is valued most...
 
For over 20 years Colin Bennett has been exploring issues of...
 
Even before controversy shook the organization to its foundation, Ornge was...
 
It will be the largest international multi-sport event ever held on...
 
We’ve all seen the headlines – BC Ferries, Ornge, la Caisse...
 
It can happen, and it’s noteworthy when it does. Government, business...
 
There seems little argument in the wake of the financial crisis...
 
Kevin Page’s mandate as the first Parliamentary Budget Officer comes to...
 
Governments are challenged to meaningfully mitigate the effects of the financial...
 
Canada is facing a huge financial challenge brought on by massive...
 
For the past one hundred years, democratic states have been moving...
 
It’s so much easier and less painful to learn from the...
 
Mandated by Congress in late 2010, the U.S. federal government is...
 
CGE Vol.13 No.4 April 2007 "The potential to lead is in...
 
CGE Vol.13 No.2 February 2007 The Connecting the Dots series is...
 
CGE Vol.13 No.2 February 2007 For too many managers, "crisis" is...
 
CGE Vol.14 No.3 March 2008 Organizations communicate "what the real game...
 
CGE Vol.13 No.2 February 2007 Canada’s Performance 2006 is the sixth...
 
CGE Vol.14 No.1 January 2008 The furor over the $300,000 that...
 
CGE Vol.14 No.2 February 2008 Community Safety and Correctional Services is...
 
CGE Vol.14 No.3 March 2008 If you spend much time at...
 
CGE Vol.13 No.7 September 2007 "If the Public Service, as a...
 
When pondering leadership, we immediately think of exercising our influence downward...
 
CGE Vol.13 No.1 January 2007 "How can I be held accountable...
 
CGE Vol.13 No.1 January 2007 Leadership How would you describe your...
 
CGE Vol.14 No.2 February 2008 Let’s say you’re a senior manager,...
 
CGE Vol.13 No.5 May 2007 How often has a new employee...
 
CGE Vol.13 No.1 January 2007 Perhaps it’s a legacy of the...
 
The public service strives to achieve positive outcomes with appropriately targeted...
 
Last fall, Alberta’s Employment and Immigration department posted online the workplace...
 
Managing public sector organizations requires living with high levels of complexity...
 
The significant challenges created by the global recession have had a...
 
Saskatchewan, like other provinces, spends almost half its budget on healthcare....
 
Building project management and business transformation capacity is critical to advancing...
 
You’ve heard it many times: public service leaders are accountable for...
 
Citizens want to talk to politicians and policymakers to explain what...
 
Dans le monde d’aujourd’hui, les entreprises, les organismes et leurs clients...
 
if possible I would love to ask you a few other...
 
En parcourant tous les diagrammes techniques qui couvrent des cartes murales...
 
On pense souvent que la fonction publique devra recruter un grand...
 
Jumeler les compétences opérationnelles avec le souci du service, les aptitudes...
 
La complexité des programmes publics ne date pas d’hier. Nul ne...
 
Après avoir consacré trois années à la mobilisation des employés et...
 
CGE Vol.13 No.9 November 2007 The new expenditure management system announced...
 
As the Public Service Commission of Canada celebrates its 100th anniversary...
 
With the hotly contested Canadian and American elections behind us, the...
 
CGE Vol.13 No.8 October 2007 A few weeks ago, the federal...
 
Like most people, public service leaders intuitively understand the value of...
 
As the global economy struggles to regain some forward momentum, Canadian...
 
This will be a defining budget for Stephen Harper. It will...
 
It is difficult to determine when the debate about the need...
 
For the next few years, the federal government’s overarching agenda will...
 
Much of the current conversation about the federal government’s economic agenda...
 
Are you a good boss? Or a bad boss? Or a...
 
Most government executives, most of the time, are focused on dealing...
 
Lee McCormack, ancien cadre supérieur du gouvernement, est directeur de la...
 
Henry Mintzberg is Canada’s most celebrated management thinker. He and colleagues...
 
Stephen M.R. Covey has sold 600,000 copies of The Speed of...
 
Comment les cadres et les élus des villes dirigentR09;ils les services...
 
A decade ago Canadian Blood Services took over a troubled agency...
 
Ne vous fiez à personne de plus de 30 ans, déclarait...
 
CGE Vol.13 No.10 December 2007 Demographics and technology, two of our...
 
In 2006 the world was feeling the aftershocks of a number...
 
The Independent Blue Ribbon Panel on Grants and Contributions called for...
 
Au Canada, le secteur bénévole et à but non lucratif vit...
 
CGE Vol.13 No.4 April 2007 Robert Parkins, editorial director, met recently...
 
Bill Greenlaw is the elected president of the Institute for Public...
 
In the past two decades, the nature of the state has...
 
Have you ever asked yourself the question: ‘How would I evaluate...
 
Please to view this Content. (Not a member? Join Today! )...
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
‘Elephant in the room’ is a metaphorical idiom for an obvious...

Member Login

Forgot Password?

Join Us

Password Reset
Please enter your e-mail address. You will receive a new password via e-mail.