CGE Vol.13 No.7 September 2007

By: Guy D’Aloisio, Mchel Laurendeau, V. Neimanis, Michael Obrecht, Nancy Porteous, Paul Prieur and Julie Witmer

An Evaluator General for Canada? You might well raise your eyebrows. Many of us did when we first encountered the idea. While the notion of a strong federal voice on evaluation has been discussed among government executives and House committees in various forms over the past thirty years, one does not generally drift off to sleep at night pondering the vision of an evaluator general.

It would be yet another officer of Parliament mandated through an official act – one more person competing for attention with the many commissioners of this and of that. So why are we promoting the idea?

We perceive a critical gap in federal government accountability. Nobody is providing Canadians with an objective assessment of the effectiveness of government programs – no systematic, non-partisan evaluation of program outcomes and results. The auditor general (AG) is on the alert for fraud, waste and poor management practice. That’s great! But efficiency is only part of the picture. We also need to know the results generated by programs and policies. While the Senate and other bodies occasionally provide a broad perspective on program impacts, nobody is regularly delivering high level assessments of program effectiveness.

When Canadians ask: “Is government doing things right?” they look to the AG for answers. There is a regular flow of information on program economy and efficiency from the AG to Parliament and the public, primarily through the media. But when they ask: “Is Government doing the right things?” the line goes dead.

Of course, most departments and agencies have their own program evaluation shops. Couldn’t they provide high-level information on program effectiveness? To be honest…we think not!

Despite the sincerity and dedication of program evaluation units within departments and agencies, there are at least three reasons why they cannot provide the high-level effectiveness information that would be associated with an evaluator general.

First, information from a myriad of internal evaluation offices lacks the credibility of a central integrated office. Suppose that we had no AG, only 80 or so internal audit offices led by different personalities with different communication styles and different credentials. Would the press be covering their reports? Would Parliament be as interested? Just as the AG is Canada’s voice on the management of federal programs, we need an evaluator general as the recognized authority on program results.

Second, evaluations of many departmental and agency programs are not meaningful at the federal policy level. Parliament and journalists are not going to get excited about a report on the effectiveness of, say, a $20 million program. That accounts for less than 0.01% of annual government expenditures. To inform policy-making and resource allocation decisions, Parliament and the public need big picture evaluations that cut across the programming of individual departments.

Third, evaluations from internal evaluation offices are vulnerable to bias. Even when a department uses external consultants to evaluate a program, the results and recommendations may be manipulated. What deputy minister or agency head wants the world to know that one of their programs is less than perfect? Words may be spun or sections may be dropped from evaluation reports to cast the program and the department in a more favourable light. Evaluations emerging from internal processes may have been shaped and moulded at many stages, from the initial identification of issues to the final wording of observations.

We are not saying that internal program evaluation shops should be closed. Quite the contrary, they should be strengthened and expanded. But their mandate should be to generate essential information on program performance that will help government executives improve their programs. When it comes to providing unbiased assessment of government-wide programming the internal evaluation shops are handicapped by their lack of visibility, their limited focus and the politics of program preservation.

So, you might ask, what would the evaluator general and his or her office actually do? Why has Parliament not given the auditor general a mandate to report on program results? If we have managed to pique your interest in the accountability gap, you can find answers to these and other related questions in the longer version online (see the next article in Contents for an English or French version).