Canadian Government Executive - Volume 24 - Issue 04

July/August 2018 // Canadian Government Executive / 29 measurement of the impact of novel pro- gramming. New systems have been set in place for tracking the progress of government com- mitments and ongoing programs. At this stage, the publicly-accessible information that is being generated by these tracking systems is primarily descriptive. Progress tends to be reported using terms such as “underway on track” or “attention re- quired” rather than with hard data that could be useful to Parliament. However, there is some evidence that the tracking systems may be providing Cabinet Minis- ters with useful data on results. In 2016, an article promoting the idea of an Evaluator General for Australia ap- peared in The Mandarin magazine. The vision for an Evaluator General presented in the article differs in many respects from the one being advocated in Canada, but it reflects the same interest in improving the usefulness of evaluation to strategic deci- sion making. The article has influenced thinking within government. For example, in 2018, officials of the Department of In- dustry, Innovation and Science, when commenting on a draft review of the Pub- lic Governance, Performance and Account- ability Act, called for a stronger focus on evaluation, rather than audit, as the func- tion for assessing program impact. They proposed the establishment of an Austra- lian National Evaluation Office headed by an Evaluator General. Parliament needs to access the world’s evaluative information According to one estimate, there are over 1,600 program evaluation reports from de- partments and agencies in an online, open- access database maintained by the Trea- sury Board Secretariat. If these reports on internal evaluations by departments and agencies contained enough information to support strategic decision making, our Parliamentarians could simply ask their staff to search the public database, extract reports and summarize findings. However, issues debated by Parliament tend to be complex. They usually tran- scend individual programs and require in- formation on cost-effectiveness that inter- nal evaluations may not provide. Subjects under debate by Parliament are frequently interdepartmental, interjurisdictional or international in scope. Relevant informa- tion lies not only in internal evaluations of federal programs but also in evaluation reports from around the world. Extraction of useful evidence from the vast amount of available evaluative infor- mation requires third party expertise in assessing the credibility of source evalu- ations, the validity and reliability of their impact measures, and their applicabil- ity to specific Canadian contexts. Further, Parliament may sometimes need quick, original evaluative studies to provide ad- ditional evidence before voting on an is- sue. Members of Parliament may also request longer-term evaluation studies to provide support for future decision mak- ing. The time is right for a new mechanism to provide Parliamentarians with strate- gic level advice on evidence related to the proven or potential effectiveness of major programs or policies that government or private members put forward for consid- eration. With a triumvirate comprised of the Auditor General, the Parliamentary Budget Officer and an Evaluator Gen- eral, Parliament would have arms-length advice on the propriety of government spending, the credibility of government budgets, and the likelihood that programs and policies will achieve desired objec- tives. Evaluative advice for the Cabinet Just as Parliament at large needs the ad- vice of an Evaluator General, so the Prime Minister and Cabinet need a Chief Evalu- ation Advisor, a position parallel to that of Chief Science Advisor. Compared to other Members of Parliament, the Prime Minis- ter and Cabinet already have considerably greater access to providers of evidence. They are supported by the Privy Coun- cil Office and by Deputy Ministers with their networks of policy advisors, program managers and Chief Evaluation Officers. But currently they have no one who has a global perspective on evaluation and the capacity to lead rapid, highly focused studies to complement the existing evi- dence base. A Chief Evaluation Advisor reporting to the Prime Minister and the President of the Treasury Board could fill the gap. A Parliamentary Evidence Officer? Some thinkers envisage an Agent of Par- liament who could assemble evidence PROGRAM EVALUATION for members of the House from many sources, not only from evaluation studies but also from scientific research and tra- ditional knowledge. A “Parliamentary Evi- dence Officer” would have an expert staff of evaluators, science brokers and keepers of traditional knowledge to offer advice to Members of Parliament on the evidence base underlying proposals. But whatever the structure – Evaluator General, Parlia- mentary Evidence Officer or Chief Evalu- ation Advisor – greater access to evidence from systematic evaluation studies con- ducted around the world would improve the evidence base for strategic level deci- sion making. We need arms-length structures for gen- erating advice on evidence from evalua- tion and supplying it to Parliamentarians, the press and the public in a timeframe and in a format that will make it useful to all concerned. M ichael O brecht is an Ottawa-based Credentialed Evaluator with experi- ence in federal public service and management consulting. M ichel L aurendeau is a leading con- sultant in performance measurement and evaluation of federal programs and policies. N icholas G ruen is a policy econo- mist, entrepreneur and commentator, founder of Lateral Economics, Visiting Professor at King’s College London Policy Institute and Adjunct Professor at UTS Business School.