Performance Measurement
February 8, 2013

Reliable performance: Standards for the numbers

Just as with public financial reporting, performance reporting is here to stay. Just as with financial reporting, it will never be entirely free of controversy. If, as seems to be the case, performance reporting is going to continue to be an important element of our political and managerial systems in the public domain, we need to start thinking about how to make it as consensual and useful as we can.

Government by numbers has always to some extent been with us. After the Norman conquest of Britain in 1066, the new overloads created the Doomsday book just so they could count what it was exactly they now owned.

But the use of numbers goes much further. Governments – even autocratic ones – have always sought to trumpet their achievements through measurement. They boast about how many roads they’ve built, how big the dams are they’ve constructed, how many children they have educated and above all how well the economy has grown under their guidance. Especially since the middle of the 19th century, in the West at least, there has been an explosion of measuring social “facts” like population, disease, education, crime, trade, inflation, economic growth and other socio-economic indicators.

More recently, roughly since the 1990s, there has been an explosion in measuring the activities and results of public institutions: organizations, programs and even governments themselves (e.g., the World Bank’s “governance” indicators).

This “performance movement,” as some have called it, is wide and deep. Wide in that it has spread across many different countries with often vastly differing administrative traditions, and deep in that in some countries, like the U.K., it has penetrated almost every aspect of public activity and gone on for two decades or more.

I have argued elsewhere that this movement is very unlikely to fade away; the performance reporting “genie” is out of the bottle and won’t be put back in, even if it changes. The Coalition government that was formed in Britain in 2010 appeared to be committed to trying to do just that, but analysis of their Departmental Business Plans shows that they have if anything more targets, milestones and measurements than their predecessors.

Needed improvement
So, whatever the criticisms of the “performance movement” (and there are many, some justified, others less so) it is not going to go away. The real issue is how do we improve it, make it work better and to what end?

At the heart of whatever use performance reporting is put to – policymaking, management, citizen-choice, accountability – must lie the issue of validity and reliability of measurements. What standards of accuracy and appropriateness should we have and who should determine and monitor them?

The parallel I would draw here is with public finance, especially spending. Over the past century or more we have steadily evolved methods of accounting for public money that are fairly consensual in most advanced democracies. Moreover accounting standards have become increasingly international. In most emerging economies the issue is not what standards are needed to measure and report on public spending, but how to achieve the right levels of accuracy and appropriateness. Of course, there are still many debates about some specifics of how to measure, budget and report on public finances but there is a large degree of consensus about basic counting practices. There are bigger debates about what to do with the information once we have it, but these are precisely to be expected.

Uniform standards
The idea I floated recently in Britain in a White Paper was that we could evolve similar reporting standards for public sector performance and efficiency reporting.

In my recent book on Theories of Performance, I pointed out that although there are some commonly used concepts within the performance movement, such “performance” itself, “efficiency,” “outputs,” “outcomes,” “productivity,” “value for money,” etc, the definitions of these terms varied both within and across jurisdictions.

A useful starting point therefore would be to get some consensus on what we mean by these general terms and enforce their adoption in public agencies. An attempt was made to do the first part of this back in 2001 in the U.K. when HM Treasury, the Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics jointly published a booklet called “Choosing the Right FABRIC – a framework for performance information,” which included a set of agreed definitions.

Unfortunately, with no one to police this new framework, the use of these standards concepts soon broke down, with HM Treasury itself leading the way to misuse of the terminology upon which it had just agreed. But the fact it could happen at all showed it was possible.

The UK National Audit Office, to be fair, has tried in various ways to introduce more standard use of terms through its “value for money” reports over the past 20 years, but it seems to have had little impact on central government departments and agencies, who continue to adopt a cavalier attitude to defining performance reporting, especially when reporting so-called “efficiency” savings.

So there is a strong case for developing what I’ve called Generally Accepted Reporting Standards for Public Performance, or GARSPP. In my view we have sufficient experience now of reporting the performance of public agencies and programs to develop such agreed standards, even if at first they are in the form of broad principles and definitions.

Establishing such standards would be a process rather than an event. It would take time to get even an initial framework established, and it would undoubtedly evolve over time and never reach an end point. It would probably be subject to some controversy; indeed it would be a bad sign if it wasn’t because lack of debate would mean it wasn’t very important. But a process of gradually creating greater areas of agreement and consensus, and narrowing the areas of controversy, seems perfectly feasible.

Who should be responsible for developing such standards? Clearly it would need to be a body that could command respect for its technical competence and independence from direct political interference, and be able to draw together political, expert and managerial stakeholders to build a consensus. In some jurisdictions this might be a role for a supreme audit body or a Congressional or Parliamentary Budget Office perhaps? In others, especially federal systems, it might be necessary to create a special body specifically with the remit to create and police such standards.

Whoever it is, it needs to be someone other than the government of the day and its civil service or, as in the U.K. case, standards would be rapidly politicized.

Dr. Colin Talbot is professor of government at the University of Manchester and director of Policy@Manchester. He has researched and advised on issues of performance reporting to many public organizations and governments, including Treasury Board of Canada.

About this author

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Performance Measurement
 
Leadership, service delivery, change management, performance measurement, big data, transparency, digital...
 
Recently, there has been considerable interest in what is being referred...
 
Over the years, governments have been adopting Lean management principles to...
 
As the resettlement of Syrian Refugees exemplifies, government programs, currently running...
 
With businesses and government agencies increasingly operating in a highly information-based...
 
Risk is always present in any undertaking, no matter the size...
 
The Government of Canada’s newly announced Policy on Results replaces the...
 
In this episode, J. Richard Jones share information about the Canadian...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
For several months now, the Liberal government ministers have been using...
 
Global market trends are accelerating to increase the pressure on commercial...
 
As a consultant, I work primarily with private sector organizations including...
 
Here’s some bad news for organizations contemplating projects that have something...
 
If the purpose of the Government of Canada’s performance measurement system...
 
Performance auditing can lead to more efficient, effective, and economical program...
 
The latest Auditor General’s report on Shared Services Canada (SSC) and...
 
Public sector innovation, once dismissed as a contradiction in terms, is...
 
The necessity for government executives to manage change, and achieve results,...
 
Every organization does it, but often with ambivalence: the performance measurement...
 
There is an apparent disconnect in performance measurement. Today, agencies measure...
 
The government of Canada’s transformation agenda continues to drive change across...
 
Cost savings in government programs and services used to be achieved...
 
As we move into 2015 – with a looming federal election...
 
Over the last four decades, public sector organizations have institutionalized program...
 
When you purchase clothes, do you like off-the-rack, or do you...
 
All governments, including Canada’s, strive to manage taxpayers’ dollars more efficiently....
 
Getting a clear definition of policy processes and functions can be...
 
While government agencies do not have “bottom lines” in the way...
 
Most of you already know that project management is a means...
 
The taxpayers are wary. Politicians worry about retaining their jobs. Everyone...
 
There has never been a time since the introduction of program...
 
The management of resources has always been taken for granted in...
 
In the last two decades, managing for results has gained momentum...
 
For the first time, the Consortium of Universities for Evaluation Education...
 
The government of New Zealand, and the State Services Commission in...
 
While it has been around for over 130 years, the concept...
 
Performance measurement and public reporting are intended to achieve two complementary...
 
Canada has over 400 administrative tribunals that are probably the least...
 
Last summer, the Vice Chief of the Defence Staff (VCDS) signed...
 
The core mandate of the Consortium of Universities for Evaluation Education...
 
the author fails to provide a complete picture (the good and...
 
Why talk about ethics at all right now? After all, Canada’s...
 
we now have what one federal ADM described to me as...
 
The reduction or mitigation of risks or harms is a major...
 
What are Canadian universities doing to maintain their relevance and innovative...
 
Managing for results has gained significant momentum in the public sector...
 
“Is performance management still relevant?” was the thought-provoking question posed by...
 
Within the last year, the Treasury Board of Canada Secretariat (TBS)...
 
Results matter in the public sector, particularly in these challenging times...
 
Today’s public sector environment is one in which many managers and...
 
Just as with public financial reporting, performance reporting is here to...
 
Being a high-performing organization is like being a high-performing individual: it...
 
With the current reality of deficit reduction, cost cutting and overall...
 
This article is New and Improved! After an exhaustive review cycle...
 
Canada was at the table when the “Commonwealth of Nations” was...
 
If you believe that performance measurement is a vital part of...
 
The last great international effort to refashion the public sector took...
 
Beginning on April 1, 2013, and over a five-year cycle, the...
 
Governments are increasingly using benchmarking to improve organizational performance; to determine...
 
Senior public sector managers, deputy heads of departments and parliamentarians need...
 
Canada is a hockey-loving nation and now is the time of...
 
Too few public sector organizations have truly embedded performance thinking into...
 
CGE Vol.13 No.3 March 2007 Governing is never easy. And in...
 
CGE Vol.13 No.2 February 2007 Improving organizational health and employee engagement...
 
CGE Vol.13 No.1 January 2007 The proposed Federal Accountability Act is...
 
CGE Vol.14 No.2 February 2008 The passage of the Federal Accountability...
 
CGE Vol. 14 No.4 April 2008 The Tellier-Mazankowski advisory committee on...
 
Performance management leads to either disdain or glee in public sector...
 
Les administrations publiques utilisent de plus en plus l’analyse comparative pour...
 
The redesign of corporate tax administration for Ontario is a major...
 
Coincidental with the push for more attention to be paid to...
 
The BC Public Service has had an intense focus on improving...
 
La reconception de l’administration de l’imposition des sociétés en Ontario est...
 
Dans son budget de lutte contre la récession (et d’économies dans...
 
The Harper government’s recession fighting (and government saving) budget heralded an...
 
Many governments have bought into the notion of the Balanced Scorecard,...
 
David P. Norton est le cocréateur, avec Robert Kaplan, du Tableau...
 
Together with Robert Kaplan, David P. Norton is the creator of...
 
Il y a dix ans, la Société canadienne du sang (SCS)...
 
Début septembre, la fonction publique de l’Ontario (FPO) célébrait le 10e...
 
Bon nombre d’administrations publiques ont adopté le tableau de bord prospectif,...
 
Today’s public sector managers encounter increasingly complex corporate issues that call...
 
Performance management seems to be everywhere these days, with much of...
 
The two key tools used by the Canadian government to measure...
 
The principle is unassailable: Canadians have the right to know whether...
 
L’administration publique s’efforce d’obtenir des résultats tangibles en mettant en œuvre...
 
Une étude récente commandée par l’OCDE conclut que, malgré le fait...
 
A recent study commissioned by the OECD concludes that while the...
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
Leadership, service delivery, change management, performance measurement, big data, transparency, digital...