This article is New and Improved! After an exhaustive review cycle with an extensive team of researchers and editors, we’ve enhanced it from the previous version!
Were we successful in grabbing your attention with the first paragraph? Did the “New and Improved” draw you in? Perhaps a more pertinent question would have been: did the introduction mean anything to you? After all, it’s packed full of vague promises that what follows is the answer to all your performance measurement problems. Guaranteed!
Perhaps it’s simply a marketing ploy that we’ve copied after seeing it time after time on television commercials or print advertisements for products as diverse as toothpaste, laundry detergent, shampoo and soft drinks.
Using vague words such as improved, better, enhanced, streamlined, and strengthened is just one of five weaker techniques used. Other “worst practices” that we have seen include: reporting measures without context; counts of activities without outcomes; perverse indicators and information overload.
Reporting measures without context: Politicians love to cite large numbers, particularly in the millions of dollars. This is a wonderful headline grabbing technique, but it often presents an incomplete picture to the reader if the number is not given in context. For example, while the mayor of a city may claim that $100 million is to be spent on new roads over the next two years, if given in context – namely that $1 billion is required to deal with the infrastructure deficit – the number is less meaningful, as only 10 percent of the requirement is being dealt with. This context is a vital part of reporting on the measure.
Counts of activities without outcomes: How many measures have you seen that include, for example, the creation of 12 policies over the fiscal year? This illustrates a performance measure that lacks context and is simply the count of an output as opposed to the measurement of an impact or an outcome. Another example: 30,000 projects were completed as a result of the government’s Economic Action Plan; while this is an impressive figure, additional context in terms of impact on Canadians would provide increased insight and relevance.
Governments across the world have been proclaiming the success of stimulus spending by releasing statistics of the number of jobs “created or saved.” The count of something that may or may not have happened is not a recommended technique and, arguably, a number that lacks meaning, particularly if you only give a range of figures.
Another “worst practice” to avoid is setting perverse indicators that may have the opposite or unintended effect of what was planned. Consider an organization that has chosen to concentrate on and report the time (in days) taken to process an application and return a result to the applicant. While the turn-around time may have decreased, this may have been to the detriment of quality, which may similarly decrease as staff may have to rush or shortcut steps to meet the new service standard to process applications. Monitoring multiple measures is a way to guard against this perverse effect.
Information overload may cause “analysis paralysis.” We have observed occasions where executive reporting decks are cluttered and contain too much information that does not contribute to reporting on results. Such information should be summarized and the additional information either eliminated or moved to an appendix. Dashboards or other reporting tools need only contain key information that will assist executives in decision-making. Sometimes simply asking the executives what measures they need to see to make decisions can identify those key indicators to report on.
Avoiding some of these all-too-common “worst practices” will decrease the amount of meaningless measurement that is in use within your organization, along with reducing the effort to gather and manage the information in the first place.
Nick Simmons is a senior consultant in the Transformation and Change practice with Interis Consulting (Nick.Simmons@interis.ca). Murray Kronick is a principal in the Planning and Performance practice with Interis Consulting (Murray.Kronick@interis.ca).
