In today’s world of heightened focus on good governance and accountability in the public sector, few would dispute that legislative and internal audits play an indispensable role.

But how exactly do auditors choose the subject matters they look at? The “audit universe” – the array of departments, agencies, policies, and programs they are mandated to audit – is seemingly unlimited. Since auditors can’t possibly look at everything, choices among competing topics must be made.

In fact, few decisions are more important for auditors than the topics to be audited, how they are chosen, and their sequence. These decisions affect the credibility, transparency, and value of the audit organization, and, ultimately, the impact on improving delivery of public services.

And yet, there isn’t much guidance available for audit organizations to consult. To help, the CCAF-FCVI has published its most recent Discussion Paper, Approaches to Audit Selection and Multi-Year Planning. CCAF is a premier Canadian research and education foundation dedicated to public sector performance auditing, governance and accountability. This Discussion Paper shares practices, experiences, and tips from audit offices from 15 jurisdictions, in Canada and internationally, as it leads users through the key decisions to be made when designing a process for audit selection and multi-year planning.

Program managers and government executives will also find it helpful to understand how audit organizations choose their topics. Why? Because their perspectives and insights are important to the audit selection process. CCAF Associate Neil Maxwell, the lead author of the paper, said, “As audit organizations aim to add value and optimize their selection of audit topics, it is an opportunity to continuously improve their outreach to stakeholders. This outreach helps audit organizations to have the information they need to choose audits that are more relevant and better meet public expectations.”

A decision support framework

Through this research project, CCAF learned about many great practices being used in the audit community. But there really is no single best practice. Our aim is to help audit organizations better understand the options available to them and decide on the approach that best suits their needs and circumstances.

The Discussion Paper sets out a decision support framework that guides users through the key questions that should be considered.

It is organized by the four phases of the audit selection and multi-year planning process (see figure 1). For each phase, the paper outlines the actions to take and the key questions to ask when making design decisions—44 questions in all. For each question, it presents the range of options and important considerations for deciding among them. Throughout, the paper shares tips and examples based on the experiences of audit offices of different sizes and natures.

Highlighted below are the questions and decisions that have the greatest impact on an audit organization’s work, organized phase by phase.

In Phase 1 – Setting Overall Direction – the audit organization sets the direction that will guide the design of the planning process. The organization will need to consider its mandate, applicable audit standards, the needs of the oversight body, and any objectives and priorities it may have set for itself. Critical decisions in this phase include:

  • How should oversight bodies’ needs influence audit selection? The oversight body (e.g. Public Accounts Committee or Departmental Audit Committee) is one of the audit organization’s primary clients and plays an important role in fulfilling public sector accountability. The organization should understand the oversight body’s characteristics and needs, and then decide how to factor these into the plan, if at all.
  • Should certain priorities or themes be given preference in audit selection? The audit organization may establish focus areas, such as specific sectors of government, First Nations’ issues, public safety and security, government-wide (horizontal) initiatives.
  • Should audits be selected only where problems are suspected? The audit organization may choose to audit only areas where performance problems are suspected or known to exist. Alternatively, it could audit areas that are deemed significant based on risk, whether or not there are indications of problems.
  • How should follow-ups of previous audits be incorporated? The organization will need to determine the right mix of new audits and follow-ups, so that sufficient attention is paid to unresolved issues it previously identified.
  • Should products and services other than audits be provided? The audit organization may choose to offer a variety of products that will be covered by its selection process and multi-year plan, including non-audit products such as document review, analysis of performance data, or survey.

In Phase 2 – Identifying Potential Audits – the audit organization determines how it will identify possible topics and assess these against selection criteria. The organization should ask the following critical questions:

  • Which internal staff should be involved, and how? Different individuals bring unique perspectives to audit selection, and involving staff can help create internal buy-in. Should all staff be engaged or just some? And what role do they play?
  • How should external stakeholders be engaged? The organization needs to determine which external stakeholders—oversight bodies, senior government officials, employee associations, civil society organizations, the media, and academics—it will consult and whether it will reach them through written communications, direct contact, or a combination.
  • How to ensure enough of the right information is collected and analyzed to select audits? The audit organization should determine, in advance, what quantity and quality of information it will need to select audits, without being excessive.

In Phase 3 – Selecting Audits and Developing a Multi-Year Audit Plan – attention turns to selecting among possible topics and developing the multi-year plan. Three critical questions are:

  • How will audit proposals be analyzed and chosen?
  • What is the right mix of audit topics, and how should they be timed and sequenced for maximum effectiveness?
  • Will there be sufficient resources and capacity to implement the plan?

Finally, Phase 4 – Applying the Plan – involves communicating, implementing, monitoring, and adjusting the plan, with an accounting after the fact. The organization will need to make the following decisions:

  • What should be disclosed? The organization will want to communicate the plan internally, and perhaps externally, to gain support for the plan and engage key stakeholders. The organization will need to decide what to communicate about the process and resulting plan, including how much detail to present about individual audits and the rationale for their selection.
  • How should the plan be monitored and adapted? Properly implementing the plan can require making adjustments as events unfold, stakeholders’ interests change, and lessons are learned.
  • How will the effectiveness of audit selection and multi-year audit planning be reported on? To report meaningfully on how well audit selection has worked, the organization might report on the relevance of the audits chosen or on the achievement of objectives, both those that governed the plan and overall objectives for the performance audit practice.

Sidebar

Sharing Canadian experience with international auditors

CCAF initially developed the Discussion Paper to meet the needs of our international partners, the national audit offices of Cameroon, Ghana, Tanzania and Vietnam. With generous funding from Global Affairs Canada, CCAF has a long-standing international capacity-building program in these offices and in their national parliamentary oversight bodies. As part of the roll-out of the Discussion Paper, CCAF also organized and hosted an intensive, two-week mentoring program for senior decision-makers from each of our partner audit offices. Several Canadian offices explained their multi-year planning processes to the participants.

“This project illustrated the great value of audit organizations collaborating and learning from one another,” said CCAF’s Vice-President of International Programs, Yves Gauthier. “Our international partners benefited significantly from exposure to Canadian audit offices’ experience, through both the Discussion Paper and the mentoring program. And it was inspiring to see how many Canadian auditors were eager to share their knowledge with the global audit community and contribute to improving audit practices in developing countries.”

CCAF’s Approaches to Audit Selection and Multi-Year Planning discussion paper is available at www.ccaf-fcvi.com. The CCAF would like to thank the Government Internal Auditors Council of Canada (GIACC) for their support and solicitation of this article.

John Reed is President and CEO of  CCAF-FCVI Inc.