Confronting the Misunderstood – Canadian Government Executive

NEWS

SEARCH

ICT
AccountabilityCommunicationProgram Evaluation
December 19, 2017

Confronting the Misunderstood

Is the role of evaluation clear enough?  Does it provide a ‘safe-space’ for managers to agree to cooperate, to take risks, and in the end, to believe in the merits of the evaluation process?  

There is growing pressure on governments around the world to be more responsive. This is true for Canada. It is no less the case for Afghanistan. This includes increased demands for good governance, accountability, transparency and the delivery of results. It suggests that ‘stewardship’ over resources and processes is simply not enough. It is a requirement that extends beyond producing program outputs. As calls for greater responsibility and results intensify, the need for the evaluation of policies, programs and projects increases. Governments and organizations do successfully implement programs and policies, but questions remain. Did the policies and programs have any impact?  Did they produce the actual and intended outcomes? This has everything to do with the much-misunderstood distinction between ‘doing things well’ versus ‘doing the right thing.’ Given the needs of stakeholders, evaluation provides vital evidence on both issues. It generates findings and conclusions about a program’s ‘relevance’, ‘adequacy’ and ‘effectiveness’, but also improved program delivery and efficiency. Taken together, the results of evaluation contribute to improved public trust in government, and confidence that public money is well-spent.

There is no debate about the usefulness of evaluation in improving management, safeguarding transparency and supporting accountability. But fear often remains within organizations. Evaluation is looked upon with suspicion. Communications and public relations departments do not like the critical aspects contained in an evaluation report. They prefer good news stories. Heads of organizations too are often uneasy that an evaluation will only underscore the weaknesses of their organization. They are concerned that political masters, boards, and central funding agencies will ignore the positives, and focus only on the negatives. For program managers and others, the loss of budget or damage to an organization’s reputation is a real nightmare. It raises concerns about the role of evaluation within the concept of public management.  Is the role of evaluation clear enough?  Does it provide a ‘safe-space’ for managers to agree to cooperate, to take risks, and in the end, to believe in the merits of the evaluation process?

For example, during 2014, development donors (including Canada) requested the Free and Fair Election Forum of Afghanistan (FEFA) to establish a monitoring and evaluation department. The intention was to track progress, measure performance, and judge the overall merits, value and worth of the project. I was appointed to establish the department, and track $US 3.5 million in program expenditures (Election Observation Mission). This started with the creation of a Monitoring and Evaluation (M&E) system. At the outset, there was much doubt among management and staff. Many questions surfaced. What was the true role of this new department and its Head? Was the M&E Head a donor insider, the ‘guy’ who reported to donors on what was really going on in the organization? While some staff had considerable knowledge of M&E, especially its obligation to report to donors, few were aware or spoke about other features of project M&E.  None referred to improved management. None spoke of about transparency and accountability or even a link to potential, additional funding from donor agencies to ensure project success.

The first year was very difficult. No amount of dialogue would convince the team about the real role of M&E.  There was considerable resistance. Some even expressed the desire to avoid the accountability entirely. Eventually, following much patience and persistence, some headway was made. Staff gradually became more familiar with basic M&E concepts. This led to a better understanding of the roles and responsibilities of the M&E Head.  As acceptance grew, attitudes and behaviors started to shift. They saw M&E as fulfilling a need. Whether to go forward with M&E, or not, ‘was simply not an option!’ Cooperation gradually grew within the organization. Requests for provision of data were agreed upon and complied with. Attendance at annual ‘critical reflection’ sessions was reassuring.  Even the value of recommendations for improvement was acknowledged and welcomed. Over time, steady progress resulted in a clearer understanding about M&E’s role, how it could help, who should participate, and the tangible results that were linked with collaborative efforts.

The results of M&E activities were also demonstrated in more practical ways. It was viewed as a function that could help overcome persistent problems challenging the organization. Again, a simple example illustrates this. There was a lack of coordination between the Administration Department and Finance Department, which delayed many of the activities conducted by the organization. A form was developed (called Form A). It was recommended that the Administration Department should complete the form and submit it to Finance at least three days prior to conducting an event or purchasing any equipment. The form was also copied to the M&E Department for tracking purposes. Within a month, significant improvements were noted. There was better coordination among the departments.  Increased efficiencies resulted.  Events were conducted on time.  Equipment arrived when needed.  No longer did staff waste time and energy in ‘blame game’ pursuits.

Gradually, resistance and acceptance of the role of M&E reversed. As knowledge increased, improved practice followed. What was originally thought to be a nightmare changed? The disaster that was expected to befall the organization, failed to materialize. In fact, it did not lead to the cutting of funds as a punishment.  What came more to the forefront was the view that M&E was a means for improvement. It was a vital tool for both management and staff.

Even the role of the M&E Head evolved.  It now serves as a focal point of contact between donors and the organization. The position is called upon to answer questions, in an open and transparent way, on the progress of all programming. It is called upon to address the challenges and risks during program implementation. A systematic and comprehensive report is provided each quarter and serves as the basis for discussions. But what is the overall take away here, especially for an M&E initiative in a faraway place such as Afghanistan? It is simple. First, senior executives should ask how familiar are they really about the merits of program evaluation? Second, before launching a M&E system, organizations should expect to take the time needed to work with staff and management.  Third, clarify what M&E is really about. Do not assume that everyone’s awareness and attitudes are clear from the start. Fourth, the friendlier the discussion, the more likely is the prospect that a good M&E system will result. Try not to be too ‘preachy’. Fifth, the appeal to concrete examples of organizations or countries who benefit from M&E practices may help ease anxieties. Thus, the clearer the conceptualization, the more likely the acceptance, and the better the opportunity for evaluators to help organizations achieve improvement goals.

As a final note, there is always room for improvement. This is especially true when working towards establishing high-level principles such as transparency, accountability and learning.  Readers should not be misled.  Such initiatives, whether in a developed or developing context, are more akin to a marathon, and should not be a sprint in ‘real-time’. It requires joint efforts by all stakeholders, whether management or staff of an implementing organization, evaluation professionals or donor funding agencies. It is an effort that is dedicated towards measuring reality, veracity and certainty.  This is what becomes the basis for enlarged trust and confidence. And within a developing world context, these efforts can often be even more challenging, and sometimes lead to perceptions of ‘Do or Die’. But it is a challenge worth taking.  The benefits and impacts, in spite of the risks, are worth pursuing.

For more information on the Free and Fair Election Forum of Afghanistan (FEFA), see http://www.fefa.org.af

Abdul Majeed is a member of the Canadian Evaluation Society (CES) and a Monitoring and Evaluation Consultant in Kabul, Afghanistan as well as an Evaluation Specialist, “Urban Development Support Project” at the World Bank. 

About this author

Abdul Majeed

Abdul Majeed

Abdul Majeed is a member of the Canadian Evaluation Society (CES) and a Monitoring and Evaluation Consultant in Kabul, Afghanistan as well as an Evaluation Specialist, “Urban Development Support Project” at the World Bank.

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Accountability
 
Prime Minister Justin Trudeau has, on many occasions, expressed his commitment...
 
In this episode, CGE radio show host J. Richard Jones speaks...
 
There is growing pressure on governments around the world to be...
 
Public servants are responsible for providing advice and support to the...
 
Today, the challenges facing governments are increasingly shifting away from traditional,...
 
Management in the Government of Canada is continuously searching for new...
 
Please to view this Content. (Not a member? Join Today! )...
 
Canadian Government Executive media through its upcoming TechGov event is providing the...
 
Ships docked in a harbour may not be going anywhere; however,...
 
The International Monetary Fund (IMF) in issuing its annual review of...
 
There’s no shortage of organizations claiming to have a digital transformation...
 
Cybersecurity professionals have sounded the alarm for years, and they are...
 
Cost estimation is becoming an extremely important skill within government due...
 
In this special episode of CGE Radio, your host John Jones...
 
Security professionals have an obligation to communicate risks and recommendations to...
 
Risk is always present in any undertaking, no matter the size...
 
Cost overruns have become institutionalized in the federal government, according to...
 
Last year, procurement Minister Jody Foote was prompted by the swirling...
 
Professionals, managers, and executives in the cost estimation industry can gain...
 
In this episode, hear from Carl Hammersburg, Manager, Government and Healthcare...
 
A new study from the Conference Board of Canada gives our...
 
In the world that we are living in today, free and...
 
The delivery method developed by Sir Michael Barber, chief adviser to...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
Canadian doctors were told that climate change impacts human health and...
 
Even as talks between the government and federal workers affected problems...
 
The largest effort in 20 years to seek public input on...
 
Ottawa has overhauled the process by which justices are picked for...
 
Please to view this Content. (Not a member? Join Today! )...
 
In this episode, editor-in-chief, Patrice Dutil talks about the need for...
 
As much a 20 per cent of grade seven students in...
 
Upon receiving numerous complaints regarding add-on fees that turn making economy...
 
Are you absolutely clear what the government wants to achieve? Are...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Please to view this Content. (Not a member? Join Today! )...
 
Written By Jason McNaught The Public Service Alliance of Canada was...
 
Independence has long been regarded as a cornerstone of the auditing...
 
Canada is a diverse nation, in language, culture, geography, and, ultimately,...
 
Public sector organizations are under increasing pressure to identify all risks...
 
The government of Canada has implemented several measures over the past...
 
Whether at the territorial, provincial or federal government level, internal audit...
 
An organization’s reputation can take years to build but it can...
 
On October 30, Ontario began second reading of the Public Sector...
 
In the well-known children’s story, an Emperor falls victim to the...
 
The Nova Scotia Office of the Ombudsman is a small operation...
 
I think the ombudsman needs to be independent, because without independence...
 
Today’s business environment changes rapidly to adjust to evolving conditions and...
 
The best internal auditors actually are really good managers first. I...
 
The recent controversy about the actions of some staff members in...
 
Most professionals don’t need more than a sentence at a cocktail...
 
Recent research by the Institute of Internal Auditors Canada aims to...
 
When is it that a politician becomes part of the governing...
 
It’s been a busy couple of weeks on the information, privacy...
 
In 1996, a new budget watchdog, the Parliamentary Budget Officer, was...
 
The Ontario government is moving forward with the creation of a...
 
The Office of the Public Sector Integrity Commissioner of Canada (PSIC)...
 
After the Auditor General’s (AG) report was released on April 30,...
 
In the U.K. system, Permanent Secretaries are what we call Deputy...
 
We are living in a period of rapid change and limited...
 
US public sector employees don’t trust their management to do the...
 
The news of Mark Carney’s nomination as the new Governor of...
 
Following Singapore’s independence in 1965, the controversial leadership of Lee Kuan...
 
Over the past few years, the preparation and delivery of the...
 
In healthcare, cost-cutting can result in cutting what is valued most...
 
For over 20 years Colin Bennett has been exploring issues of...
 
Even before controversy shook the organization to its foundation, Ornge was...
 
It will be the largest international multi-sport event ever held on...
 
We’ve all seen the headlines – BC Ferries, Ornge, la Caisse...
 
It can happen, and it’s noteworthy when it does. Government, business...
 
Kevin Page’s mandate as the first Parliamentary Budget Officer comes to...
 
Governments are challenged to meaningfully mitigate the effects of the financial...
 
Canada is facing a huge financial challenge brought on by massive...
 
For the past one hundred years, democratic states have been moving...
 
It’s so much easier and less painful to learn from the...
 
CGE Vol.13 No.2 February 2007 Canada’s Performance 2006 is the sixth...
 
CGE Vol.14 No.1 January 2008 The furor over the $300,000 that...
 
CGE Vol.13 No.7 September 2007 "If the Public Service, as a...
 
When pondering leadership, we immediately think of exercising our influence downward...
 
CGE Vol.13 No.1 January 2007 "How can I be held accountable...
 
CGE Vol.14 No.2 February 2008 Let’s say you’re a senior manager,...
 
CGE Vol.13 No.1 January 2007 Perhaps it’s a legacy of the...
 
Last fall, Alberta’s Employment and Immigration department posted online the workplace...
 
As the global economy struggles to regain some forward momentum, Canadian...
 
This will be a defining budget for Stephen Harper. It will...
 
It is difficult to determine when the debate about the need...
 
For the next few years, the federal government’s overarching agenda will...
 
Much of the current conversation about the federal government’s economic agenda...
 
In 2006 the world was feeling the aftershocks of a number...
 
The Independent Blue Ribbon Panel on Grants and Contributions called for...
 
Au Canada, le secteur bénévole et à but non lucratif vit...
 
CGE Vol.13 No.4 April 2007 Robert Parkins, editorial director, met recently...
 
Bill Greenlaw is the elected president of the Institute for Public...
 
In the past two decades, the nature of the state has...
 
Have you ever asked yourself the question: ‘How would I evaluate...
 
Please to view this Content. (Not a member? Join Today! )...
Prime Minister Justin Trudeau has, on many occasions, expressed his commitment...

Member Login

Forgot Password?

Join Us

Password Reset
Please enter your e-mail address. You will receive a new password via e-mail.