ManagementPerformance MeasurementProgram EvaluationPublic Sector
June 20, 2016

Putting PIUs First: A Private Sector Perspective

An external evaluator can be especially helpful in identifying the primary intended users (PIU) of the evaluation. Key to the approach is scoping out the primary purposes and intended uses of the evaluation. The external evaluator’s relationship with the PIUs is key to evaluation success.

As a consultant, I work primarily with private sector organizations including the “big-five” banks, professional governing bodies and regulators for accountants, lawyers and trustees in bankruptcy. These businesses have deeply rooted performance measurement cultures, but are often unfamiliar with program evaluation. Managing evaluations within these organizations often entails accentuating evaluation capacity building while simultaneously enhancing the utility of the evaluation process and its findings.

In situations like these, I turn to Michael Quinn Patton’s Utilization-focused Evaluation. Patton emphasizes placing primary intended users (PIUs) at the centre of the evaluation. His approach describes multiple steps to managing evaluations while promoting the utility of the evaluation. Parts of his framework focuses on the need to build relationships that promote not only evaluation management, but also evaluation capacity.

Patton also describes how role definition is important not just for the external consultant but for the organization’s personnel. An external evaluator can be especially helpful in identifying the primary intended users (PIU) of the evaluation. Key to the approach is scoping out the primary purposes and intended uses of the evaluation. The external evaluator’s relationship with the PIUs is key to evaluation success. It entails meeting the company’s evaluation objectives. It must also ensure that evaluation findings can be used in program-related decisions. The external evaluator can also lead by example in engaging stakeholders in meaningful decisions linked to the evaluation function.

Patton contends that fostering evaluation is a personal mission. My experiences confirm this. Cultivating relationships with people in the organization that want to learn about the evaluation process and its ties to program improvement is a key to enhanced evaluation use. These people do not just represent the authority in the organization. They are strategically located. They are close to the programs. They are enthusiastic, committed, competent and curious about evaluation.

The adoption of a “user-friendly” evaluation tactic that emphasizes collaboration, participation and empowerment increases the odds that evaluations will be utilized, and deemed a valuable process in the company or corporation.

Figure 1 summarizes Patton’s approach to managing evaluations for enhanced utility. In my experience, the multi-step method serves best as a road map for the external consultant and private sector organization. The clear logic is that all interested parties must be convinced about what the evaluation process will deliver. They must know what their roles are. This goes a long way to creating comfort with evaluation, especially in an organizational culture where the reflex is not in that direction of evaluation. The framework is general enough to be flexible and it can be adapted. It can assist any organization build evaluation organizational capacity to adjust to the evaluation process.

 

Pattons

Figure 1
Patton’s Methodology for Managing Evaluations for Greater Utility

Successful implementation

The first step in my implementation of evaluation activities and simultaneous capacity building is to first establish stakeholder needs. It must identify primary intended users. The external evaluator must clearly assist the organization in defining roles and responsibilities of all stakeholders. These include the PIUs such as (a) the organization’s leadership through a project advisory board/committee; (b) the evaluation proponent or facilitator within the organization–which I refer to as the “evaluation champion” –and (c) the program operation staff. Having clear roles and responsibilities allows the external evaluator to forge relationships of different natures with the various interested parties.

It also helps to scope the time-commitments of various parties. While the evaluator sports many hats in these relationships, s/he wears only one at a time. For example, in meetings with organizational leadership, the evaluator is the expert, hired by the organization to complete a project, and must convey progress on bringing that project to its successful completion. But when the evaluator interacts with the organization’s evaluation champion, the former becomes an enabler, empowering the champion to make sense of the evaluation process and resultant data, and to sustain evaluative efforts after project completion.

I have been often asked why is there the need for an evaluation management framework to build these types of relationships. My response is that the framework allows all involved parties to see where they are on the evaluation roadmap. Once roles and responsibilities are identified, each party within the company better understands the evaluation process and where they fit. The feeling of comfort and reassurance that this provides cannot be underestimated. The framework’s orientation function may seem elementary, but in organizations with no evaluation culture, it is immensely beneficial in building such capacity of the individuals involved.

Lessons learned

The multi-step framework and its use as a road map, especially in stakeholder analysis and identification of PIUs to clearly delineate roles and responsibilities, also helps in building of relationships within the organization. The collective effort made to understand the historical context of the organization and program is invaluable. It is important for external evaluators to know what efforts have been made before to bring about program improvement, how well they worked, and who was involved in the process.

Most people are naturally reluctant to venture down a road they have already unsuccessfully travelled. They are especially suspicious about being led by someone external to their company. The nature and strength of relationships forged via the framework also provides confidence to the evaluator in adherence to ethical and professional obligations without fear of reprisal during times of organizational pushback. The fostering of relationships within a context of professional conduct helps to ensure relevance of evaluation processes and utilization of results within the organization on an ongoing basis. Here, the external evaluator is viewed not only as the “expert” but also the “enabler.”

Though I have had much success using this PIU-focused evaluation framework, not all evaluations have run smoothly. Recently, my team and I completed a comprehensive evaluation of the national certification program for a group of financial professionals. The framework identified the course for the evaluation, clarified people’s roles, identified PIUs and marshalled the efforts of an internal evaluation champion. When it came time to make our recommendations, which would have improved and altered the course of the program, we received a great deal of organizational pushback. But the organization’s evaluation champion echoed our opinions. Further, the capacity we had been building with the executive steering committee allowed them to better understand the evaluative processes and resulting recommendations.

Because we fostered strong relationships within the organization, our suggestions were not viewed as threatening. In the end, our recommendations were implemented, some program operations were altered and an improved program emerged. I do not believe that any of this could have realized if not for the application of the multi-step evaluation framework and its focus on PIUs. It provides a clear path, where few alternatives exist, for evaluation management and capacity building within a private sector organization that lack an evaluation culture.

A. Sidiq Ali, PhD CE, is Managing Director of Research & Evaluation Consulting, Inc. His blog is www.rae-consult.com

About this author

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Management
 
The Government of Canada was the nation’s largest employer with 274,370...
 
A Globe and Mail journalist made an interesting observation of trade...
 
According to the 2018 Mercer Global Talent Trends Study , 39 per cent of employees would leave...
 
The Ontario Public Service (OPS) is at a crossroads. The 2018...
 
Summary and Recommendations Labour markets in Canada and around the world...
 
Buying Social. Responsible sourcing. Ethical procurement. Sustainable procurement. Whatever name you...
 
In this episode, Lori Turnbull discusses the positive side of disruption,...
 
When consultant Scott Eblin started working on The Next Level in 2004, he was...
 
The “Magic Loogie” scene is vintage Seinfeld. It is a pastiche...
 
Carpenters and electricians have tools: hammers, saws, pliers, plungers and flashlights....
 
Lately, all levels of the Canadian public service have devoted much...
 
The word ‘dysfunctional’ is often used to describe individuals, families, relationships,...
 
When James Cattell, a delivery manager at the UK’s Department for...
 
Kotter International found that 90 per cent of managers and employees...
 
In this episode of CGE Radio, J. Richard Jones talks with...
 
In the 5th century BC, Nehemiah rebuilt the walls of Jerusalem....
 
In the spring of 2018, the Auditor General of Canada, Michael...
 
A few years back, consultants with ghSMART told us the biggest...
 
Wildlife Biologist, professional hockey player and public servant, born and raised...
 
At work, we are often trying to satisfy a bundle of...
 
Large-scale change occurs when a sizeable body of people gets behind...
 
We all ask why governments fail, often spectacularly, to carry out...
 
Why is strategic planning so dreaded? How often do we actually...
 
As the resettlement of Syrian Refugees exemplifies, government programs, currently running...
 
Over the last several months, as a participant in the OCG’s...
 
High in the Arctic, the Inuksuk is a timeless cultural symbol...
 
At the 2017 IPAC conference in Charlottetown, George Ross, Editor-in-Chief of...
 
Can government remember? Is it condemned to repeat mistakes? Or does...
 
‘Elephant in the room’ is a metaphorical idiom for an obvious...
 
Compared to audit professionals and their preoccupation with “independence”, evaluation specialists...
 
Having a clear definition of success is one of the key...
 
Time is of the essence for Alice’s White Rabbit. In business,...
 
Professional development is tremendously important to members of Canada’s public sector....
 
The International Civil Service Effectiveness (InCiSE) Index project, a collaboration between...
 
Yogi Berra, Maurice (The Rocket) Richard, Bill Russell, Michael Jordan, and...
 
Three years before his death in 2011, Jack Layton released a...
 
In today’s workplace, individuals increasingly face dynamic and difficult challenges that...
 
We are happy to share with you the September 2017 issue...
 
The health care system in Canada, known as Medicare, is publicly...
 
In March, the Canada Revenue Agency’s website was shut down by...
 
The answer my friends is blowin’ in the wind … BOB...
 
Water is essential to sustain all forms of life and more...
 
We are thrilled to offer the July/August issue of Canadian Government...
 
We are happy to share with you the May/June issue of...
 
The International Monetary Fund (IMF) in issuing its annual review of...
 
In about two weeks, APEX will be holding its 2017 Symposium...
 
As governments around the world move towards the digital transformation of...
 
Cost estimation is becoming an extremely important skill within government due...
 
We have all heard the sayings: “Home, sweet home”; “There’s no...
 
Risk is always present in any undertaking, no matter the size...
 
As Canada’s government agencies strive to improve delivery and efficiency under...
 
Last year, procurement Minister Jody Foote was prompted by the swirling...
 
Bill Gates once said that “Technology is just a tool. In...
 
Sustainable Development Goals (SDGs) are part of the new global “Agenda...
 
Laskey Hart guest hosts CGE Radio this week with a summary...
 
With baby boomers continuing to hang onto their office desks and...
 
In every industry, the competition for the talent pool is on...
 
The Conference Board of Canada recently published a report titled Leveraging...
 
While internal auditing has been considerably strengthened in most jurisdictions in...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
Cyber-attacks threaten Canadian individuals and companies every minute of every hour....
 
Ontario’s provincial government needs to act fast in building a coordinated...
 
The bootleg fentanyl overdose crisis that is sweeping across Western provinces...
 
The association representing more than 42,000 physicians and medical students in...
 
A study conducted by Robert Half, the world’s largest specialized staffing...
 
A recent report suggests several strategies how governments and the private...
 
The latest information from IBM Cloud covers: Consolidating Complex Environments Consolidating...
 
In order to find out which strategies and priorities CFOs are...
 
A small but rapidly growing number of businesses have started hiring...
 
Sometimes it’s not the job that counts, but what comes after....
 
The possibility of mail delivery disruption on Friday this week loomed...
 
CFOs around the world are changing – but an EY survey...
 
There’s a fine line between confidence and naivety – a line...
 
Around the world, more and more workers are seeking employment. At...
 
Canada Post today warned the public about the potential for a...
 
In a little more than a year, Canadian banks and lending...
 
Digitizing certain aspects of the hiring process appeals to both employers...
 
As populations grow and age, the demand for services increases. As...
 
As a consultant, I work primarily with private sector organizations including...
 
On June 7, over 40 senior executives from within the public...
 
Dispersed organisations can be very powerful structures. There is ample evidence...
 
By Nicole Verkindt Despite growing up in a small town in...
 
The agency responsible for safeguarding the Pentagon and several other buildings...
 
The actions and decisions of public servants have consequences for the...
 
By Jason Zhang  The best way to cost effectively manage the retention...
 
Protests by irate taxi drivers, roadside scuffles involving cabbies and Uber...
 
He is often referred to as the federal government’s first “chief...
 
By Michael Murphy Not all assets can and should be equally...
 
When I find myself in times of trouble, Mother Mary comes...
 
People caught a glimpse of the work of partisan advisers in...
 
Copyright owners have struggled to protect their intellectual property since the...
 
Canada’s largest organization advocating for retirees from federal, police and military...
 
Here’s some bad news for organizations contemplating projects that have something...
 
The adoption last year by the Canada Border Services Agency of...
 
Two years ago, Bixi, a not-for-profit, para-municipal bicycle-share firm of the...
 
The melting Arctic ice caused by climate change is opening up...
 
The Prime Minister is now well into the consolidation phase of...
 
Canadians expect their public services to be delivered in a way...
 
It appears it was not just the Mounties that were affected...
 
Every reputable learning institution offers leadership development training nowadays. So why...
 
Many programs at all levels of government provide funds to community-based...
 
Nearly half of Canadian organizations are falling behind on implementation of...
 
If you want to improve your management procedures, search Google. No,...
 
A while back there have been numerous media reports about the...
 
Q: Where is Comptrollership going? Comptrollership in the Government of Canada...
 
Performance auditing can lead to more efficient, effective, and economical program...
 
Rushing to keep up with deadlines but never getting ahead is...
 
The idea of converged infrastructure and converged systems in the datacentre...
 
Hackers are zeroing in on users of SSL/TLS encryption and no...
 
That the new Liberal Government has embraced ‘open government’ is hardly...
 
Meet Bob Heart.  He is an outstanding employee who works hard...
 
A new study released yesterday, Securing the C-Suite, Cybersecurity Perspectives from...
 
The bloodletting at once-mighty Canadian mobile tech giant, BlackBerry continues as...
 
There is no shortage of examples of businesses that effectively used...
 
Rogers Media announced yesterday cost-cutting moves that will see some 200...
 
Microsoft, on Tuesday, ended support for older versions of Internet Explorer...
 
Written by Tim Wacker Almost a quarter century ago, when most...
 
Last week the Wall Street Journal, citing anonymous sources, reported that...
 
Written By Jason McNaught The Public Service Alliance of Canada was...
 
A leader’s credibility begins with personal success. It ends with helping...
 
Written by Donald Farmer Too often, we base Business Intelligence today...
 
Written by  Gail Vallance Barrington Anyone who has commissioned a program...
 
In late July, Amazon startled investors with the tiniest of profit....
 
Skills International, a global talent and career management firm, has launched...
 
When it comes to training, statistics show small businesses are more...
 
Today, TD Canada Trust released the results of a survey studying...
 
Management has its upsides, but no one said it would be...
 
Independence has long been regarded as a cornerstone of the auditing...
 
Congratulations. You’ve achieved professional recognition for your performance and the public...
 
In 2011, PwC began a conversation with global defence leaders on...
 
Canada is a diverse nation, in language, culture, geography, and, ultimately,...
 
Public sector organizations are under increasing pressure to identify all risks...
 
The government of Canada has implemented several measures over the past...
 
The economy is sluggish…oil prices are down…stock markets are up but...
 
Whether at the territorial, provincial or federal government level, internal audit...
 
On a cold, grey Sunday afternoon last November, about 2,500 residents...
 
An organization’s reputation can take years to build but it can...
 
Change definitely happens, and it is happening faster and faster as...
 
On October 30, Ontario began second reading of the Public Sector...
 
commentEmail””:””ipfrazer@ipfconsulting.com”” Related posts: Looking Forward to Looking Back...
 
The success of organizations, and the Canadian economy, is directly tied...
 
In the well-known children’s story, an Emperor falls victim to the...
 
The Nova Scotia Office of the Ombudsman is a small operation...
 
I think the ombudsman needs to be independent, because without independence...
 
Today’s business environment changes rapidly to adjust to evolving conditions and...
 
There’s a cloud looming over public sector employers in Canada. Baby...
 
“”corporate governance”” Related posts: Blueprint 2020 an opportunity...
 
The best internal auditors actually are really good managers first. I...
 
In the middle of an empty field on the edge of...
 
The recent controversy about the actions of some staff members in...
 
Most professionals don’t need more than a sentence at a cocktail...
 
Recent research by the Institute of Internal Auditors Canada aims to...
 
Today’s tasty hamburger bun is a marvel of industrial cuisine and...
 
In this last week before Christmas and the holidays, this is...
 
As part of the Public Policy Forum’s university lecture series, Mel...
 
Respecting, processing and cross-pollinating thoughts from diverse cultures will advance team...
 
When is it that a politician becomes part of the governing...
 
It is something that few senior managers spend much time thinking...
 
It’s been a busy couple of weeks on the information, privacy...
 
In 1996, a new budget watchdog, the Parliamentary Budget Officer, was...
 
The publication of Donald Savoie’s latest book, Whatever Happened to the...
 
The Ontario government is moving forward with the creation of a...
 
He blew up the factory! Jack Welch, as CEO of General...
 
There is no doubt that the challenges faced by governments are...
 
When printing press technology was diffused through Europe, adult literature preceded...
 
The Office of the Public Sector Integrity Commissioner of Canada (PSIC)...
 
After the Auditor General’s (AG) report was released on April 30,...
 
Entering 2013, Canadian employers and unions, public and private sector alike,...
 
In the U.K. system, Permanent Secretaries are what we call Deputy...
 
As both a management consultant and a Boston Marathoner who was...
 
I believe that the greatest challenge and opportunity for management is...
 
The Asian Development Bank estimates that the global demand for infrastructure...
 
n Saskatchewan, a drug shortage team in the Ministry of Health...
 
Three decades ago, the federal public service was proportionately larger, generally...
 
We are living in a period of rapid change and limited...
 
US public sector employees don’t trust their management to do the...
 
The news of Mark Carney’s nomination as the new Governor of...
 
Following Singapore’s independence in 1965, the controversial leadership of Lee Kuan...
 
Over the past few years, the preparation and delivery of the...
 
In healthcare, cost-cutting can result in cutting what is valued most...
 
For over 20 years Colin Bennett has been exploring issues of...
 
Even before controversy shook the organization to its foundation, Ornge was...
 
It will be the largest international multi-sport event ever held on...
 
We’ve all seen the headlines – BC Ferries, Ornge, la Caisse...
 
It can happen, and it’s noteworthy when it does. Government, business...
 
There seems little argument in the wake of the financial crisis...
 
Kevin Page’s mandate as the first Parliamentary Budget Officer comes to...
 
Governments are challenged to meaningfully mitigate the effects of the financial...
 
Canada is facing a huge financial challenge brought on by massive...
 
For the past one hundred years, democratic states have been moving...
 
It’s so much easier and less painful to learn from the...
 
Mandated by Congress in late 2010, the U.S. federal government is...
 
CGE Vol.13 No.4 April 2007 "The potential to lead is in...
 
CGE Vol.13 No.2 February 2007 The Connecting the Dots series is...
 
CGE Vol.13 No.2 February 2007 For too many managers, "crisis" is...
 
CGE Vol.14 No.3 March 2008 Organizations communicate "what the real game...
 
CGE Vol.14 No.2 February 2008 Community Safety and Correctional Services is...
 
CGE Vol.13 No.1 January 2007 Leadership How would you describe your...
 
CGE Vol.14 No.3 March 2008 If you spend much time at...
 
CGE Vol.13 No.5 May 2007 How often has a new employee...
 
CGE Vol.13 No.7 September 2007 "If the Public Service, as a...
 
When pondering leadership, we immediately think of exercising our influence downward...
 
CGE Vol.13 No.1 January 2007 "How can I be held accountable...
 
CGE Vol.13 No.2 February 2007 Canada’s Performance 2006 is the sixth...
 
CGE Vol.14 No.1 January 2008 The furor over the $300,000 that...
 
CGE Vol.14 No.2 February 2008 Let’s say you’re a senior manager,...
 
CGE Vol.13 No.1 January 2007 Perhaps it’s a legacy of the...
 
Managing public sector organizations requires living with high levels of complexity...
 
The significant challenges created by the global recession have had a...
 
Saskatchewan, like other provinces, spends almost half its budget on healthcare....
 
Building project management and business transformation capacity is critical to advancing...
 
You’ve heard it many times: public service leaders are accountable for...
 
Citizens want to talk to politicians and policymakers to explain what...
 
Dans le monde d’aujourd’hui, les entreprises, les organismes et leurs clients...
 
if possible I would love to ask you a few other...
 
En parcourant tous les diagrammes techniques qui couvrent des cartes murales...
 
On pense souvent que la fonction publique devra recruter un grand...
 
Jumeler les compétences opérationnelles avec le souci du service, les aptitudes...
 
La complexité des programmes publics ne date pas d’hier. Nul ne...
 
Après avoir consacré trois années à la mobilisation des employés et...
 
CGE Vol.13 No.9 November 2007 The new expenditure management system announced...
 
As the Public Service Commission of Canada celebrates its 100th anniversary...
 
With the hotly contested Canadian and American elections behind us, the...
 
CGE Vol.13 No.8 October 2007 A few weeks ago, the federal...
 
Henry Mintzberg is Canada’s most celebrated management thinker. He and colleagues...
 
Stephen M.R. Covey has sold 600,000 copies of The Speed of...
 
Comment les cadres et les élus des villes dirigentR09;ils les services...
 
A decade ago Canadian Blood Services took over a troubled agency...
 
Ne vous fiez à personne de plus de 30 ans, déclarait...
 
CGE Vol.13 No.10 December 2007 Demographics and technology, two of our...
 
Are you a good boss? Or a bad boss? Or a...
 
Most government executives, most of the time, are focused on dealing...
 
Like most people, public service leaders intuitively understand the value of...
 
Lee McCormack, ancien cadre supérieur du gouvernement, est directeur de la...
 
The public service strives to achieve positive outcomes with appropriately targeted...
 
The Independent Blue Ribbon Panel on Grants and Contributions called for...
 
As the global economy struggles to regain some forward momentum, Canadian...
 
Au Canada, le secteur bénévole et à but non lucratif vit...
 
This will be a defining budget for Stephen Harper. It will...
 
It is difficult to determine when the debate about the need...
 
Bill Greenlaw is the elected president of the Institute for Public...
 
For the next few years, the federal government’s overarching agenda will...
 
Have you ever asked yourself the question: ‘How would I evaluate...
 
Much of the current conversation about the federal government’s economic agenda...
 
Last fall, Alberta’s Employment and Immigration department posted online the workplace...
 
In 2006 the world was feeling the aftershocks of a number...
 
CGE Vol.13 No.4 April 2007 Robert Parkins, editorial director, met recently...
 
In the past two decades, the nature of the state has...
 
In recent months, the attention of Canadians has been focused on...
The Government of Canada was the nation’s largest employer with 274,370...