DevelopmentGovernmentLeadershipPerformance
July 24, 2017

Using Behavioural Insights to Improve Government Programs and Services: The Case of Job Match

Departmental officials from all levels, particularly senior management, must view testing and experimentation as being all about trial and error and not about pointing out failure. Experiments such as these are important if we are to determine what works and what does not.

Behavioural Insights (BI) is a multidisciplinary approach that uses principles and methodologies from the behavioural sciences such as behavioural economics, marketing, and psychology. It is attracting considerable attention from researchers and policy makers around the world as a valuable and promising tool to improve governments’ programs, service delivery processes and relevant client outcomes. In the Government of Canada, interest for this approach has also widened in recent years and an increasing number of departments and agencies (e.g., the Privy Council Office, Natural Resources Canada, Canada Revenue Agency) are now actively promoting its use in areas of relevant interest.

BI provides policy makers with effective tools to affect positive behavioural changes among individuals. Among them, the so-called nudges have been used in a variety of contexts. They are light-touch interventions that can help individuals to make positive behavioural changes without enforcing limitations on the choices that are available to them.

In late 2015, Employment and Social Development Canada (ESDC) put in place a small BI research team within its Strategic and Service Policy Branch. (The research team is situated in the ESDC Innovation Lab.) This multidisciplinary team, composed of highly skilled analysts coming from various areas of expertise and backgrounds (economics, public administration, psychology, political science), is now leading more than half a dozen projects in various social, labour, and learning policy areas using rigorous research designs.

Nudging Job Bank Users: A Behavioural Economics Experiment is the first BI project to be conducted by the team. Conducted between early 2015 and late 2016, it consists of a series of trials that sought to measure the impacts of introducing nudges into e-mails to Canadians to improve the uptake of the Job Match service. This article describes the context, the nudges developed, and the key findings from this first trial. The conclusion section also presents a few lessons learned.

Case Study: Increasing Job Match Uptake

Job Bank is the Government of Canada’s main source for jobs and labour market information, and Job Match is one of its key tools. Launched in late 2015, Job Match connects job seekers to available jobs that fit their skills, experiences, and knowledge.

To benefit from this service, users need to first complete and activate an online profile about their skills and experience. Evidence from internal administrative data showed, however, that many users created their user account, but never got through to completing it. We hypothesized that this can happen due to a number of reasons, including not having some of the information necessary to complete their user account, frustration with the amount of details requested, fatigue, and procrastination. The main behaviour that we thus wanted to target in this first trial was increasing the number of users returning to the Job Bank website and completing their user account by addressing some of these behavioural barriers.

We decided to use a simple method: developing a trial where we sent e-mails to 3,800 users who hadn’t completed their user accounts and using behavioural insights to nudge them in completing it. In doing so, we experimented with four ideas from behavioural science to see what would work best in the context of Job Match:

  1. Framing – Simple changes in the way a particular message is presented/worded can have a substantial influence on people’s behaviour.
  2. Social norms – When people perceive a particular behaviour to be common among others, they are more likely to engage in that behaviour themselves.
  3. Call-to-Action/Salience – People are heavily influenced by what catches their attention and is most noticeable. Focusing the attention of users on one clear action to be taken (e.g., presenting the desired action on a large button with a contrasting background color) can increase the likelihood that they will engage in this action.
  4. Sense of Commitment – People are more likely to follow through with a goal-directed behaviour if they perceive themselves to be committed to the broader goal.

This led us to design five different versions of the e-mail we sent to users with incomplete user accounts. The first e-mail was a standard program e-mail that was not explicitly informed by BI and this served as our ‘control condition.’  Each of the remaining four e-mails included a ‘nudge’ based on one of the four behavioural principles described above. For example, the nudge based on the social norms principle said, “Thousands of Canadians have already completed their Job Match profiles and are being matched to new job opportunities.” To build on users’ sense of commitment, another nudge said:

You are only one step away from being matched with an employer looking for your skills and experience:

  •  I want to find a new job
  •  I am using Job Bank
  •  I have a Job Seeker Account
  •  I have completed my Job Match profile

All our BI-informed e-mails were identical except for the inclusion of these nudges. To compare the impacts of these e-mails, we used a Randomized Controlled Trial (RCT). RCTs are considered to be the ‘gold standard’ in impact evaluation because they enable us to test the impact caused by the nudges we’re interested in, while controlling for other external variables. To do this, we randomly assigned participants into five groups (≈757 participants each), and each group received one of the five e-mails (i.e., one of the four e-mails with nudges or the standard e-mail without the nudge). We then tracked the number of users clicking on the link in the e-mail, as well as the number of users completing and activating their Job seeker profile with each of these e-mails.

We found that, on average, our four BI-informed e-mails increased the number of people clicking on the link by 77 percent, compared to the standard e-mail. Specifically, BI-informed e-mails generated an average of 184 clicks each, whereas the standard e-mail generated 104 clicks.1 Among those who clicked on the link, the BI-informed e-mails generated 43 profile activations on average, compared to 21 activations with the standard e-mail (a 106 percent improvement). The differences in impact between the four BI-informed e-mails were minor and were not always statistically significant, suggesting that the four behavioural principles worked equally well in this context.

Overall, this small experiment demonstrated that nudges do matter. It showed that the rigorous application of behavioural principles can be a simple, cost-effective way of increasing program take-up. We conducted a number of other trials to explore different touch points through which BI can be leveraged to increase the number of users benefitting from the Job Match service. These trials and the sharing of their key findings across the department have helped establish the usefulness of BI tools as an innovative way to improve our programs and the delivery of our services. This in turn has generated increased interest in the Department for its application in other areas.

Lessons learned

This first ESDC nudge trial implemented in the context of the Job Match service taught us a number of useful lessons regarding the use of behavioural principles in a public policy context. In particular, the success of this type of initiative requires the presence of a number of key conditions:

  • Leadership from middle level managers as well as a buy-in from senior management.
  • Continuous and close collaboration between the program experts and departmental colleagues with complementary expertise in disciplines such as research, web system and program policy.
  • Access to high quality data and the promotion and use of rigorous research designs allowing reliable and robust measures of impacts in order to generate credible results.
  • Research protocols must also be flexible enough to adapt to program and IT constraints inherent in any organisation.

Last but not least, departmental officials from all levels, particularly senior management, must view testing and experimentation as being all about trial and error and not about pointing out failure. Experiments such as these are important if we are to determine what works and what does not.

 

Mathieu Audet, Stéphane Gascon, Hasti Rahbar, Monica Soliman are all members of the  Service Research Division, Strategic and Service Policy Branch, Employment and Social Development Canada. The views expressed here are their own, and do not represent government policy.

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Development
 
Don’t ignore that gut feeling.             Too often we clutch at...
 
There has been a lot of discussion around the future of...
 
Earlier this year, the federal government kicked off an auction for...
 
Canada’s public sector can help set the tone for a strong...
 
OTTAWA, July 31 – CPA Canada, (Chartered Professional Accountants of Canada) announced their Public Sector Conference ,...
 
Thanks to the development of new technologies that make previously unimaginable...
 
Since debuting in 2000, Survivor has traversed the world and humanity...
 
In late March the ONS detailed the occupations in the UK...
 
In the next year, one in three US federal government workers...
 
Despite a rapid expansion in digital democracy initiatives around the globe,...
 
In the show, J. Richard Jones talks with Colin Earp, Partner...
 
Deforestation interventions in Latin America and Asia may be more effective...
 
Below is a piece of design fiction focused on what it...
 
Service delivery is an integral part of the work of governments...
 
In recent years, growing evidence about the importance of early childhood...
 
In the past 18 months or so, new governments have taken...
 
Could robot doctors be on the horizon? A study published February...
 
We are excited to announce that the February/March 2019 issue of...
 
The use of “nudges” in policymaking has been a major trend...
 
There is an interesting paradox playing out with the public sector...
 
Last week, Perpetual Guardian, a New Zealand-based financial services company, announced...
 
Canadian Government Executive, in collaboration with Dalhousie University, is seeking your...
 
Imagine your favorite world class service provider replacing the Board of...
 
Lately, all levels of the Canadian public service have devoted much...
 
We have been hearing quite a lot about superclusters. The Government...
 
In this episode of CGE Radio, J. Richard Jones talks with...
 
In 2017, identity theft cost Americans $16.8 billion, and, in just...
 
In this episode of CGE Radio, J. Richard Jones interviews Alex...
 
It is time to reboot democracy. This year, the French constitution,...
 
We are excited to share with you the October/November 2018 edition...
 
Ontario Northland is a crown corporation that provides transportation solutions to...
 
Canada’s international reputation for welcoming and integrating newcomers is unparalleled. At...
 
In the three years to 2020, Canada will welcome almost a...
 
Mining embodies progress. Mining provides civilization with the materials required for...
 
On today’s show, J. Richard Jones sits down for a chat...
 
We are excited to share with you the May/June issue of...
 
In this episode of CGE Radio, J. Richard Jones speaks with...
 
As the world grows more digital – and increasingly mobile –...
 
Throughout Canada, healthcare systems are challenged by population growth, aging, and...
 
Over half a billion children live in countries that are falling...
 
According to a recent McKinsey study there is a potential for...
 
On the global scene, technology has revolutionized and automated the work...
 
Today on CGE Radio, we speak with Craig Szelestowski, President and...
 
The adoption of new technologies provides many opportunities, poses unique challenges...
 
“Culture eats strategy for breakfast.” This quote from management specialist Peter...
 
On a recent visit to Ottawa, I had the opportunity to...
 
Professional development is tremendously important to members of Canada’s public sector....
 
Over the last two years, an ever-growing number of organizations around...
 
Ships docked in a harbour may not be going anywhere; however,...
 
Canadian Government Executive Media (CGE) and CATAAlliance , Canada’s one voice for...
 
Do Labour Market Development Agreements (LMDAs) make a difference? Our evaluation...
 
The design needed and practiced in government departments and the broader...
 
As social media becomes ever-more prevalent in the lives of today’s...
 
Evaluation is performed for a range of reasons:  to improve programs,...
 
Radical candour sounds rather outré as a prescription for government executives....
 
In this episode of CGE Radio, J. Richard Jones is joined...
 
Behavioural Insights (BI) is a multidisciplinary approach that uses principles and...
 
In March, the Canada Revenue Agency’s website was shut down by...
 
How does a mandate for innovation challenge evaluation? Increasingly, the public...
 
As the face of the Association of Professional Executives of the...
 
One of the writers in this month’s issue started a note...
 
Excellence in leadership is instrumental in serving the public interest. New-age...
 
Canadian Government Executive Media (CGE) announced last Friday, that George Ross...
 
We are thrilled to offer the July/August issue of Canadian Government...
 
At this time a year ago, the 88,000 displaced people of...
 
“Former Metro banker stole $73,000.”  “Enbridge sues ex-employee.”  “Woman stole $51,000...
 
The recent introduction of the new Government of Canada (GC) IT...
 
So, what can be said about Canada’s public administration as the...
 
The way forward to engaging Millennials in the public sector Passionate,...
 
We are happy to share with you the May/June issue of...
 
The Canadian Advanced Technology Alliance (CATA Alliance) is Canada’s One Voice...
 
Since Confederation in 1867, more than 17 million immigrants landed in...
 
In today’s world of heightened focus on good governance and accountability...
 
Orchestrating the transition from present to emergent leadership Millennial is the...
 
As ‘month 13’ for Canada’s Syrian refugee arrivals comes and goes,...
 
As governments around the world move towards the digital transformation of...
 
The way organizations recruit, shape and manage their workforce to find...
 
The promise of web 2.0 was the emergence of a user-driven...
 
Commodity Management (the official term for life-cycle asset management of goods...
 
During the past decade, the international university-level student population in Canada...
 
We have all heard the sayings: “Home, sweet home”; “There’s no...
 
Imagine a Canadian government that shares best practices in service delivery....
 
There is a new buzz in policy circles. Advances in data...
 
The North Atlantic Treaty Organization, with its 28 independent member countries...
 
The internal audit function in the Canadian public sector finds itself...
 
As Canada’s government agencies strive to improve delivery and efficiency under...
 
To boldly go where no one has gone before STAR TREK:...
 
In recent months, Canadians have borne witness to a new cycle...
 
Professionals, managers, and executives in the cost estimation industry can gain...
 
Canadian Government Executive kicked off its CGE Leadership Series for 2017...
 
The rapid advances in digital technology, demographic and societal shift even...
 
The critical challenge facing the public service is changing its culture....
 
Having the right talent is critical to delivering right results. But...
 
Sustainable Development Goals (SDGs) are part of the new global “Agenda...
 
An Interview with Larry Ostola, Toronto’s Director of Museums and Heritage...
 
For all its subtleties and mysteries, the Westminster system of government...
 
When Marshall McLuhan coined the phrase “the medium is the message”...
 
Laskey Hart guest hosts CGE Radio this week with a summary...
 
The non-profit sector is operating within an era of increased accountability,...
 
Earlier this year the Ontario Ministry of Health announced that it...
 
Gladwell made Anders Ericsson famous—or, at least, his research work. A...
 
Canada’s top soldier said the Armed Forces continue to be locked...
 
BC Hydro said it is pushing through with it $9-billion, hydroelectric megaproject on...
 
Yesterday, Federal and Atlantic Provincial Ministers of the environment met to...
 
After writing the book “Megatrends: The Impact of Infrastructure on Ontario’s...
 
Governments around the world are seeking to tap technologies such as...
 
Signaling a realignment of Canada’s involvement with NATO, Prime Minister Justin...
 
As an executive you’ve probably experienced frustration when you ask for...
 
The Public Health Agency of Canada has long been aware of...
 
Dispersed organisations can be very powerful structures. There is ample evidence...
 
By Nicole Verkindt Despite growing up in a small town in...
 
Canada, today, became a full supporter of the United Nations Declaration...
 
A research project at that explores the design and feasibility of...
 
A landmark Supreme Court ruling has paved the way for some...
 
Years and years ago when I was unemployed, being able to...
 
Local job figures blew away expectations as records indicated that 41,000...
 
At first glance, the title may seem counterintuitive. By their very...
 
The Liberal government’s first federal budget laid out $11.9 billion over...
 
The Liberal government is expected to announce on Tuesday a new...
 
A leader’s credibility begins with personal success. It ends with helping...
 
Monday was the final day for Canadians to donate money to...
 
Huawei is bringing back to Canada and expanding its information communication...
 
Many women entrepreneurs in Canada struggle to access capital, technology, networks...
 
Written by Tim Wacker Almost a quarter century ago, when most...
 
Written by  Brady G. Wilson You may not realize it, but...
 
Continuous, life-long learning is the future of the public service. Building...
 
Mega-ConnEX is an annual speed networking event organized by the Health...
 
Canada’s contribution to development and humanitarian assistance is recognized around the...
 
Deliberations and negotiations in the United Nations are intensifying on what...
 
The UN consultations on the post-2015 development framework are focused on...
 
At the turn of the millennium, the international community made a...
 
Budgets are statements of political will and power. They are performance...
 
Decades of reform in developing countries point to four enduring dilemmas:...
 
The Lough Erne Accountability Report states that the G8 has played...
 
While there can be no singular solution for good governance, building...
 
it’s not the most original saying...
 
With the demographics of an aging workforce and a significant number...
 
Shakespeare’s “dark comedy,” Measure for Measure, deals with the issues of...
 
Reform is almost always about developing the capacity of government to...
 
CGE Vol.14 No.3 March 2008 Most observers expect more than 40...
 
CGE Vol.13 No.3 March 2007 Assembly of First Nations National Chief...
 
CGE Vol. 14 No.5 May 2008 Before dawn on Monday, 18...
 
CGE Vol. 14 No.4 April 2008 “What is a Foreign Service...
 
If you want to understand someone else’s position, walk a mile...
 
CGE Vol.14 No.3 March 2008 The Commonwealth Secretariat convened the first...
 
Natural Resources Canada is accustomed to nurturing, protecting and growing Canada’s...
 
In an increasingly competitive and complex global marketplace, both employers and...
 
What makes a perfect mentor? Is it age and seniority? Perhaps...
 
Often, decisions about how to recruit are made based on habit...
 
It has been said that if everything is important, nothing is...
 
The Mosaic Index, by Professor Richard Florida, which measures the percentage...
 
Government policy agendas have long been fraught with complexity....
 
Looking at all the engineering diagrams overlaid on maps on the...
 
Gérés efficacement, les conflits peuvent favoriser la résolution conjointe des problèmes,...
 
Popular wisdom holds that the public service will need to recruit...
 
CGE Vol.14 No.2 February 2008 As I mentioned previously, one cannot...
 
CGE Vol.13 No.9 November 2007 What is the toughest job in...
 
CGE Vol.13 No.9 November 2007 Growth can mean opportunity. Significant increases...
 
The transition from the private sector into an executive role in...
 
We can learn from archetypes. They can help us lead, if...
 
The subtitle of this report, prepared by the U.S. National Governors...
 
Improving leadership development is widely recognized by HR leaders and senior...
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
Don’t ignore that gut feeling.             Too often we clutch at...