Behavioural Insights (BI) is a multidisciplinary approach that uses principles and methodologies from the behavioural sciences such as behavioural economics, marketing, and psychology. It is attracting considerable attention from researchers and policy makers around the world as a valuable and promising tool to improve governments’ programs, service delivery processes and relevant client outcomes. In the Government of Canada, interest for this approach has also widened in recent years and an increasing number of departments and agencies (e.g., the Privy Council Office, Natural Resources Canada, Canada Revenue Agency) are now actively promoting its use in areas of relevant interest.
BI provides policy makers with effective tools to affect positive behavioural changes among individuals. Among them, the so-called nudges have been used in a variety of contexts. They are light-touch interventions that can help individuals to make positive behavioural changes without enforcing limitations on the choices that are available to them.
In late 2015, Employment and Social Development Canada (ESDC) put in place a small BI research team within its Strategic and Service Policy Branch. (The research team is situated in the ESDC Innovation Lab.) This multidisciplinary team, composed of highly skilled analysts coming from various areas of expertise and backgrounds (economics, public administration, psychology, political science), is now leading more than half a dozen projects in various social, labour, and learning policy areas using rigorous research designs.
Nudging Job Bank Users: A Behavioural Economics Experiment is the first BI project to be conducted by the team. Conducted between early 2015 and late 2016, it consists of a series of trials that sought to measure the impacts of introducing nudges into e-mails to Canadians to improve the uptake of the Job Match service. This article describes the context, the nudges developed, and the key findings from this first trial. The conclusion section also presents a few lessons learned.
Case Study: Increasing Job Match Uptake
Job Bank is the Government of Canada’s main source for jobs and labour market information, and Job Match is one of its key tools. Launched in late 2015, Job Match connects job seekers to available jobs that fit their skills, experiences, and knowledge.
To benefit from this service, users need to first complete and activate an online profile about their skills and experience. Evidence from internal administrative data showed, however, that many users created their user account, but never got through to completing it. We hypothesized that this can happen due to a number of reasons, including not having some of the information necessary to complete their user account, frustration with the amount of details requested, fatigue, and procrastination. The main behaviour that we thus wanted to target in this first trial was increasing the number of users returning to the Job Bank website and completing their user account by addressing some of these behavioural barriers.
We decided to use a simple method: developing a trial where we sent e-mails to 3,800 users who hadn’t completed their user accounts and using behavioural insights to nudge them in completing it. In doing so, we experimented with four ideas from behavioural science to see what would work best in the context of Job Match:
- Framing – Simple changes in the way a particular message is presented/worded can have a substantial influence on people’s behaviour.
- Social norms – When people perceive a particular behaviour to be common among others, they are more likely to engage in that behaviour themselves.
- Call-to-Action/Salience – People are heavily influenced by what catches their attention and is most noticeable. Focusing the attention of users on one clear action to be taken (e.g., presenting the desired action on a large button with a contrasting background color) can increase the likelihood that they will engage in this action.
- Sense of Commitment – People are more likely to follow through with a goal-directed behaviour if they perceive themselves to be committed to the broader goal.
This led us to design five different versions of the e-mail we sent to users with incomplete user accounts. The first e-mail was a standard program e-mail that was not explicitly informed by BI and this served as our ‘control condition.’ Each of the remaining four e-mails included a ‘nudge’ based on one of the four behavioural principles described above. For example, the nudge based on the social norms principle said, “Thousands of Canadians have already completed their Job Match profiles and are being matched to new job opportunities.” To build on users’ sense of commitment, another nudge said:
You are only one step away from being matched with an employer looking for your skills and experience:
- I want to find a new job
- I am using Job Bank
- I have a Job Seeker Account
- I have completed my Job Match profile
All our BI-informed e-mails were identical except for the inclusion of these nudges. To compare the impacts of these e-mails, we used a Randomized Controlled Trial (RCT). RCTs are considered to be the ‘gold standard’ in impact evaluation because they enable us to test the impact caused by the nudges we’re interested in, while controlling for other external variables. To do this, we randomly assigned participants into five groups (≈757 participants each), and each group received one of the five e-mails (i.e., one of the four e-mails with nudges or the standard e-mail without the nudge). We then tracked the number of users clicking on the link in the e-mail, as well as the number of users completing and activating their Job seeker profile with each of these e-mails.
We found that, on average, our four BI-informed e-mails increased the number of people clicking on the link by 77 percent, compared to the standard e-mail. Specifically, BI-informed e-mails generated an average of 184 clicks each, whereas the standard e-mail generated 104 clicks.1 Among those who clicked on the link, the BI-informed e-mails generated 43 profile activations on average, compared to 21 activations with the standard e-mail (a 106 percent improvement). The differences in impact between the four BI-informed e-mails were minor and were not always statistically significant, suggesting that the four behavioural principles worked equally well in this context.
Overall, this small experiment demonstrated that nudges do matter. It showed that the rigorous application of behavioural principles can be a simple, cost-effective way of increasing program take-up. We conducted a number of other trials to explore different touch points through which BI can be leveraged to increase the number of users benefitting from the Job Match service. These trials and the sharing of their key findings across the department have helped establish the usefulness of BI tools as an innovative way to improve our programs and the delivery of our services. This in turn has generated increased interest in the Department for its application in other areas.
Lessons learned
This first ESDC nudge trial implemented in the context of the Job Match service taught us a number of useful lessons regarding the use of behavioural principles in a public policy context. In particular, the success of this type of initiative requires the presence of a number of key conditions:
- Leadership from middle level managers as well as a buy-in from senior management.
- Continuous and close collaboration between the program experts and departmental colleagues with complementary expertise in disciplines such as research, web system and program policy.
- Access to high quality data and the promotion and use of rigorous research designs allowing reliable and robust measures of impacts in order to generate credible results.
- Research protocols must also be flexible enough to adapt to program and IT constraints inherent in any organisation.
Last but not least, departmental officials from all levels, particularly senior management, must view testing and experimentation as being all about trial and error and not about pointing out failure. Experiments such as these are important if we are to determine what works and what does not.
Mathieu Audet, Stéphane Gascon, Hasti Rahbar, Monica Soliman are all members of the Service Research Division, Strategic and Service Policy Branch, Employment and Social Development Canada. The views expressed here are their own, and do not represent government policy.
