If your Deputy Minister asked you to identify a dozen communities that had laid off factory workers in the last quarter – how would find them? What if she asked you to track which regions have benefitted economically from the award of a large defense contract in a neighbouring city? What if she asked you to track which countries visitors came from at your trade show booth? Or which townships had an abnormally high number of potholes? Or how many retailers would be affected by the moving of a community centre?

The very thought of drilling down to the provincial, regional, municipal, or even neighbourhood level has struck fear in the hearts of many an experienced team in the past. While conventional intelligence acquired by conducting surveys or through consulting or market intelligence firms can provide valuable insights, they can be expensive, time consuming, and difficult to compare across geography and time. StatsCan offers a high standard in terms of defensibility, but these data can be aggregated in categories that may not be relevant for the analysis at hand. For example, industry or occupation groupings can be challenging to work with, and certain data are only available at a federal or provincial level. Furthermore, the most recent updates, in some instances, can be several years old.

Non-conventional intelligence offers an alternative for more granular, comparable, and timely insights at prices which can be fractions of comparable sources.

From social media, the Internet of Things, the sharing economy, and satellite imagery, this brave new world has opened up an entire blue ocean of new opportunities for policy teams in almost every department. In fact, for perhaps the first time in many years, the public service is faced not with an issue of a lack of intelligence, but rather an issue of having the capacity to make sense of all of it.

An Old Story with a New Twist

For much of the business world, this has been a well-known narrative.

Hedge funds can now use satellite imagery to count cars in Walmart parking lots to anticipate revenue surprises. Marketers mine social media feeds to provide targeted advertising on everything from clothing recommendations to restaurants in your neighbourhood. Weight-loss clinics can theoretically infer obesity rates down to a neighbourhood level through partnerships with automotive companies – a car knows how much you weigh because that information is collected to calibrate the airbag.

Given the almost endless ways that data can be leveraged, and given that the price of data will continue to decrease, the question is not so much if public servants will one day be exploring the use of non-conventional intelligence (in fact, many already are!), but rather how we can do so in a manner that is responsible, defensible, and value-creating.

Privacy

The most important aspect in your journey towards the non-conventional use of intelligence is privacy.

Engage the privacy commissioners early in the process. They will be an invaluable resource in terms of establishing and documenting your process, identifying best practices, recommended training and flagging risks that only trained experts can see. There are numerous resources online to help you start your journey, and you may already have access to internal experts within your department, such as those handling Freedom of Information requests.

If you ever continue to doubt the need for the public servant to be vigilant about privacy, re-read a few articles about Edward Snowden and Cambridge Analytica just to remind yourself about the reputational risks that governments can face in handling potentially personally identifiable information.

In fact, just this October, Statistics Canada was in the media regarding their proposed collection of data from financial institutions. This will not likely be the last time that a Canadian government’s use of personally identifiable information will make headlines.

Use Multiple Sources

Because your methodology could be unfamiliar to your audience, defensibility is even more, not less, of an imperative than in using conventional intelligence. Suppose you use Twitter to shed insight on which users around the world have demonstrated interest in immigrating to your region. Is this a representative sample given the demographic of Twitter? Using both non-conventional and conventional sources, such as LinkedIn and census data, you can build an even richer narrative and more compelling evidence-based proposal.

In recent years, the trend towards open government data sets has certainly been a cost-effective way to gather intelligence. However, supplementing these with non-conventional intelligence can provide a level of unprecedented granularity and timeliness. For example, fleets of freight trucks are often outfitted with “accelerometers” which can identify when and where a truck experiences a sudden movement. Aggregated over thousands of trucks, an infrastructure analyst could theoretically get an accurate and timely view of frequency of potholes at an intersection-by-intersection level.

Value for Money

While the cost of data can and has gone down in cost, it is by no means inexpensive. Marketers, representing both online and bricks-and-mortar retailers, can pay in the millions of dollars per year for a single subscription to certain data sets. These retailers can justify these costs because of the almost immediately measurable returns on investment.

Likewise, a policymaker needs to have a clearly articulated business case. Will the project allow you to measure more precise progress made by a particular initiative (and therefore make a better case for subsequent top ups)? Will it help you to craft a narrative that there is a gap to be filled by a program or that an imminent crisis needs to be proactively addressed? Will it give senior decision makers and funding agencies more comfort in a proposal that you are seeking support for? Will it improve your pitch when competing for investments against other governments? Will it help to validate self-reported metrics made by a transfer payment or sponsorship recipient? Will it allow you to support local startups?

A clear understanding of the value proposition will help you both in your request for funding and in keeping focused once the project has initiated.

The Benefits of Early Adoption

Instead of waiting for the perfect business case to present itself, consider that this is a unique time for governments to be proactive in exploring this opportunity.

First, some of the data providers will have little or no experience in working with government. From my experience, this can be both a challenge and an opportunity. On one hand, the government client may need some degree of sophistication to guide the vendor towards potential-use cases. On the other hand, the vendor may have increased flexibility in terms of pricing as they may be eager to showcase a real-world case study to help them unlock a new, and potentially lucrative, customer segment.

Secondly, new and exciting startups are coming out of incubators and accelerators at a pace that can be hard to keep up with. Furthermore, traditional firms, such as payment processors, are experiencing an increased interest in anonymizing and monetizing their data. Early government adopters will benefit from having a  head start in understanding the wide array of approaches and staying ahead of their peers.

Consider the use case of estimating unemployment in a region.

By determining that a certain number of phones are traveling to and remaining at work locations on a daily basis, you may be able to create a proxy for regional unemployment trends. However, there are several ways to track the travel patterns of smartphones in aggregate, such as through their GPS signals or by monitoring cellphone tower access. Each data provider will have different sample sizes, accuracy, cost, and granularity thresholds in order to provide anonymity. Having the breadth of options available will help you to find the best option to meet your needs and budget.

Take  Baby Steps

Start small. Many government departments have a “show me first” culture, and senior decision makers may want to touch and feel a proof-of-concept or minimum viable product before committing further resources.

That said, my experience has been that the move to non-conventional intelligence may ultimately not be for everyone, and conventional sources will likely always have a place in the policy maker’s toolkit. However, with the explosion of data and the exponential increase in machine power to process it, there will be an increased emphasis on getting intelligence better, faster, and cheaper than other departments, other levels of government, or even other governments across the world that may be competing with you for resources.

The key point is to start exploring.