Last September Netflix refused the Canadian Radio and Telecommunications Commission’s (CRTC) order to provide confidential subscriber information and in so doing called into question the Commission’s authority to regulate it under The Broadcasting Act.

Netflix’s decision was rooted in its concerns that any proprietary information provided to the CRTC might find its way into the public sphere and consequently the hands of the company’s competitors. This followed a similar refusal by Google – which owns YouTube – to provide confidential information to the Commission.

While these concerns are understandable, in other industries private service providers have partnered with governments to help in sector-wide planning. So the CRTC’s perspective is equally understandable, in what is ultimately a public battle for social licence to operate.

As cases like these illustrate, legitimacy in the eyes of the public is a core concern for both firms and regulators. Whether public or private, organizations that lose their social licence do so at their peril.

As concerns the regulatory environment, the CRTC’s experience with “disruptive” organizations may just be the tip of the iceberg. The emergence of big data – the collection of vast storehouses of personal and consumer information on Canadian citizens, both by government and private industry – is introducing new complexities to regulatory questions.

The choice of whether to use or pursue big or even open data strategies requires careful sector-specific consideration. Open data holds out the possibility of a future co-creation of services between government and citizens where citizens can lever government data to craft innovative service delivery options. But where public data is offered for private use, there is an omnipresent concern over socializing risk while privatizing gain.

Governments shoulder risk and take responsibility for collecting personal data and protecting the privacy and security of that information. Similarly, open government – the opening of the policy and decision-making process to citizens – could bolster government legitimacy in regulatory environments.

Yet, while efforts to make government more open, engaging and legitimate are important, they must be balanced against other public service values like efficiency, effectiveness, inclusivity and fairness. Further, despite greater openness and availability of data, the question of how best to spur innovation in any given industry is far from settled.

The recent Digital Governance Forum featured a panel on Big Data, Open Government and Regulation that explored many of the challenges facing Canadian regulators. Panelist and former CRTC chair Justice Konrad von Finckenstein lamented the absence in Canada of mechanisms to publish intended regulations online for public comment. In contrast, disruptive innovators like Netflix enjoy a visible and direct relationship with their consumers and thus have large constituencies likely to support their policy positions.

Von Finckenstein has stated elsewhere that the rise of “over-the-top” carriers like Netflix symbolizes the obsolescence of “traditional regulatory gatekeeping.” The CRTC, says von Finckenstein, has entered an era in which citizens are in control; and in this environment the old regulatory model is “seen as restrictive and clearly consumers don’t like it…it will have to change.”

Crowdsourcing may be used to improve regulatory frameworks and increase compliance. This was the case with the audit commission in the U.K. that used “an army of citizen auditors” to pour over the spending records of local councils published online. However, these are still emerging approaches and there remains substantial space to explore whether they improve compliance, create better outcomes, and facilitate more effective governance.

At the forum, Colin McKay, head of public policy and government relations at Google Canada, suggested that regulators must adjust to the level of risk with which consumers are themselves comfortable, instead of trying to regulate away every grey area. But as lawyer Christine Duhaime argues, regulatory clarity is a key issue for many: “Uncertainties caused by regulatory policy can push the risk profile of even the most exciting innovation like Bitcoin beyond what a venture capitalist can tolerate.”

At the same time, over-regulation threatens to snuff out early stage innovations before they have a chance to mature, a sentiment expressed by Bitcoin expert Andreas Antonopoulos in his testimony before the Standing Senate Committee on Banking, Trade and Commerce.

In sum, the balance between regulatory clarity and citizen-based legitimacy through public participation is one that will likely differ from sector to sector as individual regulators go about their work within their specific industries.