Best Practice
May 7, 2012

Be wary the expert’s advice

 

Wrong

David H. Freedman

Little Brown, 295 pages, $31.99

 

Government runs on expertise. Government executives seek the best knowledge and information, from experts within and without their departments, before making decisions.

But what if some – or a lot – of that expertise was faulty? What if the recommendations of experts came with a margin of error, and not a low one like most opinion surveys, but a sky-high chance of being wrong?

Science and business journalist David Freedman thinks we should be alert to that possibility. In his new book, Wrong: Why Experts Keep Failing Us – And How To Know When Not To Trust Them, he begins with the work of a researcher who has found that the findings in the most influential medical journals prove to be wrong two out of three times. And the author extends that to experts in general: “The fact is, expert wisdom usually turns out to be at best highly contested and ephemeral and at worst flat-out wrong.”

He goes further, suggesting while the mistakes usually aren’t deliberate, they aren’t wholly innocent either – indeed, we are all complicit. He notes that often people will joke that meteorologists are the only people who get paid to be wrong. “I would argue that in that sense most of our experts are paid to be wrong and are probably wrong a much higher percentage of the time than are meteorologists,” he says. “Expert pronouncements are pushed toward wrongness so strongly that in the end it’s harder, I think, to explain why they’re sometimes right. But that doesn’t mean we’re hopelessly mired in the swamp of bad advice. With a decent compass, we can find our way out.”

Admittedly, his book is a journalist’s report, with broad brush strokes from a pastiche of examples and, yes, snippets of research. But then so was Malcolm Gladwell’s The Tipping Point, whose style this book resembles, and likewise deserves some attention.

A lot of Freedman’s examples are focused on expertise transmitted through the media by scientific studies and big name consultants, often in a flurry of newsbytes.

That might be called “mass expertise,” but some of those people, and their work, makes its way into the funnel for government decisions; health executives may have depended on some of that medical research which was found to be wrong 66.7 percent of the time. Some of that work was even funded by government – the scramble for funding, he shows, often exacerbates the likelihood of going wrong. And if you have ever asked someone briefing you to make their points simpler, you may have contributed to the epidemic of wrong he is cataloguing.

Indeed, it starts with simplicity. We love advice that seems simple. He notes that when it comes to advice on improving our personal productivity, we’ll shun advice offering “the 138 things you might have to do to have some chance of partly achieving your goal, depending on which of these following 29 conditions best describe you and your situations.” Too many things to contemplate. “Instead, we look for the 12 steps, the seven habits, and, of course, the secret – that one-step recipe that enables any person to achieve any type of success under any conditions.”

Here are some other characteristics he notes we crave in expert advice:

 

  • Clear-cut: We prefer to be told the right answer without ifs, ands or buts. “Qualifications that require matching different answers to different conditions, or that may render the advice entirely inappropriate to some situations, are an unwelcome complication and make the advice seem less fundamental,” he writes. We can thoughtlessly assume that an expert who hedges his bets really isn’t on top of the matter – if he was, he would be more clearcut.
  • Doubt-free: We expect experts to transmit their advice with full confidence. So doubts aren’t usually shared, important as they may be.
  • Universal: This may be the age of customization, but we prefer one-size-fits-all-advice. It is easier to apply, and, ironically, has the ring of truth.
  • Upbeat: We don’t want to hear something can’t be fixed or if there are solutions they are elusive, murky long shots or difficult compromises or unpleasant to implement. We prefer positive advice.
  • Actionable: Expert findings must go beyond explaining things to helping us improve the situation. As Woody Allen said, “you want to feel you can control things to some degree, because if you can’t, life is scarier.”
  • Palatable: We are loaded with biases and beliefs, and are likely to reject advice that challenges our ingrained beliefs, no matter how well grounded the advice may be.

 

He adds those all up into something he calls the certainty principle: “We’re heavily biased to advice that is simple, clear-cut, actionable, universal and palatable. If an expert can explain how any of us is sure to make things better via a few simple, pleasant steps, then plenty of people are going to listen.”

That applies most heavily, of course, to diet crazes or investment tips. But it also fits the various forms of advice government executives receive from experts – and the advice experts within government sense they must give to be heard. When the advice is simple, doubt-free, universal, upbeat, actionable and palatable – when it hews to the certainty principle – we are more likely to respond positively than when it doesn’t.

The book also sifts through a variety of reasons why medical and scientific studies go bad. Of importance to government executives is the fact that the fight for funding by researchers can lead them to twist their study – and even at times their data – to get the all-important grant upon which their careers depend. Inconvenient data gets tossed out. A variety of analytical techniques might be tried until one, no matter how exotic, declares the study valid and spits out some appealing results.

Since journals tend to only publish positive findings – studies whose results back the study’s hypothesis – those that convey negative results are shoved into an office drawer, even though they may be important for our broader understanding of an issue. Too many of those negative findings, of course, and the researcher can be languishing in no-funding land, so the motivation to fudge the research grows.

Freedman suggests you be extra wary when advice coming to you displays the following characteristics:

 

  • It’s simplistic, universal, and definitive: experts often operate at the very boundary of the unanalyzable. They aren’t tackling the easy stuff. Their advice, if decent, he says, “will be complex, it will come with many qualifications, and it will be highly dependent on conditions. Because of the ifs, ands, or buts, it will be difficult to act on. Because our beliefs tend to be simplistic and optimistic, it will probably be incompatible with them. In other words, good expert advice will be at odds with every aspect of the sort of advice that draws us to it.”
  • It’s supported by only a single study, or many small or less careful ones, or animal studies: all those are common flaws with medical and scientific studies that are later proved wrong. Humans are not like rats or other animals, he says, dismissing those studies.
  • It’s groundbreaking: most expert insights that seem novel and surprising are based on a small number of less rigorous studies, because usually the rigorous studies only come after the smaller ones pave the way.
  • It’s pushed by people or organizations that stand to benefit from its acceptance. That’s something government executives know. Indeed, he declares that “government-sponsored research

About this author

0 comments

There are no comments for this post yet.

Be the first to comment. Click here.

Best Practice
 
We live in the age of paradox.  Never before in human...
 
Canada is a world leader in clean electricity, with two-thirds of...
 
The discussion around where technology is going in government tends to...
 
Leadership, service delivery, change management, performance measurement, big data, transparency, digital...
 
What are the key initiatives to support government innovation and change?...
 
Could robot doctors be on the horizon? A study published February...
 
According to a 2018 study led by Dr. Michael McGuire, Senior...
 
My observation is people often take the view, “no one else...
 
One of the benefits of living in a federation is that...
 
The role of internal audit in the public sector has changed...
 
We are excited to share with you the October/November 2018 edition...
 
As governments have sought to go digital, one of the most...
 
With the launch of the Canada Infrastructure Bank and US Congressional interest in Trump’s...
 
Why is strategic planning so dreaded? How often do we actually...
 
Most articles you’ll read within the pages of Canadian Government Executive...
 
A communications revolution has taken place but you would not know...
 
Headlines about the world of work are often dominated by the...
 
Earlier this year the Canadian government took a crucial step by...
 
The use of digital services in the day-to-day lives of most...
 
Since launching in 2014, Sweden’s radically ‘feminist’ foreign policy has gained international notoriety. While...
 
The International Civil Service Effectiveness (InCiSE) Index project, a collaboration between...
 
Today, the challenges facing governments are increasingly shifting away from traditional,...
 
Three years before his death in 2011, Jack Layton released a...
 
Canadian Government Executive Media, (CGE) publisher of Canadian Government Executive magazine...
 
Management in the Government of Canada is continuously searching for new...
 
With businesses and government agencies increasingly operating in a highly information-based...
 
Blockchain. There’s probably no other technology trend in recent years that...
 
Over the last two years, an ever-growing number of organizations around...
 
Canadian Government Executive Media (CGE) is pleased to announce its first batch...
 
In today’s workplace, individuals increasingly face dynamic and difficult challenges that...
 
Canadian Government Executive Media (CGE) is pleased to welcome Microsoft as its Platinum Sponsor for...
 
In a rather unusual, quiet manner this past summer, a new...
 
Canadian government agencies and departments are modernizing the way they do...
 
The health care system in Canada, known as Medicare, is publicly...
 
We are happy to share with you the May/June issue of...
 
Facebook Pages can be an essential tool for businesses and charities,...
 
In this special episode of CGE Radio, your host John Jones...
 
The North Atlantic Treaty Organization, with its 28 independent member countries...
 
Cost overruns have become institutionalized in the federal government, according to...
 
Last year, procurement Minister Jody Foote was prompted by the swirling...
 
The rapid advances in digital technology, demographic and societal shift even...
 
Results Washington is Washington state government’s data-driven performance management and continuous...
 
Rules and accountability are helpful in developing and standardizing processes but...
 
Ontario’s provincial government needs to act fast in building a coordinated...
 
It’s a common notion that young workers born in the mid-1990s...
 
A study conducted by Robert Half, the world’s largest specialized staffing...
 
In order to find out which strategies and priorities CFOs are...
 
A small but rapidly growing number of businesses have started hiring...
 
CFOs around the world are changing – but an EY survey...
 
There’s a fine line between confidence and naivety – a line...
 
Around the world, more and more workers are seeking employment. At...
 
As populations grow and age, the demand for services increases. As...
 
By Nicole Verkindt Despite growing up in a small town in...
 
The agency responsible for safeguarding the Pentagon and several other buildings...
 
As much a 20 per cent of grade seven students in...
 
By Jason Zhang  The best way to cost effectively manage the retention...
 
He is often referred to as the federal government’s first “chief...
 
Are you absolutely clear what the government wants to achieve? Are...
 
Copyright owners have struggled to protect their intellectual property since the...
 
Here’s some bad news for organizations contemplating projects that have something...
 
Nearly half of Canadian organizations are falling behind on implementation of...
 
The idea of converged infrastructure and converged systems in the datacentre...
 
Hackers are zeroing in on users of SSL/TLS encryption and no...
 
There is no shortage of examples of businesses that effectively used...
 
Microsoft, on Tuesday, ended support for older versions of Internet Explorer...
 
Written by Tim Wacker Almost a quarter century ago, when most...
 
Written By Chris Brown To deliver results that senior executives value,...
 
Written By Jason McNaught Contrary to what you may have heard,...
 
Written by  Benjamin Selinger and Gabriel Flores A real challenge for...
 
Written by  Patrice Dutil It’s hard to believe we are still...
 
Written by  Patrice Dutil Craig Dowden, President and Founder of Craig...
 
Written by  Brady G. Wilson You may not realize it, but...
 
Written by  Roxanne Descôteaux When I first became a manager over...
 
The good news is that the Government of Canada recognizes the...
 
You sometimes wonder why government can’t be better at just getting...
 
Hill+Knowlton Canada announced last week that it has received the prestigious...
 
I’ve been giving 110 percent for as long as I can...
 
The National Voluntary Standard for psychological health and safety in the...
 
The government of Alberta has banded together with British Columbia and...
 
When we put out the call to cities across Canada for...
 
BC Hydro was experiencing ongoing challenges with IT assets decommissioned as...
 
City councils are important. They not only represent citizens at the...
 
Business continuity management has evolved into a specialized discipline, but you...
 
Saskatchewan’s Ministry of Health is leading a new approach to strategic...
 
Whether you have read John Kotter’s classic Leading Change or not,...
 
Energy, not time, is the fundamental currency of high performance. But...
 
Across the country, public sector leaders are challenged with the daunting...
 
When Neil Armstrong stepped onto the moon, it was government that...
 
Ego can do us in. It can lead to arrogance, blindness,...
 
and the more layers between a given employee and the relevant...
 
CGE Vol.14 No.2 February 2008 Management speaks of change in positive...
 
Innovation, leadership and a commitment to building a knowledge advantage are...
 
One of the most paradoxical Biblical parables, found the Gospel of...
 
CGE Vol.14 No.1 January 2008 “We’ve entered into an era in...
 
L’une des paraboles bibliques les plus paradoxales rapportées par Luc est...
 
Chris Baker is Deputy Minister, Policy and Priorities, and Deputy Minister,...
 
CGE Vol.14 No.2 February 2008 Often overworked and understaffed, the procurement...
 
Federal, provincial and municipal governments provide billions of dollars in grants...
 
During the American presidential election campaign of 1976, Jimmy Carter, then...
 
The Ontario Public Service celebrated ten years of hosting Showcase Ontario,...
 
Tell the truth. Deep down, do you sometimes – or often...
 
What if our general approach to brainstorming was all wrong? That...
 
Nearly two decades after I read Lincoln On Leadership, the messages...
 
True leaders are proactive. They get things done. They accomplish that...
 
Management and leadership are complicated. So it would be lovely if...
 
Government revolves around power. Political parties seek power. They implement their...
 
Managers generally walk around with a what’s-going-wrong mentality. They are fixated...
 
Government runs on expertise. Government executives seek the best knowledge and...
 
Let’s resume last month’s discussion on effective change initiatives with some...
 
When Ontario Premier Dalton McGuinty set as a prime goal of...
 
By now it’s commonplace knowledge that emotional intelligence is vital to...
 
Good leaders can make bad decisions. President Kennedy blundered over the...
 
The TRIC model of leadership is enshrined in the Ontario Public...
 
A respectful workplace is a critical ingredient of successful public service...
 
For the third year in a row, the Ontario Public Service...
 
Justice on Target (JOT) takes a unique and bold approach to...
 
Perched on top a 60 meter column in a bustling square...
 
For the second year in a row, the Ontario Public Service...
 
OTTAWA – Management consultants at Accenture, a respected longtime observer of...
 
La plupart des leaders reconnaissent d’emblée que leur efficacité dépend de...
 
Most leaders will readily agree that earning and keeping the trust...
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
 
Some title Some author
Some excerpt
We live in the age of paradox.  Never before in human...