How good is your political judgment?
A Book Review on Expert Political Judgment: How good is it? How can we know?
By Philip E. Tetlock
Lobbyists are by nature considered experts on politics. We are paid to offer opinions and assessments on a wide variety of political issues and events, so we hold our political judgment in high esteem and take considerable pride in the value it delivers to our clients. The expertise to deliver accurate and timely expert political opinion is a fundamental part of our skill set as lobbyists, and certainly one of the most challenging. Still, in spite of our best efforts, even our most considered and thoughtful predictions sometimes turn out to be wrong. This is both an embarrassing and frustrating experience. An understanding of how we develop our opinions and how to identify the errors in their development is something of considerable real value. And certainly, we all want to be more right more often, if only for the elevated gravitas.
Yet political judgment is usually evaluated relative to the conventional wisdom of the time, or the judger's personal bias, as opposed to any empirical measure. For very practical reasons, it is rare that experts are judged on how their political judgments perform over time - and whether, with the passage of time, their accuracy improves.
In Expert Political Judgment, psychologist Philip E. Tetlock, a professor at the University of California at Berkeley, goes back 20 years to examine political judgments that were predictive of events to come, then he rates how accurate these opinions turned out to be. In the process he reveals all manner of nuances about how political judgments are developed, delivered, perceived and finally judged.
In November of 2005, just as the recent Federal election was called, PAAC members were witness to some fairly impressive political forecasting at the Stephen Clarkson presentation to PAAC promoting his new book. At that event, Clarkson, former Mulroney Cabinet Minister Tom Hockin and Navigator's Robin Sears all predicted Stephen Harper's Conservatives would win a minority government in the January 2006 election. Since the conservatives were behind in the polls at the time and the campaign had yet to begin, it was a daring contrarian opinion that could have easily led to a loss of face in less than two months. Moreover, even in this short period, a number of unforeseen events could and did rapidly shift public opinion -- such as the RCMP announcement of an investigation at the federal Ministry of Finance and an outbreak of gun violence in Canada's largest city. Such is the volatile nature of political events, and the vulnerability of predictions.
With that in mind the exceedingly high number of false predictions Tetlock found in his research should come as no surprise. What appear to be seasoned and wise political judgments, when reviewed through the clarity of hindsight, often turn out to be laughably wrong. This won't be news to anyone and thankfully Tetlock doesn't dwell on this or present it as some revelatory insight. Instead, Tetlock, uses this fundamental finding to delve deeply into the very nature of political judgment and in the process he develops tools for analysing it retrospectively, such as "fuzzy-set adjustments" to award partial credit when predictions were mostly correct, and "difficulty adjustments" which award more points for taking on more challenging subjects.
The most helpful part of this book may be Tetlock's observation that what experts think is less predictive of their opinions' accuracy than the manner in which they think. Tetlock concludes that those with a broad base of knowledge -- "the foxes" -- tend to perform better over the long haul than "hedgehogs," whose backgrounds are specialized in one area and who work narrowly focused within it. Interestingly enough he also finds that there is an inverse relationship between what the media or the general public value in pundits and the best scientific indicators of good judgment. In short, the more obtuse and stubborn a media pundit is, the greater his or her chance of being both very popular and very wrong. Undoubtedly this is a relief to those of us who have long since given up being astonished at how Linda McQuaig or Ann Coulter continue being published, while exciting young and broad-minded thinkers like Andrew Potter toil in relative obscurity. It leads the reader to ponder the question of just how much time is wasted by the media giving value to arguments whose sole purpose would seem to be to provide a false dynamic to political discourse.
Expert Political Judgment stands out in the field as a work of exceptional acuity and value. I can't recommend this book enough. Every contentious lobbyist should put it on their professional development list for this year.
Highly Recommended.
By Philip E. Tetlock
Lobbyists are by nature considered experts on politics. We are paid to offer opinions and assessments on a wide variety of political issues and events, so we hold our political judgment in high esteem and take considerable pride in the value it delivers to our clients. The expertise to deliver accurate and timely expert political opinion is a fundamental part of our skill set as lobbyists, and certainly one of the most challenging. Still, in spite of our best efforts, even our most considered and thoughtful predictions sometimes turn out to be wrong. This is both an embarrassing and frustrating experience. An understanding of how we develop our opinions and how to identify the errors in their development is something of considerable real value. And certainly, we all want to be more right more often, if only for the elevated gravitas.
Yet political judgment is usually evaluated relative to the conventional wisdom of the time, or the judger's personal bias, as opposed to any empirical measure. For very practical reasons, it is rare that experts are judged on how their political judgments perform over time - and whether, with the passage of time, their accuracy improves.
In Expert Political Judgment, psychologist Philip E. Tetlock, a professor at the University of California at Berkeley, goes back 20 years to examine political judgments that were predictive of events to come, then he rates how accurate these opinions turned out to be. In the process he reveals all manner of nuances about how political judgments are developed, delivered, perceived and finally judged.
In November of 2005, just as the recent Federal election was called, PAAC members were witness to some fairly impressive political forecasting at the Stephen Clarkson presentation to PAAC promoting his new book. At that event, Clarkson, former Mulroney Cabinet Minister Tom Hockin and Navigator's Robin Sears all predicted Stephen Harper's Conservatives would win a minority government in the January 2006 election. Since the conservatives were behind in the polls at the time and the campaign had yet to begin, it was a daring contrarian opinion that could have easily led to a loss of face in less than two months. Moreover, even in this short period, a number of unforeseen events could and did rapidly shift public opinion -- such as the RCMP announcement of an investigation at the federal Ministry of Finance and an outbreak of gun violence in Canada's largest city. Such is the volatile nature of political events, and the vulnerability of predictions.
With that in mind the exceedingly high number of false predictions Tetlock found in his research should come as no surprise. What appear to be seasoned and wise political judgments, when reviewed through the clarity of hindsight, often turn out to be laughably wrong. This won't be news to anyone and thankfully Tetlock doesn't dwell on this or present it as some revelatory insight. Instead, Tetlock, uses this fundamental finding to delve deeply into the very nature of political judgment and in the process he develops tools for analysing it retrospectively, such as "fuzzy-set adjustments" to award partial credit when predictions were mostly correct, and "difficulty adjustments" which award more points for taking on more challenging subjects.
The most helpful part of this book may be Tetlock's observation that what experts think is less predictive of their opinions' accuracy than the manner in which they think. Tetlock concludes that those with a broad base of knowledge -- "the foxes" -- tend to perform better over the long haul than "hedgehogs," whose backgrounds are specialized in one area and who work narrowly focused within it. Interestingly enough he also finds that there is an inverse relationship between what the media or the general public value in pundits and the best scientific indicators of good judgment. In short, the more obtuse and stubborn a media pundit is, the greater his or her chance of being both very popular and very wrong. Undoubtedly this is a relief to those of us who have long since given up being astonished at how Linda McQuaig or Ann Coulter continue being published, while exciting young and broad-minded thinkers like Andrew Potter toil in relative obscurity. It leads the reader to ponder the question of just how much time is wasted by the media giving value to arguments whose sole purpose would seem to be to provide a false dynamic to political discourse.
Expert Political Judgment stands out in the field as a work of exceptional acuity and value. I can't recommend this book enough. Every contentious lobbyist should put it on their professional development list for this year.
Highly Recommended.
posted by Stewart at Monday, May 15, 2006
<< Home