Assessing overconfidence among national security officials

(Press-News.org) National security officials are "overwhelmingly overconfident," which hinders their ability to accurately assess uncertainty, according to new research by a Dartmouth government professor. When they thought statements had a 90% chance of being true, the statements were only true about 60% of the time, according to the study. 

The findings are published in the Texas National Security Review.

About 1,900 national security officials from more than 40 NATO allies and partners were surveyed on the uncertainty of current and future states of the world, and delivered a total of 60,000 assessments. The officials were enrolled at the U.S. National War College, the Canadian Forces College, the NATO Defense College, and the Norwegian Defense Intelligence School.

In the U.S., Canada, and Europe, once military officers achieve the rank of colonel, they must obtain a master's degree at a war college or other military institution as part of their professional military education. Participating institutions also contained a large fraction of civilian national security officials who work in foreign affairs ministries and intelligence agencies, among other areas. The study thus spanned both an unusually large, and representative, group of high-ranking national security officials.

The survey contained questions on international military, political, and economic affairs and  asked respondents to estimate the chances that a statement was true such as: "In your opinion, what are the chances that NATO's members spend more money on defense than the rest of the world combined?"

Other questions involved making predictions such as: "In your opinion, what are the chances that Ukraine and Russia will officially declare a ceasefire by a certain date?"

The results showed that national security officials are overconfident about the current and future state of the world—a cognitive bias that was consistent across all respondents, civilian and military professionals, men and women, and U.S. and non-U.S. citizens. They share these biases with the general public.

"National security officials are like many of us, in the sense that we tend to think we know more than we really do. This means that national security officials, like members of the general public, are consistently overconfident," says study author Jeffrey Friedman, an associate professor of government and member of the Davidson Institute for Global Security at Dartmouth. He says that overconfidence among national security officials is similar to biases he has found when conducting similar surveys with undergraduates, masters students, and financial professionals.

"However, the study also showed that it is possible to mitigate that bias substantially with just two minutes of training," says Friedman. His research found that briefly showing national security officials data on patterns of overconfidence led study participants to make judgments that substantially reduced overconfidence–and promoted accuracy.

The study also found that national security officials have a bias towards false positives— a tendency to think false statements are true.

This was demonstrated by flipping the wording of survey questions. In a subset of surveys, half of the participants were asked: "Has ISIS killed more civilians over the last decade than Boko Horam?" while the other half were asked: "Has Boko Haram killed more civilians than ISIS?" The answers that national security officials gave to these two questions consistently added up to more than 100%.

Friedman says that this finding indicates that national security officials appear to have a "tendency towards confirming rather than refuting possibilities they are asked to consider," which could be especially problematic for national security officials given that there could potentially be multiple outcomes to consider in military scenarios, rather than just one.

The study did suggest one potential remedy for the overconfidence–reminding national security officials of the perils of being too sure of their convictions.

Before a random subset of the national security officials were given the survey, they received information about other cohorts' overconfidence and biases. Through this two-minute training and informed approach, those participants were significantly better at assessing uncertainty.

Friedman says the four military institutions who took part in the study, deserve a ton of credit for their participation. The work had a ripple effect: the first cohort was from the National War College, which was so pleased with the session that they invited Friedman back, and then other military institutions came on board following recommendations by past participants. "It was very rewarding to see how receptive the national security officials were to the insights and training, as the training was then built into the core curricula at the institutions," says Friedman.

"Any organization that cares about improving people's ability to assess uncertainty in a more accurate manner can implement this training," says Friedman. "The material is posted online and can be developed and integrated into curricula anywhere, whether it be by military officials, diplomats, intelligence officers, business leaders, or others."

"By harnessing decision science tools, we can make people's judgements better," says Friedman.

Friedman is available for comment at: Jeffrey.A.Friedman@dartmouth.edu.

###

END