Cognitive Biases in Decision Making William Siefert, M. S

Cognitive Biases in Decision Making

• William Siefert, M.S.

Acknowledgements

• Acknowledgements

• Work based on the research done by

• Dr Amos Tversky, PhD
• Dr Daniel Kahneman, PhD
• “Prospect Theory” Nobel Prize, 2002
• Dr Eric Smith, PhD
• Dr Paul Slovic, PhD

5 x 5 Risk “Cube”

Present Situation

• Risk matrices are recognized by industry as the best way to:

• consistently quantify risks, as part of a
• repeatable and quantifiable risk management process

• Risk matrices involve human:

• Numerical judgment
• Calibration – location, gradation
• Rounding, Censoring
• Data updating
• often approached with under confidence
• often distrusted by decision makers

Goal

• More accurate and repeatable Systems Engineering Decisions

• Confidence in correct assessment of probability and value
• Avoidance of specific mistakes
• Recommended actions

Heuristics and Biases

• Daniel Kahneman won the Nobel Prize in Economics in 2002 "for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty.“

Anchoring

• First impression dominates all further thought

• 1-100 wheel of fortune spun

• Number of African nations in the United Nations?

• Small number, like 12, the subjects underestimated
• Large number, like 92, the subjects overestimated

• Obviating expert opinion

• The analyst holds a circular belief that expert opinion or review is not necessary because no evidence for the need of expert opinion is present.

Heuristics and Biases

• Presence of cognitive biases

• – even in extensive and vetted analyses – can never be ruled out.

• Innate human biases, and exterior circumstances, such as the framing or context of a question, can compromise estimates, judgments and decisions.

• It is important to note that subjects often maintain a strong sense that they are acting rationally while exhibiting biases.

Likelihood

• Frequency of occurrence is objective, discrete

• Probability is continuous, fiction

• "Humans judge probabilities poorly" [Cosmides and Tooby, 1996]

• Likelihood is a subjective judgment

• (unless mathematical)
• 'Exposure' by project manager
• timeless

Case Study

• Industry risk matrix data

• 1412 original and current risk points
• Time of first entry known
• Time of last update known
• Cost, Schedule and Technical known
• Subject matter not known

• Biases revealed

• Likelihood and consequence judgment

Magnitude vs. Reliability [Griffin and Tversky, 1992]

• Magnitude perceived more valid

• Data with outstanding magnitudes but poor reliability are likely to be chosen and used

• Observation: risk matrices are magnitude driven, without regard to reliability

1. Estimation in a Pre-Define Scale Bias

• Scale magnitude effects judgment [Schwarz, 1990]

• Two questions, random 50% of subjects:

• Please estimate the average number of hours you watch television per week:

• ____ ____ __X_ ____ ____ ____

• 1-4 5-8 9-12 13-16 17-20 More

• Please estimate the average number of hours you watch television per week:

• ____ ____ __X_ ____ ____ ____

• 1-2 3-4 5-6 7-8 9-10 More

Severity Amplifiers

• Lack of control

• Lack of choice

• Lack of trust

• Lack of warning

• Lack of understanding

• Manmade

• Newness

• Dreadfulness

• Personalization

• Recallability

• Imminency

Situation assessment

• 5 x 5 Risk Matrices seek to increase risk estimation consistency

• Hypothesis: Cognitive Bias information can help improve the validity and sensitivity of risk matrix analysis and other Systems Engineering analysis

Prospect Theory

• Decision-making described with subjective assessment of:

• Probabilities
• Values
• and combinations in gambles

• Prospect Theory breaks subjective decision making into:

• preliminary ‘screening’ stage,
• probabilities and values are subjectively assessed
• secondary ‘evaluation’ stage
• combines the subjective probabilities and utilities

Humans judge probabilities poorly*

Gains and losses are not equal*

Subjective Utility

• Values considered from reference point established by the subject’s wealth and perspective

• Framing

• Gains and losses are

• subjectively valued

• 1-to-2 ratio.

Implication of Prospect Theory for the Risk Matrix

ANALYSES AND OBSERVATIONS OF INITIAL DATA

• Impediments for the appearance of cognitive biases in the industry data:

• Industry data are granular while the predictions of Prospect Theory are for continuous data
• Qualitative descriptions of 5 ranges of likelihood and consequence
• non-linear influence in the placement of risk datum points

• Nevertheless, the evidence of cognitive biases emerges from the data

3. Probability Centering Bias

• Likelihoods are pushed toward

• L = 3

• Symmetric to a first order

Cognitive Biases in Action

• Engineers:

• Schedule consequences effect careers
• Technical consequences effect job performance reviews
• Cost consequences are remote and associated with management

• Higher cognizance of Biases will be valuable at the engineering level

CONCLUSION

• First time that the effects of cognitive biases have been documented within the risk matrix

• Clear evidence that probability and value translations, as likelihood and consequence judgments, are present in industry risk matrix data

• Steps

• 1) the translations were predicted by prospect theory, 2) historical data confirmed predictions

• Risk matrices are not objective number grids

• Subjective, albeit useful, means to verify that risk items have received risk-mitigating attention.

Suggestions for Cognitive Biases improvement

• Long-term, institutional rationality

• Team approach

• Iterations

• Public review

• Expert review

• Biases and errors awareness

• Requires cultural changes

References

• L. Cosmides, and J. Tooby, Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty, Cognition 58 (1996), 1-73.

• D. Kahneman, and A. Tversky, Prospect theory: An analysis of decision under risk, Econometrica 46(2) (1979), 171-185.

• Nobel, "The Bank of Sweden Prize in Economic Sciences in memory of Alfred Nobel 2002," 2002. Retrieved March, 2006 from Nobel Foundation: http://nobelprize.org/economics/laureates/2002/index.html.

• N. Schwarz, Assessing frequency reports of mundane behaviors: Contributions of cognitive psychology to questionaire construction, Review of Personality and Social Psychology 11 (1990), 98-119.

• A. Tversky, and D. Kahneman, Advances in prospect theory: Cumulative representation of uncertainty, Journal of Risk and Uncertainty 5 (1992), 297-323.

Comments !

• Comments !

• Questions ?