• Research Vision & Mission
• Conceptual Frameworks
  • Cognitive Work Analysis
  • Ecological Interface Design
  • Trust in Automation
  • Risk Management Framework
• Themes & Projects
  • 1 - Theoretical
  • 2 - Empirical
  • 3 - Generalizability
  • 4 - Design Process
  • 5 - Cross Disciplinary
  • 6 - Meta-Scientific
• Research Platforms
  • DURESS II
  • HEPHAISTOS
  • IMMERSIVE
• Project List

Trust in Automation

Automation is becoming a larger component of many workplaces as computers become increasingly sophisticated. New automated tools are found in fields as diverse as process control, information technology and the military. Yet as automation becomes more ubiquitous, it is increasingly important that we understand human-automation interactions, and design automation in such a way as to ensure proper use. Only through proper automation design can we achieve safe, healthy and efficient interaction between people and automation, and to inform design, a theoretical model is needed.  CEL has adopted Lee and See’s (2004) conceptual model, which attempts to correlate different perspectives on trust and synthesize common findings into a single model of human trust and reliance in automation.

Trust in automation, to a large extent, guides reliance on automation. “People tend to rely on automation they trust and tend to reject automation they do not” (Lee & See, 2004). Too much trust (overtrust) can lead to an operator relying on automation when it is inadvisable or unsafe to do so; too little trust (distrust) can lead to reduced productivity and efficiency, or even reduced safety, if an operator manually controls a system that would be better managed by automation. Therefore, when designing automation and automation interfaces, one goal of design should be appropriate trust – and thence reliance.

Appropriate Trust

The notion of “appropriate trust” has three dimensions:

• Calibration: Refers to how closely an operator’s trust in automation matches the capability of the automation. In Figure 1, the diagonal line represents good calibration. Areas above this line correspond to overtrust and below correspond to distrust.
• Resolution: Refers to an operator’s ability to identify changes to conditions or goals that can have an impact on the automation’s performance, and to change his or her level of trust accordingly. In the figure below, this is indicated by the mapping between the blue areas along the trust axis and the automation capacity axis.
• Specificity: Refers to the degree to which operator trust corresponds to a specific component or aspect of the automation. Specificity has both functional and temporal aspects. High functional specificity means an operator trusts specific subfunctions or modes whereas low functional specificity indicates that the operator trusts the entire system. With high temporal specificity, an operator’s trust is affected by moment-to-moment changes in the automation’s capability. Low temporal specificity means that an operator’s trust is governed by longer-term changes in the automation’s capability.

Appropriate trust, therefore, consists of good calibration, high resolution and high specificity.

Figure 1: The relationship between calibration, resolution and automation capability (from Lee & See, 2004)

A Conceptual Model of Trust and Reliance

Lee and See’s (2004) model of automation trust and reliance (Figure 2) is based on a broad review of trust literature across domains. This model describes an action/perception feedback loop wherein an operator’s beliefs form an “information base” that inform trust as an attitude. The attitude of trust affects the operator’s intentions – including whether or not the operator intends to use the automation. This intention can translate into a reliance action, where the automation is engaged. Closing this feedback loop is the information the operator receives about the automation.

Figure 2: A conceptual model of the dynamic process that governs trust and its effect on reliance (from Lee & See, 2004)

This conceptual model identifies three levels of attributional abstraction at which information must be presented to the operator in order to engender appropriate trust in automation. The three levels, which lie along a continuum, are:

• Purpose: Refers to the reason for which the automation was designed and, in turn, how closely its current use aligns with this planned functionality. Thus purpose describes why the automation was developed.
• Process: Refers to the method by which automation achieves its goals. The operator must determine if the automation algorithms are appropriate for the current situation and if they will achieve the operator’s goals. Thus process describes how the automation operates.
• Performance: Refers to the current and historical behaviour of an automated tool including goal achievement, reliability and predictability. This can affect the perceived competence of the automation to achieve particular goals. Thus performance describes what the automation does.

In addition to the information about the automation, the appropriateness of trust is also influenced by the context in which the automation is used. This model points out that there exist several individual, organizational, cultural and environmental factors that can affect each step of the feedback loop, including beliefs, attitudes, intentions and behaviour, as well as the performance of the automation.

This model of trust in automation comes from a review of trust literature compiled by Lee and See (2004). Noting a lack of any form of “integrative review” of studies of trust – especially of trust in automation – the authors investigated literature presenting “organizational, sociological, interpersonal, psychological, and neurological perspectives on trust” (p.51), exploring connections between different theories and models of trust.

The CEL is currently applying this model is in the investigation of appropriate trust and reliance on automation in fields ranging from Combat Identification Systems to relational databases.

References:

• Lee, J.D., & See, K.A. (2004). Trust in automation: designing for appropriate reliance. Human Factors, 46(1), 50-80.

• Lee, J.D. (2006). Human factors and ergonomics in automation design. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (pp. 1570-1596). Hoboken, NJ: Wiley.