The purpose of this two-phase project was to develop a novel approach to the design of human-computer interfaces for the petrochemical industry. Inefficient handling of abnormal situations in this industry has led to increased operating costs, reduced occupational safety, and adverse environmental impact. One approach to addressing these problems is by designing model-based interfaces that present operators with the information they need to deal with abnormal situations effectively. However, there are two types of models that can be used to identify the information that should be presented in human-computer interfaces. Task-based models are like directions for navigation because they identify the actions that human operators should take for particular situations; system-based models are more like maps for navigation because they emphasize the overall structure of the plant, independent of any particular situation. Task models are efficient because they identify the information and prioritize it for pre-defined classes of situations, whereas system models are more robust because they identify the functional relationships that are potentially relevant for all situations.

In the first phase of the project [TIME 94], we explored the benefits of integrating these two types of models. The integration was first attempted for the DURESS II process simulation. Plan-Goal Graphs (PGG), a task-based method, were constructed and information requirements generated. These requirements were compared with those generated by the system-based abstraction hierarchy (AH) analysis of DURESS II. The comparison revealed that the analyses were largely complementary in terms of the information generated. Subsequently, a similar comparison was undertaken for a representative petrochemical process. An AH, Hierarchical Task Analysis (HTA), and Control Task Analysis (CTA) were performed for a key stage in the ethylene refining process. A comparison of the analysis results again supported the conclusion that system-based analyses (such as the AH) and task based analyses (such as PGG, HTA, and CTA) produce complementary information requirements to be used in the design of human-computer interfaces.

The suite of information requirements generated by the AH, HTA, and CTA were consolidated, discrepancies resolved, and redundancies eliminated. The resulting list of information requirements served as the knowledge base for the design of novel human-computer interfaces in the second phase of the project (CRD 00).

A contract was signed with Lawrence Erlbaum & Associates to write a textbook/monograph on the framework for cognitive work analysis (CWA) developed by Rasmussen. The book was written in a pedagogic fashion with many examples, the goal being to make these ideas more accessible to a much wider audience than they have been in the past. Five phases of cognitive work analysis were described: work domain representation, control task descriptions, mental strategies, social organization, and operator competencies. Each phase was thoroughly illustrated with an application example that was followed through in the book, DURESS II from the domain of process control. The thesis of the book was that, by adopting CWA, systems designers can create information technology that leads to safe, productive, and healthy work environments.

In this research, we examines the question of how people allocate their attention when controlling a complex system while watching an integrated view of the system. To achieve this, we analyzed fixations on particular objects as an indicator of attention. The subjects had to control DURESS II wearing an eyetracker with one of two interfaces, a traditional Physical Interface (P) and another fitting the requirements of Ecological Interface Design (P+F). Two other independent groups were participating in this experiment, some novices knowing nothing about the system and some experts from a previous experiment. In all cases, the visual scanning patterns seemed to exhibit colored noise which was theorized to be a characteristic signature of complexity.

This research aims to develop a framework for the modelling and design of automation. As an alternative to Rasmussen’s Decision Ladder, the framework would be used to create a psychologically relevant model of the automation that is based on the intentions of the actors rather than information-processing activities. Along with the model of the work domain created using the Abstraction-Decomposition Space, this shared mental model of the automation would then be used for task allocation between human and machine actors and for the design of automation and operator support tools (e.g., controls, displays and procedures). The framework will be developed and evaluated in the context of designing operator support tools for a petrochemical simulation.

The purpose of this study was to address the effects of interface design on operator adaptation. A longitudinal experiment lasting 6 months was conducted to compare the P+F and P interfaces for the DURESS II system under a variety of conditions, including normal trials, routine faults, and non-routine faults. Product measures (e.g., time, actions) and process measures (e.g., verbal protocols) of performance were collected. In addition, several knowledge elicitation measures were periodically administered to determine how subjects' knowledge organization evolved over time. The results showed that, on normal trials, there was very little difference in the average performance of the interface groups. The group using the P interface, however, consistently showed more variability in their performance, occasionally taking much longer than usual to complete the required tasks. This effect was found to hold even after extensive practice, and was specific to the interface, not to the subjects. Second, for both routine and non-routine faults, the P+F interface was found to lead to better performance, especially with respect to diagnosis accuracy. These effects seemed to stem from strategy differences between interface groups, which were in turn a result of the interaction between subjects' knowledge and the information provided in the interfaces. Third, subjects using the P+F interface who actively explored the system and reflected on the feedback provided were able to achieve levels of adaptation and performance not observed with subjects using the P interface. One P+F subject, however, adopted a surface approach to learning and therefore exhibited a very shallow knowledge base and poor performance (although no worse than that attained by subjects using the P interface with a similar motivation level). Thus, it seems that there are certain preconditions which have to be satisfied if the benefits of the P+F interface are to be fully enjoyed.

This study analyzed the data for normal trials of JAERI I in greater detail. The goal was to examine the effect of interface designs on participants' adaptation, focusing on the role of individual differences in process control. The experiment was conducted using the traditional physical (P) and the ecologically designed (P+F) interfaces for DURESS II. A detailed analysis of a group of early and late start-up trials was conducted in order to investigate changes in expertise. A series of performance measures were developed as indicators of adaptation. Generally, participants effectively using the P interface, unlike those using the P+F interface, used mainly standardized procedures for operating the system. All participants made few metacognitive statements while thinking aloud during normal trials. Further, participants using the P+F interface exhibited better performance on both normal and fault trials if they scored higher in the holist cognitive style test. These findings have important practical implications for the measurement of adaptation and the selection of operators.

The aim of this project was to develop novel analytical measures of adaptation performance that would allow us to further look into the existing database available from JAERI I, JAERI II and JAERI III. The research has lead to innovative methodological and empirical contribution to the study of characteristics of long-term adaptation. These measures developed are formal, analytical, and more complicated than those we used in the previous phases. The analysis was solely based on the log files and implemented by computer. Thus, the results were objective, automated and more reliable. Second, the topic of how individual differences, particularly how the Holist-Serialist cognitive style distinction, interacted with the other factors that had been investigated in shaping operator adaptation were studied. The results of this project have important implications for the selection of operators for nuclear power plants with advanced control room designs.

The fifth phase of this research program continued the research performed for the JAERI IV contract. First, the novel measures of adaptation identified and developed for that project were here extended to the analysis of adaptation to changing system structure (i.e., faults). These findings were then integrated with the previous findings from JAERI IV in order to paint a more complete picture of operator adaptation in the DURESS II microworld. Second, a literature review of alternative methods of statistical inference was carried out. The aim of this literature review was both to discover and develop a number of new statistical methods to apply to the database from JAERI I, JAERI II, and JAERI III. These novel techniques have been successfully applied to existing data, and show promise for use in future experiments.

The Abstraction-Decomposition Space (ADS) has had its theoretical foundations and modelling principles elucidated (Rasmussen, 1994; Vicente, 1999) and design methods described (Burns & Hajdukiewicz, 2004; Naikar et al., 2005; Jenkins et al., 2007). Nevertheless, there exists confusion regarding the ADS, and it is still seen by some as an ‘unclear’ modeling tool. Even proponents of the ADS acknowledge that more work needs to be done to facilitate portability and reuse of analyses captured in Abstraction-Decomposition Spaces (Vicente, 2002). To address these issues, we propose conducting an ontological analysis (Genesereth and Nilsson, 1987) in order to develop a formal model of the ADS. Such a model will allow for more clear and cogent discussion of the merits of the ADS, and provide an unambiguous reference point from which to discuss extensions of the ADS. In addition, a formal structure will facilitate reuse of full or partial Abstraction-Decomposition Spaces, and can provide a starting point to the development of tools to aid in the development, verification and display of these models.

The purposes of this project were, a) to investigate whether Rasmussen’s Risk Management framework could be used to describe how and why SARS was transmitted throughout the Greater Toronto Area, and b) to compare the results of this analysis with those found in two prior investigations of Canadian water distribution incidents. A qualitative case study methodology was used to structure this research and two tools of the Rasmussen framework, an AcciMap and a Conflict Map were used to structure the analysis of the data. The Report of the National Advisory Committee on SARS and Public Health served as the primary data source, from which evidence that supported or contradicted the predictions of the framework was extracted. The results showed that six of the seven predictions of the Rasmussen framework were supported by the events described in the report. There was considerable overlap of the contributing factors at the higher levels of the sociotechnical systems shown in the AcciMap (i.e., government, regulatory bodies) across the investigations compared in this study. As expected, there were few common factors at the lower levels of the system (i.e., work equipment and surroundings, physical processes and technical/operational management) because the work process differed greatly.

The purpose of this project was to test the usefulness of Rasmussen’s Risk Management framework for explaining how and why accidents occur in the food production domain. To this end, the framework was used to retrospectively investigate how and why BSE was transmitted through the human and animal food supply in the UK from 1986 to 1996.  A qualitative case study methodology was used to structure this research and two tools of the Rasmussen framework, an AcciMap and a Conflict Map were used to structure the analysis of the data. The primary data source was the BSE Inquiry Report; a published document over 5000 pages in length. The results showed that evidence for each of the framework’s seven predictions was identified, suggesting that this framework is useful for explaining how accidents happen in the food production domain. The specific findings (identified system weaknesses) may be useful for guiding system re-design, including policy changes.

Accidents in different complex sociotechnical systems are rarely compared using the same theoretical framework for risk management. We conducted a comparative analysis of two Canadian public health disasters involving drinking water distribution systems, the North Battleford Cryptosporidium parvum outbreak in April 2001 and the Walkerton E. coli outbreak in May 2000 using Rasmussen’s theoretical framework for risk management. Both accidents resulted from a complex interaction between all levels of a complex sociotechnical system. However, the low-level physical and individual factors differed in the two cases, whereas, the high-level governmental and regulatory factors tended to be the same. These findings may have implications for the design of public policies to minimize risk in complex sociotechnical systems.

Previous research [AECL 93] has revealed certain problems with the ecological interface (P+F) that seemed to be caused by a lack of temporal information. This project addressed this problem by looking at the impact of alternative implementations of temporal information in the context of a modified version of DURESS II. The results indicated that it is possible to integrate temporal information with the emergent features of the P+F interface by using the concept of analytical redundancy developed in control theory. An experimental evaluation indicated that such an interface can lead to improved performance, compared to more traditional interfaces, with and without trend plots. The contributions of this research were twofold: the addition of analytical redundancy to the EID framework; and, a novel display that can be used effectively in any setting where it is important to monitor a conservation relation (e.g., mass, energy, parts, money, people, charge).

The purpose of this project was to develop a theoretically based analysis framework to study adaptive behaviour in complex systems. The project focused and structured quantitative analysis of adaptive, nonlinear human work performance, and develop measurement methods and analysis tools to quantify adaptive performance. It was intended that this analysis framework will eventually be used to study adaptive behavior in various military situations. The scope of this project was to (1) develop an analysis framework for studying adaptive behavior in complex systems, (2) develop analytic and empirical measures for assessing adaptation and performance, (3) provide a preliminary discussion of how the analysis framework and measurement tools may be used in military situations, and (4) develop a research plan to test the analysis framework and measurement concepts.