This project was a continuation of ABB 95. A literature review was conducted examining the problems with computer interfaces for large systems. Theories of navigation and design concepts for aiding navigation were also reviewed. This project revealed that the loss of functional linking and connecting information was an area of EID most vulnerable to the design of ecological interfaces for large systems. This project cumulated in a plan for an investigation of the effects of different approaches to integration on large scale ecological interface design. This plan was executed in ABB 97.

This project was the final leg of the ABB research series culminating in an experimental investigation. Based on an abstraction hierarchy of the ABB plant made in ABB 96, views were created for each cell of the abstraction hierarchy. These views were then integrated in three different ways based on a novel use of a space-time approach to describing integration. The details of this approach are available in CEL 97-05 and Burns (1998). The three displays employed low spatial-high temporal integration, high spatial-low temporal integration, and high spatial-high temporal integration of means-end information. Subjects performed information search tasks as well as fault detection and diagnosis tasks. It was found that the spatial and temporal proximity of means-end related information affects the traversal of means-end connections. In particular, high spatial and temporal integration resulted in significantly faster and more accurate fault diagnosis performance. This research is a unique look at integration issues with a large plant simulation and helps to expand the application of ecological interface design to large systems.

The objectives of this research were to look closer at understanding the critical success factors in adapting to change from a fundamental point of view. To assist in studying these issues, concepts from ecological psychology and motor control, in particular coordinative structures (i.e., functionally higher-level control), provided some insight. The hypotheses for this research are noted below:

1. People can use functionally higher-level control to successfully adapt to change.
2. Functionally higher-level control can be induced by invariant, goal-relevant information about the work domain, with people attuned to this structure.
3. Functionally higher-level control can be made possible by the existence and utilization of wide opportunities for component resource selection and allocation (i.e., multiple degrees of freedom).

The research was narrowed down to three experiments: (1) adaptation to global changes in component dynamics, (2) adaptation to local changes in component dynamics, and (3) adaptation to changes in interface form. The experiments were conducted using DURESS II, a thermal-hydraulic microworld simulation environment, as the testbed. The participants used one of two different interfaces to control the process, each developed using different interface design principles (Vicente and Rasmussen, 1990). The P interface, designed using a more traditional approach (i.e., mimic design), displayed primarily physical information about the work domain. The P+F interface, designed using the principles of EID, displayed both physical and functional information about the work domain derived from an AH analysis of DURESS II.

This project was a continuation of the research performed under project WEC 95. The purpose of this work was to evaluate the validity and generalizability of a preliminary model of operator cognitive monitoring developed in the previous project for the AECB. In this phase, meetings were held with operations staff at Pickering B to obtain feedback from the report we had written for earlier work. In addition, the report was distributed to a number of operators and their comments were recorded. Additional data were collected at Pickering B by observing control operator monitoring activities over a total of approximately 65 hours. The results obtained from these activities indicate that the contents of our previous report are generalizable across operators. Moreover, additional insights were uncovered which have important implications for a revised model of operator cognitive monitoring.

This is a continuation of the research performed under project WEC 95. The purpose of this work was to evaluate the validity and generalizability of a preliminary model of operator cognitive monitoring developed in the previous project for the AECB. In this phase, meetings were held with operations staff at Pickering B to obtain feedback from the report we had written for earlier work. In addition, the report was distributed to a number of operators and their comments were recorded. Additional data were collected at Pickering B by observing control operator monitoring activities over a total of approximately 65 hours. The results obtained from these activities indicate that the contents of our previous report are generalizable across operators. Moreover, additional insights were uncovered which have important implications for a revised model of operator cognitive monitoring.

The purpose of this project was to evaluate the ecological interface design (EID) framework under a more representative set of conditions than previous research on EID. DURESS II, a fully interactive real-time simulation of a thermal-hydraulic process was used as a testbed for the project. Performance with two interfaces was compared: a P+F interface containing physical and functional system representations, and a traditional P interface based solely on a physical representation of the system. Subjects were given 4 weeks of daily practice with one of the two interfaces before their performance on normal events and unfamiliar faults was evaluated. Under normal conditions, there was no performance difference between the two interfaces. However, dual task results indicate that the P interface loads more on verbal resources, whereas the P+F interface loads more on spatial resources. Furthermore, a process tracing analysis of the fault trials showed that the P+F interface led to faster fault detection and more accurate fault diagnosis performance. In addition, a deficiency of the P+F interface was identified, suggesting a need for integrating temporal information with emergent feature displays. These findings have significant practical implications for the design of advanced interfaces for complex systems.

The purpose of this research was to improve the fluency of work by introducing "see-through" or semi-transparent user interface objects such as windows, icons, tool palettes, etc. This particular project consisted of controlled laboratory experiments which evaluated divided and focused attention and visual interference issues in semi-transparent interface designs. To this end, a series of experiments were conducted which introduced progressively more realistic task elements taken from the target task domain. The results of these experiments were subsequently used to inform our design choices within selected industrial applications, in particular, 3-D modeling, animation, and painting applications. A case study of transparent user interface objects in a working application was completed. The results of these studies indicate that transparency can improve performance, but that several obstacles must be overcome before this technology can be introduced effectively into commercial products.

We are currently conducting a field study of energy management at large Ontario industrial enterprises such as pulp mills, steel foundries, and mining operations. Industrial energy managers are responsible for reducing a company's energy costs and associated environmental impacts by inventing, planning, and implementing means to reduce or shift electricity consumption. The cognitive work involved seems to differ from industrial process control as previously studied: it is less structured, more insight-dependent, highly social, and rarely supported by dedicated information technology systems. We plan to apply the Cognitive Work Analysis theoretical framework to model the recurring features we observe to be common across industries, and in future projects apply these models to the development of general-purpose energy management support tools accessible to both large and small enterprises.

We examined the question of how automation reliability affects human performance in a multi-task environment. Subjects are asked to perform two manual tasks using a flight simulation software (The MAT battery): a tracking task, and a fuel management task. At the same time, subjects had to monitor and detect occasional failures of an automated system that controls four gauges. An eye-tracking system, ERICA, was used to determine how participants allocated their attention across the different tasks.

The goals of this research were twofold.

The first goal was to further investigate the issue of complacency or over trust in automation. To date, most of the literature on complacency had used the detection rate of automation failures as an indication of the monitoring behavior. As automation failures started to go undetected by participants, they were said to show sign of complacency. Our first study replicated one of the studies where complacency was observed, but this time with eye movement recording. As in previous studies, monitors of constant high-reliability automation performed poorly compared to participants in other conditions. However, whereas this performance might previously have been attributed to complacency, eye movement recordings showed that, following an initial lag in sampling of the automated task, participants in the constant high reliability condition began sampling more frequently – eventually matching sampling patterns of other participants. This observation tends to argue against complacency as an explanation for the poor monitoring performance. That no differences in self-reported trust in the automation between groups were observed further discounts the complacency/overtrust attribution.

The second goal of the study was to investigate the effect of context-related automation failures on operator reliance. To date, most of the existing literature on human-automation interaction dealt with random failures of the automation. Furthermore, participants were usually not given any information about how and why the automation might fail. However, in real settings, certain phases of operation or particular environmental conditions are more or less likely to be associated with automation failures, and operators usually know it. Thus, in our second study, participants were given information about the automation reliability, and why it might fail. In contrast to the first the first study, the performance of monitors of constant high reliability automation was indistinguishable from that of all other groups. Moreover, eye tracking data showed that the monitoring behavior of these participants was more consistent and more frequent than in the first study. These observations suggest that providing operators with information about the context sensitive nature of automation reliability might alleviate the monitoring performance decrement associated with highly reliable automation.

In Phase 2 of the project [HTC 97], we mapped the integrated information requirements into a pair of ecological interfaces. One interface (P+F) contained information exclusively from the system-based model while the other interface (P+F+T) contained information from both system- and task-based models. An iterative design approach was used in which process engineers and operators critiqued early designs. The final designs were implemented in software, connected to a full-scope industry simulator, and validated. An empirical evaluation compared operator performance on the two ecological interfaces and the contemporary interface.

Friendly fire is an enduring problem in modern warfare. Investigations have revealed that human errors in combat identification (CID) played a major part in most friendly fire incidents. Automated CID systems, which often comprise “question and answer systems”, have been developed and provided to soldiers. However, both field observations and empirical data suggest that soldiers tend to overly trust in and rely on these automated systems which are in fact not perfectly reliable.

The purpose of this study is to support soldiers’ appropriate trust in CID systems. This study includes two phases: in the first phase, we will conduct two successive experiments to explore soldiers’ interaction with a CID system and discover crucial information that can engender soldiers’ appropriate trust in the system; in the second phase, we will design an interface for the system based on the findings in the first phase.

Situation Awareness (SA) is a theoretical construct concerning the knowledge of the situation in relation to technology design and influence on operator performance. Regardless of the controversies of its theoretical basis, it seems practical to assess operator SA as knowledge of the situation, which is intuitively necessary to control technology appropriately. 

In conducting a previous study [SRO 04], we found that existing SA models, as a natural outcome of generalization, do not provide the necessary foundation to formulate operational definitions and to derive measurements for the process control domain. These theoretical models, while useful to communicate high level findings, often exclude complexities and details pertinent to the domain and tasks performed by operators, but include a host of cognitive aspects associated with tasks from unrelated domains. In effect, operational definitions based on such models cannot provide any precise descriptions of the cognitive processes engaged by process control operators in acquiring SA.

This project aims to develop a domain-specific SA model for the process control domain that provides the necessary foundation to formulate meaningful operational definitions and derive indicative measurements.

A contract was signed with Lawrence Erlbaum & Associates to write a textbook/monograph on the framework for cognitive work analysis (CWA) developed by Rasmussen. The book was written in a pedagogic fashion with many examples, the goal being to make these ideas more accessible to a much wider audience than they have been in the past. Five phases of cognitive work analysis were described: work domain representation, control task descriptions, mental strategies, social organization, and operator competencies. Each phase was thoroughly illustrated with an application example that was followed through in the book, DURESS II from the domain of process control. The thesis of the book was that, by adopting CWA, systems designers can create information technology that leads to safe, productive, and healthy work environments.

In a previous study, Lin et al. (1998, 2001) redesigned the interface for a patient-controlled analgesia (PCA) infusion pump using human factors principles. Experiments were conducted whereby nurses performed simulated programming tasks on the new and old interfaces. It was found that performance time was faster and concentration programming errors were eliminated with the new interface. A similar study was conducted by Ford and Rollinson (2001) on another infusion pump, and similar results were found. However, nurses were frequently interrupted when they programmed PCA pumps in reality, and so the effect of interruptions should be investigated. To this end, field observations were conducted in the Post-Anesthetic Care Unit at the Toronto General Hospital, and the types and frequencies of interruptions that the nurses encountered were documented. The results obtained from this field study were used to design more realistic experiments in order to evaluate the new and old interfaces for the drug infusion devices discussed above. Experimental interruptions were designed to reflect the cognitive demands imposed by interruptions observed in the nursing workplace.

The purpose of this study was to address the effects of interface design on operator adaptation. A longitudinal experiment lasting 6 months was conducted to compare the P+F and P interfaces for the DURESS II system under a variety of conditions, including normal trials, routine faults, and non-routine faults. Product measures (e.g., time, actions) and process measures (e.g., verbal protocols) of performance were collected. In addition, several knowledge elicitation measures were periodically administered to determine how subjects' knowledge organization evolved over time. The results showed that, on normal trials, there was very little difference in the average performance of the interface groups. The group using the P interface, however, consistently showed more variability in their performance, occasionally taking much longer than usual to complete the required tasks. This effect was found to hold even after extensive practice, and was specific to the interface, not to the subjects. Second, for both routine and non-routine faults, the P+F interface was found to lead to better performance, especially with respect to diagnosis accuracy. These effects seemed to stem from strategy differences between interface groups, which were in turn a result of the interaction between subjects' knowledge and the information provided in the interfaces. Third, subjects using the P+F interface who actively explored the system and reflected on the feedback provided were able to achieve levels of adaptation and performance not observed with subjects using the P interface. One P+F subject, however, adopted a surface approach to learning and therefore exhibited a very shallow knowledge base and poor performance (although no worse than that attained by subjects using the P interface with a similar motivation level). Thus, it seems that there are certain preconditions which have to be satisfied if the benefits of the P+F interface are to be fully enjoyed.

This study analyzed the data for normal trials of JAERI I in greater detail. The goal was to examine the effect of interface designs on participants' adaptation, focusing on the role of individual differences in process control. The experiment was conducted using the traditional physical (P) and the ecologically designed (P+F) interfaces for DURESS II. A detailed analysis of a group of early and late start-up trials was conducted in order to investigate changes in expertise. A series of performance measures were developed as indicators of adaptation. Generally, participants effectively using the P interface, unlike those using the P+F interface, used mainly standardized procedures for operating the system. All participants made few metacognitive statements while thinking aloud during normal trials. Further, participants using the P+F interface exhibited better performance on both normal and fault trials if they scored higher in the holist cognitive style test. These findings have important practical implications for the measurement of adaptation and the selection of operators.

This is the second phase of the research program investigating factors influencing human cognitive behaviour. It was built on the results of earlier work [JAERI I, JAERI IA] by investigating the effects of theoretical training on operator adaptation using the P and P+F interfaces. The theoretical training consisted of teaching an abstraction hierarchy description of DURESS II over the course of eight days. The purpose was to determine to what extent the theoretical training could provide operators with a basis for dealing with novel events (e.g., disturbances and faults). Several performance measures were used including: time measures, verbal protocol analysis, system termination counts, and an analysis of system control "recipes" written by subjects. The experimental results showed that the P+F interface groups consistently outperformed the P interface groups. Also, the Training groups improved more than the No Training groups on most performance measures, suggesting that training based on the abstraction hierarchy can lead to improved performance. There was little evidence of an interaction between interface and training.

This portion of the third phase of this research program examined whether, in the absence of training, requiring subjects to explain their actions aloud enhanced adaptation using the P+F interface for DURESS II. Participants in both the replay and self-explanation (SE) groups occasionally watched a replay of their own performance immediately after completing a trial, while the control group did not. In addition, the SE group was instructed to explain aloud the reasons for their control actions while watching the replay. The "good" SE participants included considerable more words suggesting self-explanations in their protocols than did the "poor" SE participants. There were no substantial differences between the SE, replay, and control groups on normal trials. However, the SE participants did have the best overall performance on fault trials, suggesting that self-explanation can aid performance, particularly for recurring routine faults.

This portion of the third phase of this research program compared the P+F interface to a new multi-level interface, which divides the P+F interface for DURESS II into four separate windows (corresponding to the four levels of the abstraction hierarchy). This allowed us to trace, at a crude level, operator strategies for navigating among the four views in order to use the display information. Subjects who made more frequent use of functional levels of information exhibited more accurate system control under normal conditions, and more accurate diagnosis performance under fault trials. Moreover, subjects who made efficient use of functional information exhibited faster fault compensation times. In contrast, subjects who made infrequent or inefficient use of functional information exhibited poorer performance on both normal and fault trials. These results provide specific evidence of the advantages of an abstraction hierarchy interface over more traditional interfaces which emphasize physical rather than functional information.

The aim of this project was to develop novel analytical measures of adaptation performance that would allow us to further look into the existing database available from JAERI I, JAERI II and JAERI III. The research has lead to innovative methodological and empirical contribution to the study of characteristics of long-term adaptation. These measures developed are formal, analytical, and more complicated than those we used in the previous phases. The analysis was solely based on the log files and implemented by computer. Thus, the results were objective, automated and more reliable. Second, the topic of how individual differences, particularly how the Holist-Serialist cognitive style distinction, interacted with the other factors that had been investigated in shaping operator adaptation were studied. The results of this project have important implications for the selection of operators for nuclear power plants with advanced control room designs.

The fifth phase of this research program continued the research performed for the JAERI IV contract. First, the novel measures of adaptation identified and developed for that project were here extended to the analysis of adaptation to changing system structure (i.e., faults). These findings were then integrated with the previous findings from JAERI IV in order to paint a more complete picture of operator adaptation in the DURESS II microworld. Second, a literature review of alternative methods of statistical inference was carried out. The aim of this literature review was both to discover and develop a number of new statistical methods to apply to the database from JAERI I, JAERI II, and JAERI III. These novel techniques have been successfully applied to existing data, and show promise for use in future experiments.

While interacting with complex systems, operators of these systems frequently modify their interfaces in order to achieve task goals more effectively or more efficiently (what Rasmussen (1986) has called "finishing the design"). While there is a literature on operator modifications, this literature includes only field studies and theoretical treatments, but has not proceeded to experimentation. This research describes two preliminary experiments designed to answer four questions related to operator modifications: (1) Why do operators modify their interfaces? (2) How do these modifications develop over time? (3) How does this activity affect performance? and (4) How does this activity affect understanding? The results of these experiments pointed to answers to most of these questions, and most significantly, pointed to a taxonomy of the purposes behind operator modifications in complex systems.

This project studied the behaviour of pharmacists, nurses, and physicians in the hospital setting, specifically looking at how these clinician types worked within the medication process. We observed clinicians in two separate units of a downtown teaching hospital during normal (i.e. non-emergency) conditions. Information from the observations was supplemented with both impromptu and semi-structured interviews. Data analysis focused on the relative influence of two variations within the medication process: 1) the level of technology integration of the respective units, and 2) the existing work practices followed by clinicians in each unit. The impact of these variations on clinician workflows, information sources used, and difficulties associated with work within the two units were explored. Based on the empirical data collected, a model of the Intrinsic Order Process was presented that described the medication process across clinician types and irrespective technology use and work practices.

The purpose of this thesis was to examine the generalizability of Ecological Interface Design (EID) to a new work domain. Computer networks provide a source of complexity unique among domains studied from the perspective of EID: because of the ability of network operators to add and remove devices, and change configurations, the work domain itself is much more fluid than those previously studied. A prototype interface was created to test the validity of Rasmussen’s Abstraction Hierarchy in determining information requirements that would assist users in monitoring a network to detect and diagnose faults. A pilot experiment was conducted to test the interface in an experimental setting; this was followed by a more complete experiment, the results of which faster detection times, improved rate of detection under higher loads, and improved quality of diagnosis (with greater consistency under higher loads) indicate that the EID framework is applicable to this new domain.

The purpose of this project was to examine the effects of sensor noise on Ecological Interface Design (EID). More specifically, the project aimed to investigate how sensor noise will affect operators’ control strategies and performance.

Modern process control plants are made of several components which are often structured in a complex manner. In order to monitor the interactions between all the different components, sensors are installed at strategic locations throughout the plant. One of the main functions of sensors is to probe the physical plant (world) and send the acquired information to an interface, so that it can be monitored by operators.

Despite the current state of technology when it comes to sensing devices, information transmitted by the instrumentation and control equipment is often noisy. That is, sensors will have a certain range of inaccuracy and propagation of the signal from the sensing device to the control panel is likely to contain some noise. In that sense, data about the state of world will be uncertain, affecting both the interface content and the ways operators will cope with the noisy equipment.

The purpose of this project was to examine the effects of sensor noise on Ecological Interface Design (EID). While a large number of studies have shown that EID improves performance, there has not been to date an investigation of the effects of the presence or magnitude of sensor noise on performance and control strategies using an EID display. A series of experiments were conducted using the DURESS II microworld. The principles of Dynamical System Theory (DST) were used to analyse strategy shifts while operators had to cope with EID displays that contain sensor noise.

Results from this project may help in improving the design process of interfaces for complex systems as well as determining the robustness of EID framework and its applicability in real industrial settings.

This work was motivated by an applied problem (i.e., how to improve the design of existing human-computer interfaces for network management) and a basic research question (i.e., how ecological interface design (EID) can support collaborative work). We completed a position paper to summarize why EID is expected to provide support for collaborative work in complex sociotechnical systems. We also conducted a field study at a network operating centre in industry to understand the cognitive and collaborative demands associated with real-time network management, and the state-of-the-art in human-computer interfaces for network management. We then proposed two different approaches to modelling heterogeneous telecommunication networks that provide service to diverse end users, both of which utilized the abstraction hierarchy (Rasmussen, 1986) framework. These two approaches have the potential to yield vastly different EID interfaces, even if both are intended to support work in the same work domain.

The purpose of this project was to test the usefulness of Rasmussen’s Risk Management framework for explaining how and why accidents occur in the food production domain. To this end, the framework was used to retrospectively investigate how and why BSE was transmitted through the human and animal food supply in the UK from 1986 to 1996.  A qualitative case study methodology was used to structure this research and two tools of the Rasmussen framework, an AcciMap and a Conflict Map were used to structure the analysis of the data. The primary data source was the BSE Inquiry Report; a published document over 5000 pages in length. The results showed that evidence for each of the framework’s seven predictions was identified, suggesting that this framework is useful for explaining how accidents happen in the food production domain. The specific findings (identified system weaknesses) may be useful for guiding system re-design, including policy changes.

Accidents in different complex sociotechnical systems are rarely compared using the same theoretical framework for risk management. We conducted a comparative analysis of two Canadian public health disasters involving drinking water distribution systems, the North Battleford Cryptosporidium parvum outbreak in April 2001 and the Walkerton E. coli outbreak in May 2000 using Rasmussen’s theoretical framework for risk management. Both accidents resulted from a complex interaction between all levels of a complex sociotechnical system. However, the low-level physical and individual factors differed in the two cases, whereas, the high-level governmental and regulatory factors tended to be the same. These findings may have implications for the design of public policies to minimize risk in complex sociotechnical systems.

The purposes of this project were, a) to investigate whether Rasmussen’s Risk Management framework could be used to describe how and why SARS was transmitted throughout the Greater Toronto Area, and b) to compare the results of this analysis with those found in two prior investigations of Canadian water distribution incidents. A qualitative case study methodology was used to structure this research and two tools of the Rasmussen framework, an AcciMap and a Conflict Map were used to structure the analysis of the data. The Report of the National Advisory Committee on SARS and Public Health served as the primary data source, from which evidence that supported or contradicted the predictions of the framework was extracted. The results showed that six of the seven predictions of the Rasmussen framework were supported by the events described in the report. There was considerable overlap of the contributing factors at the higher levels of the sociotechnical systems shown in the AcciMap (i.e., government, regulatory bodies) across the investigations compared in this study. As expected, there were few common factors at the lower levels of the system (i.e., work equipment and surroundings, physical processes and technical/operational management) because the work process differed greatly.

This project was an analysis of the Walkerton outbreak from the perspective of Rasmussen's framework for risk management in a dynamic society. The results demonstrated how the outbreak can be understood in terms of interacting events at multiple levels, ranging from the immediate physical circumstances to the activities of local and provincial governments.

The purpose of this project was to map the theoretical implications of EID to effects on the quality of work. The theoretical implications under investigation were: skill enhancement, worker competence, and decision latitude (strategic degrees of freedom). Relevant work literature and case studies were reviewed. The field of social epidemiology emerged as an important body of theory relating the three theoretical implications of EID to working health. The review indicated that EID should have a positive impact on workplace in terms of health, given that organizational and social climate are congruent with EID's philosophy.

The purpose of this project is to investigate whether Situation Awareness (SA) can be supported through Ecological Interface Design (EID). SA research investigates how “knowledge of the situation” can influence decision-making and performance while EID research examines how system information during operations can be conveyed through human-machine interfaces. Thus, it is hypothesized that ecological interfaces can support SA and adequate SA relies on information of ecological interfaces.

In this project, an ecological interface was developed for the secondary side of a boiling water reactor plant operating in Sweden. The ecological interface was implemented in a full-scoped simulator and evaluated using SA measures against a computerized version of the plant interface. The ecological interface was found to support SA in beyond-design basis scenarios better than the current plant interface. However, the current plant interface seemed to support SA better than the ecological interface when control actions were required during within-design basis scenarios. The empirical study indicates that EID is capable of supporting SA, particularly in beyond-design basis scenarios.

This project studied the monitoring behaviour of nuclear power plant operators during normal operations at the Pickering Nuclear Generating Station B. Operators were observed during actual operations at the station. As well, a literature review of factors influencing monitoring behaviour was conducted. This study revealed that operators develop a number of creative adaptation strategies to overcome the deficiencies in the existing control room interface, thereby minimizing the demands associated with monitoring and improving performance.

The purpose of this project was to develop a theoretically based analysis framework to study adaptive behaviour in complex systems. The project focused and structured quantitative analysis of adaptive, nonlinear human work performance, and develop measurement methods and analysis tools to quantify adaptive performance. It was intended that this analysis framework will eventually be used to study adaptive behavior in various military situations. The scope of this project was to (1) develop an analysis framework for studying adaptive behavior in complex systems, (2) develop analytic and empirical measures for assessing adaptation and performance, (3) provide a preliminary discussion of how the analysis framework and measurement tools may be used in military situations, and (4) develop a research plan to test the analysis framework and measurement concepts.

Measuring adaptive expert behavior has become an issue of interest and controversy, particularly in complex work environments where conditions can change unexpectedly. As a result, there is a need to develop a set of tools that can help assess adaptive performance and distinguish between expert and novice behavior. The purpose of this research was to develop and test measures of adaptive performance for these types of environments. In particular, the project scope was to: 1) develop and evaluate a within-trial measure for assessing adaptive performance, and 2) conduct a pilot study on operator adaptation to global changes in process time constants using DURESS II, a thermal-hydraulic microworld simulation environment. The results show that the within-trial measure developed, within-trial trajectory deviation (WTD), may be useful for assessing coupling to the work domain structure (i.e., Abstraction Hierarchy - AH). When viewing this measure across trials, the results are consistent with previously developed measures for coupling to the AH. In the pilot study, the results using previously developed measures suggest that the type of interface may affect an operator's ability to adapt to changing process dynamics. An interface designed using the physical and functional properties of the work domain tended to better support functional adaptive behavior, compared with an interface designed with a traditional approach using only physical properties of the work domain. In addition, more experienced operators were able to adapt to changing process dynamics, with a larger expertise effect in trials having large process time constants. More detailed experiments and the development of additional measures to assess adaptive performance are suggested for future research.