APPROACH:
A VA will be developed building upon the software architecture from existing VAs developed by the PI. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on human performance, CW, SA, and trust will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. Finally, the system will be deployed and tested in an analog environment.
Experiment 1: Study and characterize the use of a baseline VA (without the ability to provide advanced explanations or take initiative) on human performance, CW, SA, and trust. Subjects will be asked to resolve multiple anomalies that are representative of a flight scenario, with and without the help of the VA. This experiment will be performed in a laboratory environment.
Validation with analog: The baseline version of the VA will also be deployed and tested in NASA’s HERA analog environment. Four 45-day HERA missions will be conducted. In each campaign, each crew member will conduct roughly one 1-hr session per week during which they will work with the VA solving a number of simulated anomalies. The same measures collected in Experiment 1 will be measured here.
Experiment 2: Study the impact of giving the VA the capability to provide explanations for its actions. Human performance, CW, SA, and trust measures will be collected. A new set of subjects will participate in this study. Results of this experiment will be compared to those from Experiment 1. This experiment will be performed in a laboratory environment.
Experiment 3: Study the impact of a mixed-initiative VA (the VA has the capability to take initiative in the dialog with the user). Human performance, CW, SA, and trust measures will be collected. A new set of subjects will participate in this study. Results of this experiment will be compared to those from the previous experiments. This experiment will be conducted in a laboratory environment.
Approximately 15 subjects for each experiment are needed to get significant results based on relevant past data. The aim is to recruit healthy subjects from a population with a level of expertise representative to the astronaut population (with a degree or working towards a degree in engineering or sciences and ideally some background or prior knowledge in aerospace engineering). Subjects will go through a 2h training session that includes 1) General information about how the main equipment in the spacecraft works. 2) Information on how the VA works. 3) Information about the task at hand. In each of the experiments, subjects will work on resolving several anomaly scenarios individually.
SA will be measured objectively using the Situation Awareness Global Assessment Technique (SAGAT) test and subjectively using the Situation Awareness Rating Technique (SART), using exit questionnaires. CW will be measured using a secondary task and NASA’s Task Load Index. Trust will be measured using Jian’s subjective scale for trust in automation.
RESULTS:
This investigation is currently in progress. Results will be reported when available.