The research objective of this study is to investigate the impact of using a VA to support crew members in the context of anomaly treatment during Long Duration Exploration Missions (LDEM), when ground support will be limited. This study will specifically investigate the impact of different VA capabilities related to troubleshooting anomalies on several human- related metrics including human performance, cognitive workload (CW), situational awareness (SA), and trust.

The specific research aims of this proposal are as follows:

1. Measure the impact of the baseline VA (with question answering, but without the ability to engage in dialogue or take initiative) on performance, CW, SA, and trust in a laboratory environment
   
2. Measure the impact of the enhanced VA (providing explanations and taking initiative) on performance, CW, SA, and trust in a laboratory environment
   
3. Deploy and validate the system in an analog environment
   
   

   
   ++ -- View more

   APPROACH:
   A VA will be developed building upon the software architecture from existing VAs developed by the PI. The VA will provide support for various aspects of anomaly treatment, including detecting and diagnosing the anomaly, as well as recommending a course of action. It will also have the ability to take initiative in the dialog with the user (mixed-initiative mode), and the ability to provide explanations for its actions. The impact of the VA on human performance, CW, SA, and trust will be assessed through a set of three experiments with human subjects in a laboratory environment. The first experiment will establish the baseline impact (no explanations), and subsequent experiments will study the effect of switching to the mixed-initiative mode and adding explanations. Finally, the system will be deployed and tested in an analog environment.

   Experiment 1: Study and characterize the use of a baseline VA (without the ability to provide advanced explanations or take initiative) on human performance, CW, SA, and trust. Subjects will be asked to resolve multiple anomalies that are representative of a flight scenario, with and without the help of the VA. This experiment will be performed in a laboratory environment.

   Validation with analog: The baseline version of the VA will also be deployed and tested in NASA’s HERA analog environment. Four 45-day HERA missions will be conducted. In each campaign, each crew member will conduct roughly one 1-hr session per week during which they will work with the VA solving a number of simulated anomalies. The same measures collected in Experiment 1 will be measured here.

   Experiment 2: Study the impact of giving the VA the capability to provide explanations for its actions. Human performance, CW, SA, and trust measures will be collected. A new set of subjects will participate in this study. Results of this experiment will be compared to those from Experiment 1. This experiment will be performed in a laboratory environment.

   Experiment 3: Study the impact of a mixed-initiative VA (the VA has the capability to take initiative in the dialog with the user). Human performance, CW, SA, and trust measures will be collected. A new set of subjects will participate in this study. Results of this experiment will be compared to those from the previous experiments. This experiment will be conducted in a laboratory environment.

   Approximately 15 subjects for each experiment are needed to get significant results based on relevant past data. The aim is to recruit healthy subjects from a population with a level of expertise representative to the astronaut population (with a degree or working towards a degree in engineering or sciences and ideally some background or prior knowledge in aerospace engineering). Subjects will go through a 2h training session that includes 1) General information about how the main equipment in the spacecraft works. 2) Information on how the VA works. 3) Information about the task at hand. In each of the experiments, subjects will work on resolving several anomaly scenarios individually.

   SA will be measured objectively using the Situation Awareness Global Assessment Technique (SAGAT) test and subjectively using the Situation Awareness Rating Technique (SART), using exit questionnaires. CW will be measured using a secondary task and NASA’s Task Load Index. Trust will be measured using Jian’s subjective scale for trust in automation.

   RESULTS:
   This investigation is currently in progress. Results will be reported when available.