You are here
Transparency in human-agent teaming and its effect on complacent behavior
- Date Issued:
- 2016
- Abstract/Description:
- This study examined how transparency of an intelligent agent's reasoning affected complacent behavior in a route selection task in a simulated environment. Also examined was how the information available to the operator affected those results.In two experiments, participants supervised a three-vehicle convoy as it traversed a simulated environment and re-routed the convoy when needed with the assistance of an intelligent agent, RoboLeader. Participants were randomly assigned to an Agent Reasoning Transparency condition. Participants received communications from a commander confirming either the presence or absence of activity in the area. They also received information regarding potential events along their route via icons that appeared on a map displaying the convoy route and surrounding area. Participants in Experiment 1 (low information setting) received information about their current route only; they did not receive any information about the suggested alternate route. Participants in Experiment 2 (high information setting) received information about both their current route and the agent recommended an alternative route. In the first experiment, access to agent reasoning was found to be an effective deterrent to complacent behavior when the operator has limited information about their task environment. However, the addition of information that created ambiguity for the operator encouraged complacency, resulting in reduced performance and poorer trust calibration. Agent reasoning did not increase response time or workload and appeared to have improved performance on the secondary task. These findings align with studies that have shown ambiguous information can increase workload and encourage complacency, as such, caution should be exercised when considering how transparent to make agent reasoning and what information should be included.In the second experiment, access to agent reasoning was found to have little effect on complacent behavior when the operator had complete information about the task environment. However, the addition of information that created ambiguity for the operator appeared to encourage complacency, as indicated by reduced performance and shorter decision times. Agent reasoning transparency did not increase overall workload, and operators reported higher satisfaction with their performance and reduced mental demand. Access to agent reasoning did not improve operators' secondary task performance, situation awareness, or operator trust. However, when agent reasoning transparency included ambiguous information complacent behavior was again encouraged. Unlike the first experiment, there were notable differences in complacent behavior, performance, operator trust, and situation awareness due to individual difference factors. As such, these findings would suggest that when the operator has complete information regarding their task environment, access to agent reasoning may be beneficial, but not dramatically so. However, individual difference factors will greatly influence performance outcomes. The amount of information the operator has regarding the task environment has a profound effect on the proper use of the agent. Increased environmental information resulted in more rejections of the agent recommendation regardless of the transparency of agent reasoning. The addition of agent reasoning transparency appeared to be effective at keeping the operator engaged, while complacent behavior appeared to be encouraged both when agent reasoning was either not transparent or so transparent as to become ambiguous. Even so, operators reported lower trust and usability for the agent than when environmental information was limited. Situation awareness (SA2) scores were also higher in the high information environment when agent reasoning was either not transparent or so transparent as to become ambiguous, compared to the low information environment. However, when a moderate amount of agent reasoning was available to the operator, the amount of information available to the operator had no effect on the operators' complacent behavior, subjective trust, or SA. These findings indicate that some negative outcomes resulting from the incongruous transparency of agent reasoning may be mitigated by increasing the information the operator has regarding the task environment.
Title: | Transparency in human-agent teaming and its effect on complacent behavior. |
48 views
17 downloads |
---|---|---|
Name(s): |
Wright, Julia, Author Hancock, Peter, Committee Chair Szalma, James, Committee Member Jentsch, Florian, Committee Member Chen, Jessie, Committee Member University of Central Florida, Degree Grantor |
|
Type of Resource: | text | |
Date Issued: | 2016 | |
Publisher: | University of Central Florida | |
Language(s): | English | |
Abstract/Description: | This study examined how transparency of an intelligent agent's reasoning affected complacent behavior in a route selection task in a simulated environment. Also examined was how the information available to the operator affected those results.In two experiments, participants supervised a three-vehicle convoy as it traversed a simulated environment and re-routed the convoy when needed with the assistance of an intelligent agent, RoboLeader. Participants were randomly assigned to an Agent Reasoning Transparency condition. Participants received communications from a commander confirming either the presence or absence of activity in the area. They also received information regarding potential events along their route via icons that appeared on a map displaying the convoy route and surrounding area. Participants in Experiment 1 (low information setting) received information about their current route only; they did not receive any information about the suggested alternate route. Participants in Experiment 2 (high information setting) received information about both their current route and the agent recommended an alternative route. In the first experiment, access to agent reasoning was found to be an effective deterrent to complacent behavior when the operator has limited information about their task environment. However, the addition of information that created ambiguity for the operator encouraged complacency, resulting in reduced performance and poorer trust calibration. Agent reasoning did not increase response time or workload and appeared to have improved performance on the secondary task. These findings align with studies that have shown ambiguous information can increase workload and encourage complacency, as such, caution should be exercised when considering how transparent to make agent reasoning and what information should be included.In the second experiment, access to agent reasoning was found to have little effect on complacent behavior when the operator had complete information about the task environment. However, the addition of information that created ambiguity for the operator appeared to encourage complacency, as indicated by reduced performance and shorter decision times. Agent reasoning transparency did not increase overall workload, and operators reported higher satisfaction with their performance and reduced mental demand. Access to agent reasoning did not improve operators' secondary task performance, situation awareness, or operator trust. However, when agent reasoning transparency included ambiguous information complacent behavior was again encouraged. Unlike the first experiment, there were notable differences in complacent behavior, performance, operator trust, and situation awareness due to individual difference factors. As such, these findings would suggest that when the operator has complete information regarding their task environment, access to agent reasoning may be beneficial, but not dramatically so. However, individual difference factors will greatly influence performance outcomes. The amount of information the operator has regarding the task environment has a profound effect on the proper use of the agent. Increased environmental information resulted in more rejections of the agent recommendation regardless of the transparency of agent reasoning. The addition of agent reasoning transparency appeared to be effective at keeping the operator engaged, while complacent behavior appeared to be encouraged both when agent reasoning was either not transparent or so transparent as to become ambiguous. Even so, operators reported lower trust and usability for the agent than when environmental information was limited. Situation awareness (SA2) scores were also higher in the high information environment when agent reasoning was either not transparent or so transparent as to become ambiguous, compared to the low information environment. However, when a moderate amount of agent reasoning was available to the operator, the amount of information available to the operator had no effect on the operators' complacent behavior, subjective trust, or SA. These findings indicate that some negative outcomes resulting from the incongruous transparency of agent reasoning may be mitigated by increasing the information the operator has regarding the task environment. | |
Identifier: | CFE0006422 (IID), ucf:51469 (fedora) | |
Note(s): |
2016-08-01 Ph.D. Sciences, Psychology Doctoral This record was generated from author submitted information. |
|
Subject(s): | Human-agent teaming -- human-robot interaction -- agent transparency -- complacent behavior -- decision making -- trust -- individual differences | |
Persistent Link to This Record: | http://purl.flvc.org/ucf/fd/CFE0006422 | |
Restrictions on Access: | campus 2021-08-15 | |
Host Institution: | UCF |