View All Items
(41 - 42 of 42)
Pages
- Title
- Transparency in human-agent teaming and its effect on complacent behavior.
- Creator
-
Wright, Julia, Hancock, Peter, Szalma, James, Jentsch, Florian, Chen, Jessie, University of Central Florida
- Abstract / Description
-
This study examined how transparency of an intelligent agent's reasoning affected complacent behavior in a route selection task in a simulated environment. Also examined was how the information available to the operator affected those results.In two experiments, participants supervised a three-vehicle convoy as it traversed a simulated environment and re-routed the convoy when needed with the assistance of an intelligent agent, RoboLeader. Participants were randomly assigned to an Agent...
Show moreThis study examined how transparency of an intelligent agent's reasoning affected complacent behavior in a route selection task in a simulated environment. Also examined was how the information available to the operator affected those results.In two experiments, participants supervised a three-vehicle convoy as it traversed a simulated environment and re-routed the convoy when needed with the assistance of an intelligent agent, RoboLeader. Participants were randomly assigned to an Agent Reasoning Transparency condition. Participants received communications from a commander confirming either the presence or absence of activity in the area. They also received information regarding potential events along their route via icons that appeared on a map displaying the convoy route and surrounding area. Participants in Experiment 1 (low information setting) received information about their current route only; they did not receive any information about the suggested alternate route. Participants in Experiment 2 (high information setting) received information about both their current route and the agent recommended an alternative route. In the first experiment, access to agent reasoning was found to be an effective deterrent to complacent behavior when the operator has limited information about their task environment. However, the addition of information that created ambiguity for the operator encouraged complacency, resulting in reduced performance and poorer trust calibration. Agent reasoning did not increase response time or workload and appeared to have improved performance on the secondary task. These findings align with studies that have shown ambiguous information can increase workload and encourage complacency, as such, caution should be exercised when considering how transparent to make agent reasoning and what information should be included.In the second experiment, access to agent reasoning was found to have little effect on complacent behavior when the operator had complete information about the task environment. However, the addition of information that created ambiguity for the operator appeared to encourage complacency, as indicated by reduced performance and shorter decision times. Agent reasoning transparency did not increase overall workload, and operators reported higher satisfaction with their performance and reduced mental demand. Access to agent reasoning did not improve operators' secondary task performance, situation awareness, or operator trust. However, when agent reasoning transparency included ambiguous information complacent behavior was again encouraged. Unlike the first experiment, there were notable differences in complacent behavior, performance, operator trust, and situation awareness due to individual difference factors. As such, these findings would suggest that when the operator has complete information regarding their task environment, access to agent reasoning may be beneficial, but not dramatically so. However, individual difference factors will greatly influence performance outcomes. The amount of information the operator has regarding the task environment has a profound effect on the proper use of the agent. Increased environmental information resulted in more rejections of the agent recommendation regardless of the transparency of agent reasoning. The addition of agent reasoning transparency appeared to be effective at keeping the operator engaged, while complacent behavior appeared to be encouraged both when agent reasoning was either not transparent or so transparent as to become ambiguous. Even so, operators reported lower trust and usability for the agent than when environmental information was limited. Situation awareness (SA2) scores were also higher in the high information environment when agent reasoning was either not transparent or so transparent as to become ambiguous, compared to the low information environment. However, when a moderate amount of agent reasoning was available to the operator, the amount of information available to the operator had no effect on the operators' complacent behavior, subjective trust, or SA. These findings indicate that some negative outcomes resulting from the incongruous transparency of agent reasoning may be mitigated by increasing the information the operator has regarding the task environment.
Show less - Date Issued
- 2016
- Identifier
- CFE0006422, ucf:51469
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0006422
- Title
- THE RELATIONSHIP OF 10TH-GRADE DISTRICT PROGRESS MONITORING ASSESSMENT SCORES TO FLORIDA COMPREHENSIVE ASSESSMENT TEST SCORES IN READING AND MATHEMATICS FOR 2008-2009.
- Creator
-
Underwood, Marilyn, Taylor, Rosemarye, University of Central Florida
- Abstract / Description
-
The focus of this research was to investigate the use of a district created formative benchmark assessment in reading to predict student achievement for 10th-grade students on the Florida Comprehensive Assessment Test (FCAT) in one county in north central Florida. The purpose of the study was to provide information to high school principals and teachers to better understand how students were performing and learning and to maximize use of the formative district benchmark assessment in order to...
Show moreThe focus of this research was to investigate the use of a district created formative benchmark assessment in reading to predict student achievement for 10th-grade students on the Florida Comprehensive Assessment Test (FCAT) in one county in north central Florida. The purpose of the study was to provide information to high school principals and teachers to better understand how students were performing and learning and to maximize use of the formative district benchmark assessment in order to modify instruction and positively impact student achievement. This study expanded a prior limited study which correlated district benchmark assessment scores to FCAT scores for students in grades three through five in five elementary schools in the targeted county. The high correlations suggested further study. This research focused on secondary reading, specifically in 10th grade where both state and targeted county FCAT scores were low in years preceding this research. Investigated were (a) the district formative assessment in reading as a predictor of FCAT Reading scores, (b) differences in strength of correlation and prediction among student subgroups and between high schools, and (c) any relationships between reading formative assessment scores and Mathematics FCAT scores. An additional focus of this study was to determine best leadership practices in schools where there were the highest correlations between the formative assessment and FCAT Reading scores. Research on best practices was reviewed, and principals were interviewed to determine trends and themes in practice. Tenth grade students in the seven Florida targeted district high schools were included in the study. The findings of the study supported the effective use of formative assessments both in instruction and as predictors of studentsÂÂÂÂÂÂ' performance on the FCAT. The results of the study also showed a significant correlation between performance on the reading formative assessment and performance on FCAT Mathematics. The data indicated no significant differences in the strength of correlation between student subgroups or between the high schools included in the study. Additionally, the practices of effective principals in using formative assessment data to inform instruction, gathered through personal interviews, were documented and described.
Show less - Date Issued
- 2010
- Identifier
- CFE0003214, ucf:48573
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0003214
