Current Search: Knowledge Structures (x)
View All Items
(1 - 5 of 5)
- Title
- INVESTIGATING THE RELIABILITY AND VALIDITY OF KNOWLEDGE STRUCTURE EVALUATIONS: THE INFLUENCE OF RATER ERROR AND RATER LIMITATIONS.
- Creator
-
Harper-Sciarini, Michelle, Jentsch, Florian, University of Central Florida
- Abstract / Description
-
The likelihood of conducting safe operations increases when operators ave effectively integrated their knowledge of the operation into meaningful relationships, referred to as knowledge structures (KSs). Unlike knowing isolated facts about an operation, well integrated KSs reflect a deeper understanding. It is, however, only the isolated facts that are often evaluated in training environments. To know whether an operator has formed well integrated KSs, KS evaluation methods must be employed....
Show moreThe likelihood of conducting safe operations increases when operators ave effectively integrated their knowledge of the operation into meaningful relationships, referred to as knowledge structures (KSs). Unlike knowing isolated facts about an operation, well integrated KSs reflect a deeper understanding. It is, however, only the isolated facts that are often evaluated in training environments. To know whether an operator has formed well integrated KSs, KS evaluation methods must be employed. Many of these methods, however, require subjective, human-rated evaluations. These ratings are often prone to the negative influence of a raterÂÂÂÂÂÂ's limitations such as rater biases and cognitive limitations; therefore, the extent to which KS evaluations are beneficial is dependent on the degree to which the raterÂÂÂÂÂÂ's limitations can be mitigated. The main objective of this study was to identify factors that will mitigate rater limitations and test their influence on the reliability and validity of KS evaluations. These factors were identified through the delineation of a framework that represents how a raterÂÂÂÂÂÂ's limitations will influence the cognitive processes that occur during the evaluation process. From this framework, one factor (i.e., operation knowledge), and three mitigation techniques (i.e., frame-of-reference training, reducing the complexity of the KSs, and providing referent material) were identified. Ninety-two participants rated the accuracy of eight KSs over a period of two days. Results indicated that reliability was higher after training. Furthermore, several interactions indicated that the benefits of domain knowledge, referent material, and reduced complexity existed within subsets of the participants. For example, reduced complexity only increased reliability among evaluators with less knowledge of the operation. Also, referent material increased reliability only for those who scored less complex KSs. Both the practical and theoretical implications of these results are provided.
Show less - Date Issued
- 2010
- Identifier
- CFE0002973, ucf:47950
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0002973
- Title
- Three Studies Examining the Potential for Relational Reasoning to Enhance Expertise in Complex Audit Domains.
- Creator
-
Holt, Matthew, Sutton, Steven, Arnold, Vicky, Roberts, Robin, Dillard, Jesse, University of Central Florida
- Abstract / Description
-
This dissertation consists of three studies that explore the potential for relational reasoning to advance research on the facilitation of expertise in complex audit domains. Study One seeks to explicate the potential that theory and methods from relational reasoning and associated research have to advance the audit expertise research stream. The implications for future research on facilitating auditing expertise are discussed in synchrony with future research questions, including whether or...
Show moreThis dissertation consists of three studies that explore the potential for relational reasoning to advance research on the facilitation of expertise in complex audit domains. Study One seeks to explicate the potential that theory and methods from relational reasoning and associated research have to advance the audit expertise research stream. The implications for future research on facilitating auditing expertise are discussed in synchrony with future research questions, including whether or not such strategies will be effective in domains with more than minor relational complexity. Studies Two and Three experimentally examine the use of metacognitive skills intended to enhance relational knowledge, which is considered to be a fundamental component of domain expertise. Study Two investigates the effects of alternate forms of prompting for analogical comparison and Study Three explores the impact of combining analogical comparison with direct instruction on discerning the relational structure of a domain. The results of Study Two do not support the expected positive effects of the analogical comparison interventions. Implementation of effective interventions to prompt the comparison requires further research. Additionally, the results of Study Three do not support the hypotheses, by conventional standards. However, there is some evidence of positive effects associated with the analogical comparison intervention. This dissertation contributes to the literature on audit expertise by describing how relational reasoning can play a role in advancing research in this stream and by providing some preliminary information regarding the effectiveness of specific implementations aimed at enhancing relational knowledge.
Show less - Date Issued
- 2018
- Identifier
- CFE0007192, ucf:52265
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0007192
- Title
- IMPROVING METACOMPREHENSION AND LEARNING THROUGH GRADUATED CONCEPT MODEL DEVELOPMENT.
- Creator
-
Kring, Eleni, Salas, Eduardo, University of Central Florida
- Abstract / Description
-
Mental model development, deeper levels of information processing, and elaboration are critical to learning. More so, individuals' metacomprehension accuracy is integral to making improvements to their knowledge base. In other words, without an accurate perception of their knowledge on a topic, learners may not know that knowledge gaps or misperceptions exist and, thus, would be less likely to correct them. Therefore, this study offered a dual-process approach that aimed at enhancing...
Show moreMental model development, deeper levels of information processing, and elaboration are critical to learning. More so, individuals' metacomprehension accuracy is integral to making improvements to their knowledge base. In other words, without an accurate perception of their knowledge on a topic, learners may not know that knowledge gaps or misperceptions exist and, thus, would be less likely to correct them. Therefore, this study offered a dual-process approach that aimed at enhancing metacomprehension. One path aimed at advancing knowledge structure development and, thus, mental model development. The other focused on promoting a deeper level of information processing through processes like elaboration. It was predicted that this iterative approach would culminate in improved metacomprehension and increased learning. Accordingly, using the Graduated Concept Model Development (GCMD) approach, the role of learner-generated concept model development in facilitating metacomprehension and knowledge acquisition was examined. Concept maps have had many roles in the learning process as mental model assessment tools and advanced organizers. However, this study examined the process of concept model building as an effective training tool. Whereas, concept maps functioning as advanced organizers are certainly beneficial, it would seem that the benefits of having a learner examine and amend the current state of their knowledge through concept model development would prove more effective for learning. In other words, learners looking at an advanced organizer of the training material may feel assured that they have a thorough understanding of it. Only when they are forced to create a representation of the material would the gaps and misperceptions in their knowledge base likely be revealed. In short, advanced organizers seem to rely on recognition, where concept model development likely requires recalling and understanding 'how' and 'why' the interrelationships between concepts exist. Therefore, the Graduated Concept Model Development (GCMD) technique offered in this study was based on the theory that knowledge acquisition improves when learners integrate new information into existing knowledge, assign elaborated meanings to concepts, correct misperceptions, close knowledge gaps, and strengthen accurate connections between concepts by posing targeted questions against their existing knowledge structures. This study placed an emphasis on meaningful learning and suggested a process by which newly introduced concepts would be manipulated for the purpose of improving metacomprehension by strengthening accurate knowledge structures and mental model development, and through deeper and elaborated information processing. Indeed, central to improving knowledge deficiencies and misunderstandings is metacomprehension, and the constructing of concepts maps was hypothesized to improve metacomprehension accuracy and, thus, learning. This study was a one-factor between-groups design with concept map type as the independent variable, manipulated at four levels: no concept map, concept map as advanced organizer, learner-built concept map with feedback, and learner-built concept map without feedback. The dependent variables included performance (percent correct) on a declarative and integrative knowledge assessment, mental model development, and metacomprehension accuracy. Participants were 68 (34 female, 34 male, ages 18-35, mean age = 21.43) undergraduate students from a major southeastern university. Participants were randomly assigned to one of the four experimental conditions, and analysis revealed no significant differences between the groups. Upon arrival, participants were randomly assigned to one of the four experimental conditions. Participants then progressed through the three stages of the experiment. In Stage I, participants completed forms regarding informed consent, general biographical information, and task self-efficacy. In Stage II, participants completed the self-paced tutorial based on the Distributed Dynamic Decision Making (DDD) model, a simulated military command and control environment aimed at creating events to encourage team coordination and performance (for a detailed description, see Kleinman & Serfaty, 1989). The manner by which participants worked through the tutorial was determined by their assigned concept map condition. Upon finishing each module of the tutorial, participants then completed a metacomprehension prediction question. In Stage III, participants completed the computer-based knowledge assessment test, covering both declarative and integrative knowledge, followed by the metacomprehension postdiction question. Participants then completed the card sort task, as the assessment of mental model development. Finally, participants completed a general study survey and were debriefed as to the purpose of the study. The entire experiment lasted approximately 2 to 3 hours. Results indicated that the GCMD condition showed a stronger indication of metacomprehension accuracy, via prediction measures, compared with the other three conditions (control, advanced organizer, and feedback), and, specifically, significantly higher correlations than the other three conditions in declarative knowledge. Self-efficacy measures also indicated that the higher metacomprehension accuracy correlation observed in the GCMD condition was likely the result of the intervention, and not due to differences in self-efficacy in that group of participants. Likewise, the feedback and GCMD conditions led to significantly high correlations for metacomprehension accuracy based on levels of understanding on the declarative knowledge tutorial module (Module 1). The feedback condition also showed similar responses for the integrative knowledge module (Module 2). The advanced organizer, feedback, and GCMD conditions were also found to have significantly high correlation of self-reported postdiction of performance on the knowledge assessment and the actual results of the knowledge assessment results. However, results also indicated that there were no significant findings between the four conditions in mental model assessment and knowledge assessment. Nevertheless, results support the relevance of accurate mental model development in knowledge assessment outcomes. Retrospectively, two opposing factors may have complicated efforts to detect additional differences between groups. From one side, the experimental measures may not have been rigorous enough to filter out the effect from the intervention itself. Conversely, software usability issues and the resulting limitations in experimental design may have worked negatively against the two concept mapping conditions and, inadvertently, suppressed effects of the intervention. Future research in the GCMD approach will likely review cognitive workload, concept mapping software design, and the sensitivity of the measures involved.
Show less - Date Issued
- 2004
- Identifier
- CFE0000312, ucf:46311
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0000312
- Title
- Identifying Type of Expertise as a Means to Measure CRM Knowledge Structures.
- Creator
-
Jimenez, Camilo, Jentsch, Florian, Shumaker, Randall, Burke, Shawn, Sims, Valerie, University of Central Florida
- Abstract / Description
-
Crew Resource Management (CRM) training has evolved since its inception in the 1980s to better accommodate the operational needs of flight crews. However, even as the aviation and research communities have pointed to the potential benefit of providing CRM training, some criticism continues to emerge periodically which claims that there is no concrete evidence of its impact on flight deck performance and safety. Therefore, it is imperative to develop tools that allow researchers and, more...
Show moreCrew Resource Management (CRM) training has evolved since its inception in the 1980s to better accommodate the operational needs of flight crews. However, even as the aviation and research communities have pointed to the potential benefit of providing CRM training, some criticism continues to emerge periodically which claims that there is no concrete evidence of its impact on flight deck performance and safety. Therefore, it is imperative to develop tools that allow researchers and, more importantly, practitioners, to more effectively and objectively assess training effectiveness and identify whether or not desired CRM behaviors are being put to practice during line operations.This study focused on evaluating pilots' CRM schemas and identifying differences in CRM knowledge structures among pilots. Differences in CRM knowledge and opinions about training could be an indication of the existence of what Hatano and Inagaki (1986) have described as two distinct types of expertise, namely, routine and adaptive expertise. The study sought to identify differences among routine and adaptive expert pilots in CRM knowledge structures (schemas), their perceptions on the value and efficacy of current CRM training evaluation, along with their opinion on how CRM training effectiveness could be more accurately assessed. Results from over 250 pilots showed that, in general, participants had a positive view of CRM training and training evaluation, regardless of their type of expertise. Some evidence of potential differences in the structural knowledge of CRM between routine and adaptive experts, as well as, differences in their opinions about CRM training, evaluation, and automation on the flight deck was also identified. Additionally, analysis of survey scores and free response items indicate the existence of a third category of experts, between routine and adaptive expertise (whom I call transitional experts). The study results provide evidence that assessment of CRM schemas could potentially be used as a way to evaluate CRM training effectiveness. The results of the study also indicate that identification of specific training needs for each group of expert may be possible through the assessment of CRM schemas and type of expertise. Implications for practice and theory, limitations of the study, and suggestions for future research are also provided.
Show less - Date Issued
- 2018
- Identifier
- CFE0007200, ucf:52264
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0007200
- Title
- The Relationship Between Comprehension of Descriptive and Sequential Expository Texts and Reader Characteristics in Typically Developing Kindergarten Children.
- Creator
-
Zadroga, Cheran, Schwartz, Jamie, Kent-Walsh, Jennifer, Nye, Chad, Lieberman, Rita, Hahs-Vaughn, Debbie, University of Central Florida
- Abstract / Description
-
Researchers have found that children need to be proficient in reading and writing expository text to succeed academically as well as in their future careers. More than ever before, children in primary grade classrooms are being exposed to and expected to comprehend a variety expository text types. However, empirical evidence to support the use of expository texts in kindergarten classrooms, in particular, is sorely lacking. To begin to fill this gap, this study was conducted to investigate...
Show moreResearchers have found that children need to be proficient in reading and writing expository text to succeed academically as well as in their future careers. More than ever before, children in primary grade classrooms are being exposed to and expected to comprehend a variety expository text types. However, empirical evidence to support the use of expository texts in kindergarten classrooms, in particular, is sorely lacking. To begin to fill this gap, this study was conducted to investigate kindergarten children's comprehension of two types of expository text structures (i.e., descriptive and sequential) commonly found in kindergarten classrooms. Specifically, the aims of the study were three fold: (1) to investigate if there is a relationship between prior knowledge and the comprehension of descriptive or sequential expository text; (2) to determine if the comprehension of descriptive and sequential expository text are important predictors of performance on the Token Test for Children-2 (TTFC-2) and the Assessment of Literacy and Language (ALL); and (3) to determine if there is a correlation between the descriptive and sequential expository text comprehension measures (i.e., retelling of expository text and answering comprehension questions) on the researcher created Expository Text Protocol.The sample included 45 typically developing kindergarten children (ages 5 years, 8 months to 6 years, 10 months). All children passed a vision and a hearing screening; were enrolled in kindergarten for the first time (no history of retention); scored within the normal range on a non-verbal intelligence screener; and, were not receiving services in the English for Speakers of Other Languages (ESOL) program or the Exceptional Student Education (ESE) program. Each child participated in two, one-hour, assessment sessions on two separate days. During the sessions, children were administered formal (i.e., TTFC-2 (&) ALL) and informal (i.e., Expository Text Protocol) assessments, counter balanced across the sessions. The standardized tests were administered in the prescribed manner. During administration of the researcher created Expository Text Protocol children listened first to either an illustrated descriptive expository text or an illustrated sequential expository text read aloud by a researcher. After the reading, the children either first retold the text without the use of the corresponding expository text or answered a set of 12 comprehension questions for each type of expository text (i.e., descriptive and sequential). The order of the retelling and comprehension questions were counter balanced across children. Simple linear regressions, multiple linear regressions, and partial correlational analyses were used to assess the data obtained in this study. The research findings indicated that a statistically significant relationship exists between the comprehension of expository text and the following reader characteristics: listening comprehension ability, language ability, and literacy ability. However, a statistically significant relationship was not found between the comprehension of the expository text types and prior knowledge. In addition, a statistically significant relationship was found between each of the two types of comprehension measures: retelling of descriptive and sequential expository texts and answering comprehension questions related to each type of text.This investigation revealed that the incorporation of descriptive and sequential expository text structures into the kindergarten curricula is appropriate and the exposure to expository texts may facilitate language and literacy growth and build upon kindergarten children's existing prior knowledge. In turn, exposure to expository texts also may be beneficial in expanding children's use of expository language found in these types of texts. Future research is needed to examine kindergarten children's comprehension of other types of expository text structures found in kindergarten classrooms.
Show less - Date Issued
- 2016
- Identifier
- CFE0006426, ucf:51479
- Format
- Document (PDF)
- PURL
- http://purl.flvc.org/ucf/fd/CFE0006426
