You are here

PERFORMANCE SUPPORT AND USABILITY:AN EXPERIMENTAL STUDY OFELECTRONIC PERFORMANCE SUPPORT INTERFACES

Download pdf | Full Screen View

Date Issued:
2005
Abstract/Description:
This study evaluated the usability of two types of performance-support interfaces that were designed using informational and experiential approaches. The experiment sought to determine whether there is a relationship between usability and the informational and experiential approaches. The general population under study was undergraduate education major students from the University of Central Florida. From the general population of three educational technology instructor-led classes, 83 students were solicited to participate in the study by completing a class activity. From the general population, a total of 63 students participated in the study. By participating in the study, the students completed a task and a questionnaire. Students were predominantly English-speaking Caucasian female education majors between the ages of 19 and 20; most of them were sophomores or juniors working part time. They possessed moderately low to high computer skills and most considered themselves to have intermediate or expert Internet skills. An experimental posttest-only comparison group research design was used to test the hypotheses posited for this study. The participants were randomly assigned to either the informational interface group (X1) or the experiential interface group (X2), and the experiment was conducted electronically via a Web-based Content Management System (CMS). The observed data consisted of five outcome measures: efficiency, errors, intuitiveness, satisfaction, and student performance. Two instruments--a checklist and an online usability questionnaire--were used to measure the five dependent variables: efficiency, intuitiveness, errors, satisfaction, and student performance. The CMS was used as the vehicle to distribute and randomize the two interfaces, obtain informed consent, distribute the instructions, distribute the online questionnaire, and collect data. First, a checklist was used to assess the students' performance completing their task, which was a copyright issue request letter. The checklist was designed as a performance criterion tool for the researcher, instructor, and participants to use. The researcher and instructor constructed the checklist to grade copyright request letters and determine students' performance. The participants had the opportunity to use the checklist as a performance criterion to create the task document (copyright request letter). The checklist consisted of ten basic yet critical sections of a successful copyright request letter. Second, an online usability questionnaire was constructed based on the Purdue Usability Testing Questionnaire (PUTQ) questions to measure interface efficiency, intuitiveness, errors, and satisfaction. While these test items have been deemed important for testing the usability of a particular system, for purposes of this study, test items were modified, deleted, and added to ensure content validity. The new survey, University of Central Florida Usability Questionnaire (UCFUQ), consisting of 20 items, was implemented in a pilot study to ensure reliability and content validity. Changes to the PUTQ were modified to fulfill a blueprint. A pilot study of the instrument yielded a reliability coefficient of .9450, and the final online usability instrument yielded a reliability coefficient of .9321. This study tested two approaches to user interface design for the Electronic Performance Support (EPS) using two HTML interface templates and the information from an existing training module. There were two interventions consisting of two interface types: informational and experiential. The SPSS Graduate Pack 10.0 for Windows was used for data analysis and statistical reporting in this study. A t test was conducted to determine if a difference existed between the two interface means. ANOVA was conducted to determine if there was an interaction between the interface group means and the demographic data factored among the five dependent variables. Results of this study indicated that students at the University of Central Florida reported no differences between the two interface types. It was postulated that the informational interface would yield a higher mean score because of its implementation of HCI guidelines, conventions, and standards. However, it was concluded that the informational interface may not be a more usable interface. Users may be as inclined to use the experiential interface as the informational interface.
Title: PERFORMANCE SUPPORT AND USABILITY:AN EXPERIMENTAL STUDY OFELECTRONIC PERFORMANCE SUPPORT INTERFACES.
0 views
0 downloads
Name(s): Rawls, Charles, Author
Hirumi, Atsusi, Committee Chair
University of Central Florida, Degree Grantor
Type of Resource: text
Date Issued: 2005
Publisher: University of Central Florida
Language(s): English
Abstract/Description: This study evaluated the usability of two types of performance-support interfaces that were designed using informational and experiential approaches. The experiment sought to determine whether there is a relationship between usability and the informational and experiential approaches. The general population under study was undergraduate education major students from the University of Central Florida. From the general population of three educational technology instructor-led classes, 83 students were solicited to participate in the study by completing a class activity. From the general population, a total of 63 students participated in the study. By participating in the study, the students completed a task and a questionnaire. Students were predominantly English-speaking Caucasian female education majors between the ages of 19 and 20; most of them were sophomores or juniors working part time. They possessed moderately low to high computer skills and most considered themselves to have intermediate or expert Internet skills. An experimental posttest-only comparison group research design was used to test the hypotheses posited for this study. The participants were randomly assigned to either the informational interface group (X1) or the experiential interface group (X2), and the experiment was conducted electronically via a Web-based Content Management System (CMS). The observed data consisted of five outcome measures: efficiency, errors, intuitiveness, satisfaction, and student performance. Two instruments--a checklist and an online usability questionnaire--were used to measure the five dependent variables: efficiency, intuitiveness, errors, satisfaction, and student performance. The CMS was used as the vehicle to distribute and randomize the two interfaces, obtain informed consent, distribute the instructions, distribute the online questionnaire, and collect data. First, a checklist was used to assess the students' performance completing their task, which was a copyright issue request letter. The checklist was designed as a performance criterion tool for the researcher, instructor, and participants to use. The researcher and instructor constructed the checklist to grade copyright request letters and determine students' performance. The participants had the opportunity to use the checklist as a performance criterion to create the task document (copyright request letter). The checklist consisted of ten basic yet critical sections of a successful copyright request letter. Second, an online usability questionnaire was constructed based on the Purdue Usability Testing Questionnaire (PUTQ) questions to measure interface efficiency, intuitiveness, errors, and satisfaction. While these test items have been deemed important for testing the usability of a particular system, for purposes of this study, test items were modified, deleted, and added to ensure content validity. The new survey, University of Central Florida Usability Questionnaire (UCFUQ), consisting of 20 items, was implemented in a pilot study to ensure reliability and content validity. Changes to the PUTQ were modified to fulfill a blueprint. A pilot study of the instrument yielded a reliability coefficient of .9450, and the final online usability instrument yielded a reliability coefficient of .9321. This study tested two approaches to user interface design for the Electronic Performance Support (EPS) using two HTML interface templates and the information from an existing training module. There were two interventions consisting of two interface types: informational and experiential. The SPSS Graduate Pack 10.0 for Windows was used for data analysis and statistical reporting in this study. A t test was conducted to determine if a difference existed between the two interface means. ANOVA was conducted to determine if there was an interaction between the interface group means and the demographic data factored among the five dependent variables. Results of this study indicated that students at the University of Central Florida reported no differences between the two interface types. It was postulated that the informational interface would yield a higher mean score because of its implementation of HCI guidelines, conventions, and standards. However, it was concluded that the informational interface may not be a more usable interface. Users may be as inclined to use the experiential interface as the informational interface.
Identifier: CFE0000807 (IID), ucf:46678 (fedora)
Note(s): 2005-12-01
Ed.D.
Education, Department of Educational Research, Technology and Leadership
Doctorate
This record was generated from author submitted information.
Subject(s): electronic performance support
usability
human computer interaction
interface design
instructional technology
Persistent Link to This Record: http://purl.flvc.org/ucf/fd/CFE0000807
Restrictions on Access: public
Host Institution: UCF

In Collections