You are here

UCF Digital Collections ยป ETD Collection

THE CONSTRUCT VALIDITY OF A SITUATIONAL JUDGMENT TEST IN A MAXIMUM PERFORMANCE CONTEXT

Download pdf | Full Screen View

Date Issued:
2006
Abstract/Description:
A Predictor Response Process model (see Ployhart, 2006) and research findings were leveraged to formulate research questions about, and generate construct validity evidence for, a new situational judgment test (SJT) designed to measure declarative and strategic knowledge. The first question asked if SJT response instructions (i.e., 'Should Do', 'Would Do') moderated the validity of an SJT in a maximum performance context. The second question asked what the upper-bound criterion-related validity coefficient is for SJTs in talent selection contexts in which typical performance is the criterion of interest. The third question asked whether the SJT used in the present study was fair for gender and ethnic-based subgroups according to Cleary's (1968) definition of test fairness. Participants were randomly assigned to complete an SJT with either 'Should Do' or 'Would Do' response instructions and their maximum decision making performance outcomes were captured during a moderate fidelity poker simulation. The findings of this study suggested knowledge, as measured by the SJT, interacted with response instructions when predicting aggregate and average performance outcomes such that the 'Should Do' SJT had stronger criterion-related validity coefficients than the 'Would Do' version. The findings also suggested the uncorrected upper-bound criterion-related validity coefficient for SJTs in selection contexts is at least moderate to strong (β = .478). Moreover, the SJT was fair according to Cleary's definition of test fairness. The implications of these findings are discussed.
Title: THE CONSTRUCT VALIDITY OF A SITUATIONAL JUDGMENT TEST IN A MAXIMUM PERFORMANCE CONTEXT.
20 views
8 downloads
Name(s): Stagl, Kevin, Author
Salas, Eduardo , Committee Chair
University of Central Florida, Degree Grantor
Type of Resource: text
Date Issued: 2006
Publisher: University of Central Florida
Language(s): English
Abstract/Description: A Predictor Response Process model (see Ployhart, 2006) and research findings were leveraged to formulate research questions about, and generate construct validity evidence for, a new situational judgment test (SJT) designed to measure declarative and strategic knowledge. The first question asked if SJT response instructions (i.e., 'Should Do', 'Would Do') moderated the validity of an SJT in a maximum performance context. The second question asked what the upper-bound criterion-related validity coefficient is for SJTs in talent selection contexts in which typical performance is the criterion of interest. The third question asked whether the SJT used in the present study was fair for gender and ethnic-based subgroups according to Cleary's (1968) definition of test fairness. Participants were randomly assigned to complete an SJT with either 'Should Do' or 'Would Do' response instructions and their maximum decision making performance outcomes were captured during a moderate fidelity poker simulation. The findings of this study suggested knowledge, as measured by the SJT, interacted with response instructions when predicting aggregate and average performance outcomes such that the 'Should Do' SJT had stronger criterion-related validity coefficients than the 'Would Do' version. The findings also suggested the uncorrected upper-bound criterion-related validity coefficient for SJTs in selection contexts is at least moderate to strong (β = .478). Moreover, the SJT was fair according to Cleary's definition of test fairness. The implications of these findings are discussed.
Identifier: CFE0001309 (IID), ucf:47020 (fedora)
Note(s): 2006-08-01
Ph.D.
Sciences, Department of Psychology
Doctorate
This record was generated from author submitted information.
Subject(s): Situational Judgment Tests
Response Instructions
Maximum Performance
Persistent Link to This Record: http://purl.flvc.org/ucf/fd/CFE0001309
Restrictions on Access: public
Host Institution: UCF

In Collections