Validating the revised Test of Spoken
English against a criterion of
Donald E. Powers, Mary A. Schedl and Susan Wilson Leung
Educational Testing Service, Princeton, NJ andFrances A.
University of California, Los Angeles, CA
A communicative competence orientation was taken to study the validity of test-
score inferences derived from the revised Test of Spoken English ( TSE ). To
implement the approach, a sample of undergraduate students, primarily native
speakers of English, provided a variety of reactions to, and judgements of, the
test responses of a sample of TSE examinees. The TSE scores of these examinees,
previously determined by of®cial TSE raters, spanned the full range of TSE score
levels. Undergraduate students were selected as `evaluators' because they, more
than most other groups, are likely to interact with TSE examinees, many of whom
become teaching assistants.
Student evaluations were captured by devising and administering a secondary
listening test ( SLT) to assess students' understanding of TSE examinees' speech,
as represented by their taped responses to tasks on the TSE. The objective was to
determine the degree to which of®cial TSE scores are predictive of listeners'
ability to understand the messages conveyed by TSE examinees. Analyses revealed
a strong association between TSE score levels and the judgements, reactions and
understanding of listeners. This ®nding applied to all TSE tasks and to nearly all
of the several different kinds of evaluations made by listeners. Along with other
information, the evidence gathered here should help the TSE program meet pro-
fessional standards for test validation. The procedures may also prove useful in
future test-development efforts as a way of determining the dif®culty of speaking
tasks ( and possibly writing tasks ).
The Test of Spoken English ( TSE) is a measure of the oral language
pro®ciency of non-native speakers of English. Its primary intended
1 ) screening graduate teaching assistants; and
Address for correspondence: Donald E. Powers, Principal Research Scientist, Educational Test-
ing Service, Mail Stop 17-R, Rosedale Road, Princeton, NJ