The present study examined the effects of lingual background and item format on the scores of students whose vernacular is not English (i.e. secondary-English speakers) within the context of a 24-item verbal ability test written in English. Two forms of the test were developed to contain 12 multiple-choice items and 12 constructed response items each; the sequence was alternated across forms. The construction items were developed by deleting the options of 12 multiple-choice items.
A two-way mixed ANOVA, with format as the within subjects factor and lingual background as the between subjects factor, was used to determine whether or not a significant interactive effect exists between students’ lingual background and item format. A confirmatory factor analysis was used to determine whether or not secondary-English speakers interpret multiple-choice and construction questions the same way. A two-group, three-factor confirmatory factor analysis was used to determine whether or not secondary-English speakers interpret construction items the same way that primary-English speakers do. Conventional wisdom suggests that the combined intellectual and language skills required to comprehend and work constructed-
response items under time pressure may impact the scores of secondary-English speakers to a larger extent than the combined intellectual and language skills required to comprehend and work multiple-choice items under the same time constraints. Contrary to what was expected, lingual background and format did not collectively affect the performance of secondary-English speakers. Item format had no significant effect on the test performance of secondary-English speakers; only lingual background was found to impact the performance of these students on both formats. The majority of the secondary-English speakers used in this study were students born and schooled in Puerto Rico, where the constructed-response format is traditionally used to test students throughout their schooling. This study confirmed the findings of Cooley & Leinhart (as cited in O’Leary, 2001) and O’Leary (2001), that frequent exposure to a test format will make a difference to performance.
