A Vision for Using an Argument-Based Framework for Validity Applied to a Comprehensive System of Assessments for English Learners in Secondary Grades
By Margaret Heritage, Caroline Wylie, Molly Faulkner-Bond, and Aída Walqui
INTRODUCTION | PART 1 | PART 2 | PART 3 | PART 4 | PART 5 | PART 6 | PART 7 | PART 8 | PART 9 | REFERENCES | APPENDICES
Part 8: Propositions, Claims, and Evidence for the Assessments in the CAS Framework
e. External Large-Scale Summative Assessments
The primary purpose that external large-scale assessments serve is for accountability in relation to federal and state policy. Others have already written extensively on how to apply an argument-based approach to validity to large-scale summative assessments, generally (e.g., Bachman, 2005; Kane, 2013); we do not attempt to duplicate those efforts here. Instead, this section focuses on aspects of validation that are unique and central to the particular role and goal of large-scale summative assessment in the context of this CAS for English Learners in secondary grades.
One such aspect is the centrality of integrated language and content within the CAS Framework. Whereas teachers have the control to use this integrated approach in designing their own classroom-based assessment opportunities, large-scale summative assessments are currently segregated along these lines – they measure either language or content, but not both. We
therefore include propositions, claims, and evidence that would be necessary to demonstrate assessment validity for this purpose. Neither the process nor the types of evidence that would be required are different from current practice on the surface – that is, the same kinds of alignment studies and standard setting panels would be required – but the construct of measurement would need to be adjusted to reflect the language and content integration.
A second important difference in our vision is the relationship of large-scale summative assessments to learning, and to teacher-generated assessments elsewhere in the system.
In the CAS Framework, these large-scale assessments are the only type of assessment that is external to the classroom. The responsibility for validating interpretations and uses for standardized assessments falls to assessment developers and test score users. Teacher-generated validity evidence from other assessment forms can serve as external criteria against which the validity of large-scale scores may be interpreted or compared. Figure 10 shows the logic model for large-scale summative assessment.
Figure 10: Logic Model for Large-Scale Summative Assessment
Propositions, claims and sources of evidence associated with this logic model are described next.
Proposition 1: The assessment is aligned to lesson-sized learning goals in a broader trajectory of learning that integrate the development of key conceptual understandings, analytic practices and the language needed to express them.
Proposition 2: The assessment is designed to reflect high-quality learning experiences that integrate academic content and language.
Proposition 3: Student scores accurately reflect student achievement with respect to grade-level standards.
Proposition 4: The assessment can appropriately be used by system-level decision-makers (e.g., SEA and LEA leaders) as an external accountability check and by teachers and school leaders to evaluate the consistency between results on classroom-based assessments and the external accountability assessment.
