A Vision for Using an Argument-Based Framework for Validity Applied to a Comprehensive System of Assessments for English Learners in Secondary Grades
By Margaret Heritage, Caroline Wylie, Molly Faulkner-Bond, and Aída Walqui
INTRODUCTION | PART 1 | PART 2 | PART 3 | PART 4 | PART 5 | PART 6 | PART 7 | PART 8 | PART 9 | REFERENCES | APPENDICES
Part 6: An Argument-Based Approach to Validity
The validation approach for the assessments in the CAS Framework draws on Kane’s (2006, 2013) structure for an argument-based approach to validity in order to delineate the evidence that needs to be accumulated and evaluated for valid interpretations and uses of the information yielded by the assessments. The CAS Framework provides users with guidance about the evidence on which the soundness of interpretations, decisions, and actions to enhance learning for secondary-grade English Learners can be judged. While this validity approach has relevance for classroom assessment in general, because of its emphasis on the assessment of integrated content and language development, it has particular application for English Learners.
An argument-based approach to validity involves two arguments: 1) an interpretation and use argument (IUA) and 2) the validity argument. An IUA specifies the proposed interpretations and uses of assessment results by laying out the network of inferences and assumptions leading from the observed performances to the conclusions and decisions based on the performances (Kane, 2013). It is useful to think of an IUA in terms of a logic model or if/then propositions that articulate the means for reaching the intended purpose (Perie & Forte, 2011). Figure 4 shows an example of a logic model for a classroom assessment of academic content knowledge and language.
Figure 4. Logic Model for Classroom Assessment of Academic Content Knowledge and Language
Building the Arguments
The first step in an argument-based approach is to specify the propositions of the IUA. A proposition might be, for example, “the assessment is aligned to learning goals.” The next step is to establish the claims that support each proposition. The claims are the fundamental criteria for appraising the extent to which each proposition is supported with specific evidence (Herman et al., 2011). The combination of the logic model, propositions, and claims form the IUA. Evidence is marshalled for each of the claims in the IUA. The evaluation of this evidence then forms the validity argument. In practice, as it is often not feasible to collect evidence for every claim at once (or at all), users should prioritize the accumulation of evidence for those claims in the IUA that are most ambitious or consequential (Kane, 2013), for example, portfolio scores to determine course placement in the next school year.
Proposition: The argument for the use of assessment for specific purposes comprises a series of propositions or hypothetical statements that link the performance on the assessment to specific interpretations of the meaning of the information yielded and the conclusions or decisions made on the basis of performance.
Claim: The fundamental criterion for appraising the extent to which each proposition is supported and needs to be substantiated with specific evidence.
To download a brief overview of An Argument-Based Approach to Validity click here.