top of page


A Vision for Using an Argument-Based Framework for Validity Applied to a Comprehensive System of Assessments for English Learners in Secondary Grades

By Margaret Heritage, Caroline Wylie, Molly Faulkner-Bond, and Aída Walqui

INTRODUCTION   |   PART 1   |   PART 2   |   PART 3    |   PART 4    |   PART 5    |   PART 6    |   PART 7    |   PART 8    |   PART 9   |   REFERENCES    |   APPENDICES

Part 8: Propositions, Claims, and Evidence for the Assessments in the CAS Framework


d. Year-Long Student Portfolio


The year-long student portfolio has four purposes: (1) monitor learning over the course of the year; (2) evaluate achievement at the end of the year; (3) support students to engage in metacognition, reflecting on their own learning path; and (4) inform localized high-stakes decisions (e.g., engagement in rigorous grade-level learning, access to the core curriculum and demanding learning opportunities, classification and reclassification) about individual students when teacher-based judgments about individual student’s strengths and weaknesses differ from performance on other assessments, including those for the purpose of accountability. The student portfolio logic model is shown in Figure 8.


Figure 8. Logic Model for Year-Long Student Portfolio

Figure 8. Logic Model for Year-Long Student Portfolio

Proposition 1: The portfolio includes a selection of items that are highlights of curricular outcomes and of what the student(s) knows and can do.

CAS Year-Long Portfolio Proposition 1

Proposition 2: Learning and progress is represented in a variety of ways. 

CAS Year-Long Portfolio Proposition 2

Proposition 3: The portfolio includes students’ self-assessments/evaluative reflections

CAS Year-Long Portfolio Proposition 3

Proposition 4: The portfolio can support teacher judgments and student metacognition about progress in order to evaluate learning.

CAS Year-Long Portfolio Proposition 4

Support for COP Review of Validity Evidence


To support COP discussions of the validity evidence, two protocols are included in Appendix B and Appendix C. The protocols provide guidance on:


  • how to analyze propositions and claims to help ensure a common understanding among teachers

  • what a teacher would present to the COP

  • what the members of the COP should listen for

  • what questions the members of the COP would ask

  • how to plan outcomes they are collectively moving towards


A process for using the protocols is shown in Figure 9.


Figure 9. Process for Using Protocols


Figure 9. Process for Using Protocols

Proposition Specific Protocols: The first protocol guides COP members through three stages of reviewing and evaluating evidence for a specific proposition: (1) the Initial Planning stage will identify the targeted proposition, ensure common understanding, and identify the types of relevant evidence that may need to be collected for the evaluation of the validity argument; (2) the validity review is the core work of the COP to review evidence for propositions’ claims and identify strengths and areas to improve; (3) during the Action Review COP members examine whether recommended changes were made or to reflect on how the discussion from the Validity Review impacted ongoing assessment development.


The COP members may decide to apply this protocol initially to some, none or all of the propositions associated with a specific type of assessment, depending on their familiarity with each of the propositions for the assessment on which they are focused. For example, the COP members may decide that they first wanted to better understand what it might mean to develop assessments that allow students to use multiple modalities. The group could use the protocol for Classroom Summative Proposition 2 and bring examples of assessments that either use multiple modalities, or that members think could be modified to incorporate multiple modalities. These discussions can help teachers to develop assessment literacy in collaborative ways. Other aspects of shared assessments are not discussed at this stage because the group elected to focus on a specific aspect.


Assessment Specific Protocols: Once the COP members have developed a level of familiarity with each proposition and its claims and evidence, they can use the protocol for the particular assessment (e.g., classroom summative). This protocol addresses all five propositions and supports a discussion about the chain of evidence that starts with the unit goals and ends with how the summative unit assessment will be used. This protocol will support COP discussions as the members articulate the chain of evidence for a specific assessment. The COP may need to take some time to decide what of the possible forms of evidence they are going to collect. For instance, will they survey students? Will they engage in peer teacher review? Careful planning will be needed prior to collecting evidence for review.


When the evidence has been amassed, the purpose of the discussion is to identify places where the evidence might not support a specific claim, or where an entire proposition cannot be supported. The COP members can then decide either to discuss how to improve the assessment, or decide when that work could be done. It is important to capture sufficiently detailed notes about the decisions that could assist future revisions if they are not going to be made immediately.


COP Discussions Using Both Types of Protocols: The entire COP discussion is intended to be formative: the logic model and propositions describe the ideal nature and conditions of an assessment. The evidence describes the current status of the assessment, and the COP discussions examine the gap between intentions and reality. Lessons learned from a review of one assessment can be applied to future assessments. The COP members might decide to focus their efforts on revising an upcoming assessment and then examine it in terms of the validity evidence to determine whether the revisions helped them get closer to the ideal model as described by the propositions. The COP might also decide to return to the elaborated protocol because they identified a specific proposition as an area of weakness across their assessments and want to work on that area in a targeted way.


It is important to note that it would not be practical or even advisable that a COP engage in this level of review for every classroom assessment. An in-depth review of one or two per semester may be sufficient, if individual and COP time is also spent on revising other assessments that may have the same weaknesses.

    The CAS will be partially implemented over three years in an iterative study of secondary-level teachers’ use of high-quality replacement units. This study will be an opportunity to try out and refine both protocols, create additional resources for teachers as needed, and potentially collect video of COP discussions to support teacher learning about these processes.


bottom of page