Before discussing any analysis I think it is important to point out that the category of "Unduplicated" is a binary grouping: either you are at-risk for academic underperformance or you aren't. The ASI is fundamentally different in that it provides a score indicating the degree to which a student is likely to struggle. In that sense the ASI can be much more useful to schools in designing and assigning students to appropriate supports and interventions.
In order to compare the two I used the category of ASI 3+ vs. Unduplicated. In prior analysis students with "headwinds" scoring 3+ consistently differentiate in their performance from those scoring two or less.
- Which screen was better for identifying students who scored "Not Meeting Standards" on their English and Math Smarter Balanced Assessments? (I chose the SBA because even with its flaws it is an objective measure of student performance.)
- Which screen had lower levels of false positives? For this I looked at students in both groups who not only did they not underperform academically, but actually met the "Exceeded Standards" level on the SBA.
- Which screen had the lower level of false negatives? In other words, which screening method failed to identify students who ultimately did score in the "Not Meeting Standards" range on the SBA.
- Bottom line: Of those students who scored "Not Meeting Standards" on the SBA, which did better at identifying them in advance: ASI 3+ or Unduplicated?
- The ASI 3+ screen was better able to identify students who would underperform on both Math and English Language Arts across almost all grade levels.
- Advantage ASI
- The ASI 3+ screen was less likely to identify students as at-risk as it had lower rates of false positives, i.e. students who were identified who scored Exceeding Standards.
- Advantage ASI
- The ASI 3+ screen was less likely to omit students who would score "Not Meeting Standards" than the Unduplicated group.
- Advantage ASI
- For those students who scored "Not Meeting Standards" on math the ASI 3+ group identified 87% of them in advance. The Unduplicated identified 73%. Win for ASI 3+. For the English Language Arts assessment, ASI 3+ identified 88% vs. 76% for Unduplicated.
- Advantage ASI