Background...

All students enter school with a combination of "headwinds" and "tailwinds". Tailwinds are the things that make school easier for students. Tailwinds may include factors such as coming from a home with parents of high education levels and economic stability, being a native English speaker, not having a disability, or being a member of the cultural majority. Each of those characteristics plays a role in helping a student experience success in school.

Headwinds on the other hand make school more difficult. Headwinds can include having economic instability at home, parents with lower levels of education, having a disability, or still learning English. The more headwinds a student has, the more difficulty they will have in maximizing their academic potential and the more “tailwinds” they will need. Tailwinds come in the form of high-quality instruction, support, and intervention.

The Academic Support Index, or ASI, quantifies these headwinds. A student’s ASI is the sum of their headwinds. Their ASI can also be considered a measure of the amount of support that they will need in order to mitigate the impact of those educational headwinds. Students with a low ASI will likely need very little additional support outside of Tier 1 instruction. Higher ASI students will likely need proportionally higher amounts of Tier 2 and sometimes Tier 3 supports.

There is a strong relationship between the ASI and academic outcomes including assessments such as the SAT, Smarter Balanced Assessments, AP and IB tests, kindergarten screeners, grade point averages, rates of college eligibility, matriculation, and degree attainment. We have studied these effects over seven years of data as well as across urban, suburban, and rural schools. To date over 400,000 students have been scored on the ASI. (See the featured post below for a list of papers and presentations on the ASI).

Because the ASI is able to reliably predict student outcomes you have to opportunity to interrupt that predictability by using the ASI to make sure that you are identifying the right students for early intervention and support. With effective intervention, predictive analytics can become preventive analytics.

Monday, July 20, 2015

Boosting Test Performance for Academically At-Risk Students and Interrupting the Predictability of Student Outcomes

The California High School Exit Exam is a critical gatekeeper for students who struggle academically. Failing to pass the CAHSEE during the initial administration in the tenth grade has significant impacts on students’ opportunities to prepare for post-secondary educational options. In addition to CAHSEE being a requirement for receiving a high-school diploma, failing to pass the exam often puts students into a remedial academic track limiting access to higher level English, math, and science courses. Successful completion of higher-level courses in high school is a proven and requisite measure of college preparedness.  Across the state of California, CAHSEE passing rates have been intractably linked to race throughout the history of the exam: On average, White and Asian students have higher passing rates, while African American and Hispanic/Latino students have lower passing rates. Similar patterns have been seen in our school district.
Historical CAHSEE English Language Arts Passing Rates by ASI:

Using analysis of historical performance data to guide intervention, we were able to significantly increase the first time passing rates of academically at-risk African American and Hispanic/Latino students.  A combination of screens that targeted students with an Academic Support Index of three or higher in conjunction with results from a standards-based tenth grade assessment given during the first week of school was used to identify and assign 44 students to an alternative testing environment.  While the majority of student took the exam in the school gymnasium, the intervention group tested in a setting consisting of a later start time, proctors with whom they were familiar, and an academically more homogeneous population.  This intervention resulted in higher passing rates (98% for English Language Arts and 93% for Math) than both the control (84% for ELA and 89% for Math) and the school overall (91% for ELA and 91% for Math).  The intervention passing rates for the ELA test were statistically significant versus the control.  The results can be seen on the table below. IEP refers to students with Individualized Education Plans (Special Education). ELN refers to students who are new to speaking English (less than 12 months).




Although none of the screens looked solely for students’ race, the intervention group was made up entirely of non-white students.  The higher passing rates for the intervention group increased the school-wide passing rates on the English Language Arts test for African-American students by eleven percentage points (13% over the running three year average) and six percentage points for Hispanic/Latino students over the prior year. Thus, we were able to interrupt the racialized predictability of passing rates on the CAHSEE during the 2015 census administration using the Academic Support Index.

Note how the overall correlation coefficient (r squared) dropped significantly during the 2015 year when the interventions were initiated.  This is evidence that we were able to reduce the predicability of student outcomes on this exam.

Click here to see the poster we presented at the California Educational Research Association in December of 2015.