Last fall at an EdSource conference, a lawyer who was one of the speakers stated "Good data is a civil right". It made me think about the data we have and how we are using it. Its not just enough to have good data, we have to use it wisely and effectively.
One of the most valuable ways to use data in education is to better understand the students we serve. Data can help us understand who our students are, what they know, and what they have yet to learn. The better we know our students, the more effective we become in helping them succeed. We also need to pay attention to what we are doing to try to improve student outcomes. When we identify practices or programs that are making a difference, we need to share them with other educators. Likewise, when an intervention isn't getting the results we hoped for, we need to go back to the drawing board, revisit our theory of action, and either make adjustments or go a different direction. We owe it to students to make sure we are providing them with the best possible education.
This table includes some additional outcomes:
We generally look at student data first by overall school (or district) performance, then by site and/or grade, and eventually we start looking at student subgroups. I always found it frustrating to look at student data in the standard "buckets" of race/ethnicity, gender, socioeconomic status, etc. Data analyzed in these categories was of limited use as students are more than single characteristics but rather a combination of many factors; factors that all contribute to their performance. Additionally, within each bucket there can be tremendous diversity. Treating each bucket as a homogenous group can reinforce stereotypes and lead to inaccurate and harmful narratives.
The factors that we know make learning more difficult are not equally distributed across all of the buckets, especially when it comes to the buckets within race and ethnicity. In our district only 8% of white students are socioeconomcally disadvantaged compared to over 50% for black and Hispanic/Latino students. Socioeconomic status has a large influence on academic outcomes. Similar disproportionality can be found in English Learner status (majority of whom are Hispanic/Latino) as well as in Special Education (disproportionately black students). Without controlling for these disproportionate elements, comparing the results of white students against those of black or Hispanic/Latino students all but guarantees a gap in achievement.
***
In our professional settings we have students who struggle and students who are successful. Struggling students could be said to have a lot of headwinds, or things that make learning more difficult. Successful students on the other hand generally have a lot of tailwinds, or things that make learning easier.
And within our schools and classrooms we have a continuum of those students:
The research is quite clear on which factors constitute "headwinds" and correlate to lower student performance. Examples include being an English Learner, having a disability, being socioeconomically disadvantaged, or having parents with a low level of education. Sometimes these headwinds occur in isolation but that is more often the exception than the rule. English Learners are more likely to also be poorer than native speakers and special education is strongly correlated to poverty. There is a cumulative effect when students have more than one of these headwinds and the result can go from a strong breeze to a full on gale.
Tailwinds on the other hand are strongly correlated to higher student performance. Tailwinds can include factors such as having highly educated parents, financially stability, or being a member of the cultural majority. Like the headwinds, these factors rarely occur in isolation.
Quantifying the Headwinds
In an attempt to quantify these headwinds, I analyzed the relationship between the demographic "buckets" and students' cumulative grade point average (GPA). I ran the t-test for all demographic fields and their corresponding subgroups and used the following rules to quantify the relationship between the demographic sub-group and their cumulative GPA.
- 2 Points: The confidence interval overlapped with a GPA target of 2.50.
- 1 Point: The confidence interval did not overlap with a GPA target of 2.50 but was below the confidence interval of other demographic fields.
Here are two examples:
The table below summarizes the results of this analysis where "headwind" points were assigned:
Now I have a way to quantify those headwinds for each and every student. Here are a couple of examples of how those calculations might work:
- Student A is an English Learner (2 points), male (1 point) , and his parents did not graduate from high school (2 points). His total headwind score is 5 points.
- Student B is a white (0 points), female (0 points), Hispanic (1 point), Initially Fluent English Speaker, whose parents have graduate degrees: Her total headwinds is 1 point.
- Student C is an African American male (2 points), with a disability (2 points), poor (1 point), whose parents took some college classes but didn't earn a degree (1 point): His total headwinds are 6 points.
So now we are able to put some numbers along that continuum of tailwinds and headwinds. We call the sum of a students headwinds their "Academic Support Index" or ASI. It is a measure of the amount of support they may need in order to maximize his or her academic potential.
As you can imagine, students with an Academic Support Index, or ASI, of 0 tend to do well in most learning situations, and if they don't, they have resources outside of the classroom that can supplement. On the other hand, students with a high (or very high) ASI need a lot of additional support.
In Summary:
• A student's ASI is the sum of the “headwinds” that make learning more challenging
• “Headwinds”:
outside our locus of control. As educators we can't change the headwinds.
• The power
is in the “Tailwinds”:
–Within
our locus of control
–Come
in the form of instruction, interventions, and support
• The ASI is a measure of the amount of support a student may need to meet his/her potential
• The ASI can also be considered as a measure of our obligation to provide that support
***
What is the relationship between the ASI and student outcomes?
The next step was to take a look at the ASI and student outcomes. Was their a relationship between a student's quantifiable headwinds and their academic performance? I started by looking at student data at key stages in students' education: Meeting reading targets at the end of the third grade, middle school performance on assessments, and high school cumulative GPA. The charts below all demonstrate a strong relationship between the ASI and student performance.
This table includes some additional outcomes:
***
At what point does the ASI predict under performance?
So, as we all know, a single headwind doesn't mean a student is going to under perform. Strong tailwinds can negate the impact of a single headwind. For example, an English Learner whose parents have advanced degrees and are working at high paying jobs is going to have all sorts of compensatory supports that may counteract the headwind. Similarly, just having the headwind of being male is hardly a predictor of academic under performance. All it means is that statistically males under perform females. But, add being poor and having a disability and it is quite another story.
In order to better understand when the headwinds are significant enough to impact student performance to a level we would define as "under performing" we calculated the confidence intervals for cumulative GPA for the various ASI scores. With a target of a GPA of 2.50 you can see that students with an ASI of 0-2 all are above 2.50. At an ASI of 3 we begin to see a decline at and below the 2.50 threshold.
These two groups were compared using the Chi test. The results confirmed that ASI 0-2 and ASI 3+ were two distinct groups when it comes to academic performance as measured by cumulative GPA.
This pattern was seen in a number of other measures. Some examples looking at middle school Smarter Balanced Assessment performance are seen below. The yellow line represents the grade level target for proficiency. In both examples ASI 0-2 (green) are above the proficiency target and ASI 3+ (dusty rose) are below.
We are seeing the same separation by these ASI clusters on a survey we are doing with our students that measures Academic Self-Perception. Academic Self-Perception is composed of two elements: Self-efficacy, or a students expectation of success on a task, and self-concept. Self-concept is influenced by how they judge themselves against their own prior performance as well as the performance of their peers. It is no surprise that students who tend to under perform both against the standards of proficiency as well as compared to many of their peers will have a lower level of academic self-perception. I discussed one consideration for working with struggling students in a previous post.
So now we can see that the ASI strongly aligns with student outcomes and that there are two clusters of performance within the ASI spectrum.
The real value in the ASI is in helping you better understand your student population. From there you will be able to better identify students for support, design interventions, and evaluate programs.