Simple Linear Regression, Student Growth Percentiles, Effect Size, What Does All This Mean?
Illinois educators now know that the old federal accountability system, No Child Left Behind (NCLB), is no longer applicable and Illinois schools will now be measured by a new accountability system titled Every Student Succeeds Act (ESSA).
What is the difference in these two systems? The main difference is that under NCLB all students had to achieve proficiency (meet or exceed state education standards) and ESSA uses proficiency and other measures to determine school accountability. NCLB was signed into law by President Bush in 2002. All states had to meet the standards within 12 years or by the end of the 2013-14 school year. NCLB supported standards-based education reform based on the premise that setting high standards and establishing measurable goals could improve individual outcomes in education.
Congress replaced NCLB with ESSA in 2015. The Illinois ESSA plan for 2018-19 includes, for K-8, the following criteria:
- 75% for Core Academic Indicators
- 50% for student growth for ELA and Math
- 20% for ELA and Math Proficiency
- 5% for EL Proficiency
- 25% for Student Success/School Quality Indicators
- 20% for Chronic Absenteeism
- 5% for Climate Survey
2018-19 Indicators for High School are the following:
- 75% for Core Academic Indicators
- 50% for HS Graduation
- 20% for ELA and Math Proficiency
- 5% for EL Proficiency
- 25% for Student Success/School Quality Indicators
- 7.5% for Chronic Absenteeism
- 5% for Climate Survey
- 6.25% for 9th Grade On-Track
- 6.25% for College and Career Readiness
In my opinion the main difference between NCLB and ESSA is not only the inclusion of the Student Success/School Quality Indicators but the concept that student academic accountability now includes student growth as the main indicator for proficiency. While ELA and Math proficiency still account for 20% of the school's score for K-8 and HS, 50% of the school score for K-8 is now based on student growth. For high schools 50% of the student academic accountability score will be HS graduation rate. HS graduation rate is the indicator for 2018-19 but ISBE is considering changing to PSAT and SAT student growth metric in the future. Student growth is a totally different metric than proficiency.
Student growth is defined in the Illinois ESSA Plan to be measured by "Simple Linear Regression." The definition of this concept
is "Simple linear regression is a statistical method that allows us to summarize and study relationships between two continuous (quantitative) variables: One variable, denoted x, is regarded as the predictor, explanatory, or independent variable." As an example, for Illinois accountability purposes, this is the comparison of a student's third grade PARCC Math score to the same student's fourth grade PARCC Math Score. It is a statistical calculation to determine if the student grew less, the same or more than all other students in the state scored based on the same third grade score. This statistic can also be calculated comparing to all students in the same grade at the same school, etc.
To put it in easier to understand words, students and schools will now be judged not only by whether the student(s) meet the proficiency standard, they will be judged in this methodology whether they made adequate growth compared to other students who started at the same point.
To further complicate this discussion, I have recently found out that ISBE is considering using a different methodology than simple linear regression to determine student growth. The National Center for Improvement of Educational Assessment, the group that is leading the ISBE Technical Advisory Committee (TAC), has recommended the use of Student Growth Percentiles (SGP) to measure student growth. This method is also a regression method in which student results are compared to the original score or what is called conditional status.
You may be wondering why I am writing about these student growth statistical methodologies. The reason I think you need to be aware of these factors is that schools will be designated a grade, A through F, for its accountability score as determined by the Illinois ESSA plan. How the State determines the score could affect your district and your schools in either a positive or negative way.
Under NCLB I do not think many citizens, especially parents, paid much attention to the school's meets-and-exceeds scores. Parents knew or think they knew that the schools they are sending their children to are performing well. For example, my old school district, Ball Chatham School District, has experienced rapid student growth over the past 30 years because parents think the schools meet the needs of their children. Houses continue to be built every year and the population in the school district continues to increase. This increase is almost solely due to the excellent reputation of the schools.
In states that instituted a grading system (A-F) for each of its schools, the dynamic of what parents think of their schools is different. Everybody went to school, everybody knows the difference between a letter grade of A and a letter grade of F.
The ISBE Technical Advisory Committee is considering setting a quota for each letter grade. For example, 20% A's, 20% B's, etc. They are also considering assigning a letter grade for each school by the amount of spending per student. For example, for those schools spending within the top 20% per student, each school will be assigned a letter grade based on 20% A's, 20% B's, etc.
Please pay attention to what is happening at the Illinois State Board of Education meetings and the recommendations made by the TAC. These recommendations may have long term implications for your districts and your schools.
Evidence Based Practices
We all know by now that Illinois has implemented a new funding formula for dispensing state funds for public education. It is called the Evidence Based Funding formula (EBM). There are no state policy mandates connecting EBM and ESSA, there are no strings attached. However, EBM's formula was based on 26 evidence-based factors and the formula commutated what a prototypical school would need to implement these evidence-based factors.
I think the key term is evidence-based factors. School districts, under ESSA, have a greater burden to use evidence-based factors when making decisions, not only how they will spend the money they may be getting from the state, but also how they are improving the education accountability indicators as outlined in ESSA. Questions such as the following need to be asked and answered when considering these factors:
- If we are going to spend these new dollars on X, how will we know it is effective?
- What are the outcomes we are expecting?
- What data elements do we have?
- How do we measure the "effect"?
- What is the "Effect Size"?
The 26 evidence-based elements in the EBM were decided using the statistical terminology of effect size. Much of the research of John Hattie was included in the research for EBM. Among Hattie's 2017 updated
list of factors
that positively effect education (from highest to lowest) are the following:
- Teacher estimates of student achievement = 1.62 Effect Size
- Collective teacher efficacy = 1.57 Effect Size
- Student self-reported grades = 1.33 Effect Size
- Teaching strategies that include cognitive task analysis = 1.29 Effect Size
- Teaching strategies emphasizing response to intervention = 1.29 Effect Size
As school leaders you need to become knowledgeable about "Effect Size" and use this and other evidence-based approaches to making decisions to spend money wisely and to increase student accountability indicators. The future of education will center on using data to make decisions and will no longer rely on "implementing the new flavor of the week." IASA will be conducting ISBE-approved academies on ESSA and these new requirements. If you want to know more about these academies contact me at
[email protected]
or my assistant Melissa at
[email protected]
and we can schedule this new academy in your school district or in your region.
Tip of the Week
I use my Apple Watch to not only monitor my workouts but to also monitor my health. Invest in some type of activity tracker to do the same for you. It will change your life.
