Statewide Data: White Paper

Information is Power
Educators Seek Better Ways to Use Educational Data for Student Success

Nearly everyone involved with education could tell you that schools have a significant amount of information when it comes to students. There is demographic information, attendance data, state and local test scores, participation in courses and activities, and more. The question is, are all schools able to translate the information they have into actions that will improve teaching and learning? Based on a series of listening sessions with educators across New York state in 2019 and recent interviews with school leaders, the answer is no. For all of the information that educators have access to, it is more than challenging for most to quickly and easily access information that enables them to make changes that help students. There are pockets of success with the use of data to drive powerful instructional changes and some promising new approaches, such as the use of the data visualization tool Tableau described on page 3.

Yet, the fact is that there is no universal, statewide practice or system that allows teachers to quickly and easily see what is going on with their students. The 2019 listening sessions with more than 400 public school stakeholders and recent interviews suggest that educational leaders recognize that they need access to better information systems and analysis. They are saying that we can do better for the students we serve. The purpose of this white paper is to bring new urgency to the conversation about leveraging the information that exists to help students succeed – all while protecting student privacy and ensuring data security.

‘That’s the Hook. That’s the Wow’

For all the information that is collected and stored in various systems across the state, teachers and instructional leaders report that they don’t have access to an easy-to-use system that allows them to “get their arms around” a better picture of student progress. Clifford Bird, principal of the 400-student Abram Lansing Elementary School in Albany County’s Cohoes City School District, talked about the amount of time that can be spent analyzing student achievement data without an action step to show for it.

“I have been an advocate for getting data that can actually trickle down into the classroom where the changes need to be made,” Bird said. “Me working in my office doesn’t make any difference to a fourth grader who’s challenged in reading. Getting a better understanding of what data tells that 4th grade teacher we should be looking at makes a whole world of difference.”

There are many factors currently working against teachers gaining that precise understanding of what will help their students learn. First, many of the existing systems are cumbersome for administrators to use, let alone time-crunched teachers. Also, as standards evolve it is difficult to ensure that student performance is being measured for the highest priority learning objectives. Finally, most educators are not steeped in data analysis.
“People who are proficient with data analysis can look at a spreadsheet, and analyze it and make decisions,” said Theresa Billington, a leader in the work that Capital Region BOCES in Albany is doing with partner schools using the Tableau platform. “People who don’t do that every day, it takes longer to figure out what you are looking at and then you have less time to make decisions. If we can present the data in a way that it’s easy for people to digest and understand, we can spend more time on the critical aspects of making those changes.”

Participants in the 2019 Statewide Data Conversations described the current system as one that was built or designed for compliance with data reporting requirements. This is far different from a tool that gives teachers what they need to make real-time instructional decisions to drive student growth and achievement.

“Classroom teachers are overwhelmed,” said Kristy McGrath, superintendent of the approximately 1,100-student Adirondack Central School District in Oneida County. “If we can get simple, usable data…that’s the hook. That’s the WOW. I’ve had many teachers that once they see the use of it, that’s when they want more and more and more of it.”

What Do We Mean by Data?

Data refer to the various types of information that schools have as it relates to students — from demographic data to performance on various assessments and other information such as attendance, participation in activities and more. Student information must be handled in the most secure and responsible manner possible. Strengthening how schools use information to improve student learning should not come at any cost to protecting student privacy and data security. As it relates to this project, the focus is on how the existing information that schools have can be better utilized to understand individual student needs and systemwide trends to improve teaching and increase learning.

Ensuring ‘Every Student Succeeds’

Since at least the passage of the No Child Left Behind Act of 2001, schools have been focused on the performance of students grouped by racial/ethnic and demographic backgrounds. These “subgroups” are: economically-disadvantaged students; students from major racial/ethnic groups; children with disabilities; and English language learners. The Every Student Succeeds Act, adopted in 2016, has brought an even greater focus on the performance of students from all backgrounds.

Schools are evaluated not just on how students are doing in the aggregate, but on both student growth and achievement for each subgroup. When a subgroup is not meeting established benchmarks, a school or district will be identified as needing improvement. Bird, from Cohoes, said that the analysis of student subgroup performance in the past helped identify how each was performing. Yet, the district had no analysis tool to help pinpoint which specific standards were preventing a given subgroup from meeting targets. “We had lots and lots of data to send to people, nothing that was definitive enough to change instruction,” he said. “So, we spent a great deal of time trying to improve everyone’s instruction in the hopes that a rising tide lifts all boats.”

Fast forward to the spring of 2021 and Cohoes is utilizing the Tableau visual data analysis tool with the support of Billington and her team at Capital Region BOCES. Tableau connects student performance with the individual standards and allows educators to see how students performed on state assessment questions connected to priority standards. The results can then be filtered by subgroup with the click of a button. “Now we don’t have to worry about the tide so much. We can just worry about particular boats and that they are doing what they need to do.”

The COVID Impact on Learning

With the COVID-19 pandemic stretching into a third school year, there is widespread concern that the impact of the public health emergency on education will last well into the future. Adirondack Central School District’s McGrath said this highlights the urgency of making sure schools understand the areas that students have mastered and where they need more help. Having accurate, current data is the only way to do this.

“More than ever we need to identify where those gaps of learning loss are and how to more strategically target them,” she said.

The situation is exacerbated by the impact of COVID on state assessments. There was an entire year with no information and a second year with incomplete results. This reinforces the need for each school district to have a curriculum aligned to state learning standards, a systematic practice of assessing student progress, and a way to easily view and analyze the results. It’s not realistic to expect teachers to do all of this on their own, McGrath said. It can only come from a centralized system and systematic support.

“You can’t just leave teachers on their own to do this work, to identify the assessments and to analyze the data,” McGrath said. “As we think about the model you have to kind of lead in a centralized way…You need to utilize an online platform so that some of the data manipulation is built into it.”

Equity? Not when Only Some Have Access to Quality Data Tools

Noelle Short, superintendent of the Long Lake Central School District in Hamilton County believes that better use of data could undoubtedly improve learning there. However, as the superintendent of a school system in the Adirondacks that draws just 70 students from 354 square miles, she has the experience of going to regional meetings and looking at attractive charts and graphs of academic results that don’t include her district because it does not have enough students in a given cohort.
Inconsistency in the tools that schools have access to is part of almost any conversation with educators about using data to improve learning.

In such a small district with limited resources, she feels that the systems in the state are not set up to promote effective use of data for all. “I need data that doesn’t exclude us because we don’t have enough kids,” she said. “I need it to be something that is more equitable…I need the people that handle the data to think about how they serve it up to us so that someone who is well educated and is smart, but has so many other things going on can take it and deliver it in a way that we can use.”

Frustration with data is not at all limited to districts based on size or resources. A prominent theme in the fall 2019 conversations was inconsistency and inequity in data systems between districts. In some cases it has to do with resources and in others it is simply the capacity and talent to leverage data. “In some places it’s a well-oiled machine,” McGrath said. “And in some places, there is a lot of data, lots of things accessible but the true ability to sort through it, have discussions and utilize it to impact instruction is probably very minimal.”

The result is a patchwork system where the ability to identify trends, ask the right questions and make changes to improve learning is almost randomized.

Conclusion: Ample Information, Little Clarity

The challenges and inconsistencies with the use of data to shape teaching and learning is prompting educators to ask if there is a better way to improve student outcomes and level the playing field. In the 2019 statewide data conversations and recent interviews, educators were asked what the use of data would look like in an ideal world. Here are some general themes that emerged:

• Systems that are more interconnected so that teachers and others can see information in “real time.”
• More consistency across districts, providing everyone with the ability to leverage the information that already exists to help students.
• Data made easier for teachers to analyze, understand and use.
• More clarity provided to all stakeholders about the different types of information collected and how it is used. Simplify whenever possible.

At the heart of these conversations is the fact that students only have one opportunity to go through our schools. Educators want to ensure that they are doing their very best to help students learn, grow and realize their potential. The discussions also reveal a growing recognition that there is power in the information that we have about our students. The question that still needs to be answered is: What would it take to better use this information to make decisions that have a powerful impact on our children and their learning?<

Tableau: An Example of Dynamic Information Analysis

A few years ago, a team from Capital Region BOCES and the Northeast Regional Information Center began pioneering the use of the data analytics platform Tableau to better understand the implications of student performance data. What is data analytics? This refers to analyzing multiple data points to compare past performance, draw conclusions and drive conversations about how to improve. Tableau does this with accompanying visuals that help people understand what they are looking at across connected data points. Here is an example: If a school district wants to know how economically-disadvantaged students did on the fourth grade English Language Arts assessment by priority standards compared to the region, with a few mouse clicks, Tableau could generate a heat map that shows the standards where students did better in shades of green and those where their performance was below the region in shades of red.

Theresa Billington, who has helped lead the rollout of Tableau at Capital Region BOCES, recalled one of the first times she shared the heat maps with a team of educators. It was a trend analysis showing how the district did on standards over time. In just one visual, they were able to see those standards that needed more instructional time or new strategies.

“Teachers wanted more,” she said, adding that their reaction was along the lines of, “‘Where has this been our whole life?’ They were very appreciative, ready to dig down and do the work and start to think about improving practice and supporting students.”

Click here to download a printer-friendly version of this paper.

About the statewide data conversation project:

The district superintendents of the 37 Boards of Cooperative Educational Services (BOCES) in New York have identified the need for education stakeholders to have meaningful and productive conversation about the best way to use student data to improve education. This paper is a continuation of the conversation, as the BOCES leaders recognize that a failure to come together to ensure best practices, both to improve instruction and ensure privacy and security, will result in missed opportunities to strengthen education for the students of New York. Funding to support this continued conversation was provided, in part, through a grant from the Bill & Melinda Gates Foundation. Additional financial and staff support was provided by Capital Region BOCES.