Science Teachers ’ Data Use Practices : A Descriptive Analysis Virginia

There is a debate on students’ low science achievement in the United States, particularly among low income, African American students, and Latino students. An important part of the education community’s response to low achievement generally and in science specifically has been the implementation of high stakes accountability policies. Because of accountability’s emphasis on educator data use, much research has examined different facets of it throughout educational organizations, but research has not focused on the extent to which data use might be content-specific. The purpose of this paper, then, was to investigate the data use practices of science teachers. Drawing from a broader study of science teachers in grades 5-8 across six school districts, this study reports results from teacher surveys and interviews. The findings indicate that while there Education Policy Analysis Archives, Vol. 24 No. 86 2 were examples of science-specific data use, most of the science teachers used data in ways consistent with previous content-agnostic research. Implications for future research, policy, and practice are discussed.


Introduction
There is a debate on students' low science achievement in the United States, particularly among low income students, African American students, and Latino students.Though the average eighth grader performed slightly better on the NAEP science test in 2011 than in 2009, African American and Latino students as well as students eligible for the federal free or reduced lunch program scored 27-35 points below White students (NCES, 2011).An important part of the education community's response to low achievement generally and in science specifically has been the implementation of high stakes accountability policies.Implemented at the state level beginning in the 1980s, accountability policies, in theory, should lead to school improvement by focusing educators' attention on student outcomes, as measured by standardized assessments (O'Day & Smith, 1993;Smith & O'Day, 1990).Implied by these policies is that educators will use the data from the standardized assessments to improve how schools and districts meet their students' needs (Hamilton & Stecher, 2006;Hamilton, Stecher, & Klein, 2002;Hanushek & Raymond, 2005).The No Child Left Behind Act (NCLB), passed in 2001, established accountability as federal policy as well.
Most research on educator data use, however, either has been agnostic to the particular subject area from which the data are derived or has focused on data use in English language arts or mathematics.As a result, what scholars have reported as the ways in which teachers use data also has focused on 'general' data use or on the use of reading and/or mathematics data.Given that until 2007 accountability policies such as NCLB only required annual testing in English language arts and mathematics, this narrower view makes sense.Beginning in 2007, however, NCLB required states to test students in science once in elementary school, once in middle school, and once in high school.Furthermore, the unique nature of science (e.g., the heavy emphasis on scientific practices in addition to scientific knowledge) warrants additional investigation into the specific data use practices of science educators.
This study also was carried out in a rapidly changing data use landscape, namely because of the rapid expansion of technology in classrooms, including more sophisticated data systems and digital curricula.The mandate for more and better educator data use led to an expansion in the use of data warehouses (Kerr et al., 2006;Wayman et al., 2004;Wayman & Stringfield, 2006) and, more recently, in the creation of program-based student management systems1 , such as the one embedded in the online science curriculum that was the focus of the broader study from which these findings are drawn.
Therefore, the purpose of this paper is to investigate the data use practices of science teachers.Working from the premise that science is unique when compared to other disciplines because of the focus on hands-on, inquiry-based learning (Minner, Levy, & Century, 2010) and performance-based assessments (NRC, 2000;Songer & Ruiz-Primo, 2012), this exploratory study set out to understand what data science teachers use, in what ways, and what data they want to use.This study focused on science teachers in grades 5-8 for two reasons.First, beginning in sixth grade and sometimes even as early as fifth grade, science teachers usually are departmentalized and only teach science and so they teach science on a regular basis, unlike many elementary school teachers (Milner, Sondergeld, Demir, Johnson, & Czerniak, 2012).Second, the middle school years in particular are a crucial time for students in science because research suggests that it is then that students cement either their interest (Auger & Blackhurst, 2005;Bandura, Barbaranelli, Caprara, & Pastorelli, 2001;Riegle-Crumb, Moore, & Ramos-Wada, 2011;Tai, Liu, Maltese, & Fan, 2006) or disinterest (Aschbacher, Li, & Roth, 2010;NRC, 2011) in science.In order to guide the investigation, we used the following research questions: 1. What data do science teachers use in their classrooms?2. What data do science teachers want to use in their classrooms?
In the remainder of this section, we review the literature on data use and accountability, and then teacher data use more specifically.Then we describe some of the 'best practices' in science instruction and assessment and explain why these might lead to unique data use practices in science classrooms, and what those practices might look like.

Test-based Accountability and Data Use
Accountability rests on the premise that the right policies can induce teachers and educators to make better instructional decisions based on data.As a result, major state and federal education reform efforts have included the creation of accountability systems that utilize high stakes standardized tests in certain subject areas to focus educators' attention on student learning outcomes (Hanushek, 2003;Hanushek & Raymond, 2002;O'Day & Smith, 1993;Opfer, Henry, & Mashburn, 2008).Educational data use is a key tenet of these accountability policies.In theory, educator analysis of the student achievement data collected each year is meant to improve teacher practice, student learning, and how schools function generally (Hamilton & Stecher, 2006;Hamilton et al., 2002;Hanushek & Raymond, 2005;O'Day & Smith, 1993;Smith & O'Day, 1990).The reauthorization of ESEA in 2001 (NCLB), in particular, placed renewed emphasis on using student data to improve educational decision-making and student outcomes (Fuhrman & Elmore, 2004;Lachat & Smith, 2005;Supovitz, 2009;Wayman, 2005).

Teacher Data Use
While there has been little research on the data use of science teachers specifically, much research has explored teacher data use generally.
Kinds of data teachers use.Research consistently has found that teachers use a wide variety of student data in their work, though most of this research has not clarified whether the data teachers use are specific to particular subject areas.These data fall into two general categories: Achievement or performance data and non-academic data.Within the general performance data category, teachers' data use breaks down even further depending on how the data are collected.By far the most common source of data for teachers is state achievement tests (Beaver & Weinbaum, 2015;Gallagher et al., 2008;Marsh et al., 2006;Wayman, Cho, Jimerson, & Snodgrass Rangel, 2015), and from district or industry-created interim progress or benchmark tests (Marsh et al., 2006), which often are designed to prepare students to take end-of-year state assessments (Supovitz, 2012).Indeed, in their recent study of state performance data, Beaver and Weinbaum (2015) described their use by teachers as "pervasive".
Teacher academic data use is not limited to state or interim assessments, however, but also includes classroom work and observations of students during class.Classroom work includes teacher-or campus-made tests and quizzes as well as homework (Gallagher et al., 2008;Marsh et al., 2006;Wayman et al., 2010).Other classroom data that teachers have reported collecting and using include observations of student work in class and student questioning and discussions (Marsh et al., 2006).Finally, teachers also use several kinds of non-academic student data.These include attendance and student mobility data, demographic data, and data relating to students' statuses, such as whether a student has an individualized education plan (IEP) or is categorized as an English language learner in need of language supports (Gallagher et al., 2008;Marsh et al., 2006;Wayman et al., 2010).
Types of teacher data use.Research also suggests that teachers' use of data varies quite a bit, though none appears to be specific to a particular subject area.Beaver and Weinbaum (2015) propose three categories of teacher data use: Using data to inform future analysis and collection efforts; school-level responses to data analysis; and student-level interventions.They form the three categories based on an extensive review of the literature, though it should be noted that their application of the categories was limited to teachers' use of state assessment data.Teacher data use that falls into the first category includes: Setting school-wide goals, planning interim assessments, and creating school improvement plans.Data use that would fit into the second category includes: Curriculum alignment (Marsh et al., 2006), identifying teacher needs for professional development identified (Breiter & Light, 2006;Supovitz & Klein, 2003), refining course options for students (Kerr et al., 2006), and narrowing the focus of instruction to tested subjects and skills (Hamilton et al, 2005;Marsh et al., 2006).Finally, the third category of teacher data use comprises student-level interventions such as targeting specific groups of students for additional support, or identifying particular skills for which students need more practice (Booher-Jennings, 2005;Breiter & Light, 2006;Hamilton et al., 2005).
Factors influencing teacher data use.Several factors influence or shape the data teachers use, and the ways they use them.As several scholars have argued, data use is not an obvious activity; rather, it is fraught with uncertainty because everything from the validity of the data themselves to their meaning and relevance are contested and re-negotiated through daily practice (O'Day, 2002;Spillane, 2012).For example, research suggests that across different contexts, how teachers interpret what counts as data can vary widely (Cho & Wayman, 2014).Indeed, the sense teachers make of data is shaped by their own beliefs and prior knowledge about the data and their students (Coburn, 2001;Coburn & Turner, 2011;Palmer & Snodgrass Rangel, 2011;Spillane et al., 2002).
Research has not investigated the extent to which data use varies across subject areas.Most work that has been done has focused on the highly tested areas of mathematics and reading, and has not highlighted differences that might be specific to one area or the other.Little work has set out to examine whether teachers data use varies across subject areas, though there is reason to believe that it might.The next section discusses why teacher data use might differ across subject areas, specifically in science.

Best Practices in Science Instruction and Assessment
According to the National Research Council (NRC, 2001(NRC, , 2012)), there are three purposes of assessment in science education: Formative assessment, summative assessment, and assessment for program evaluation.Educators (and researchers) often mistakenly rely on standardized assessments to meet the needs of all three assessment purposes.Based on its utilization of a limited number of formats (paper and pencil or computer-based) and one question type (multiple choice), this form of assessment is limited in its ability to measure performance outside of concept development, such as scientific explanations, argumentation, or the process skills associated with carrying out scientific investigations (NRC, 2012;Quellmalz, Timms, & Buckley, 2005).Best practices call for assessments to be specifically designed for the purpose of assessing content domain or the science practices of interest (NRC, 2001).
Several education policy organizations have pushed towards reforming science education by focusing not only on science content but also science practices, often through hands-on and inquirybased instruction (Furtak, Seidel, Iverson, & Briggs, 2012;Minner et al., 2010).Inquiry-based instruction implies a different approach to assessment than traditional standardized tests.Instead of a reliance on multiple choice tests, science teachers are expected to administer a variety of formative and summative assessments that require students to demonstrate or perform their knowledge (Davis, Petish, & Smithey, 2006;NRC, 2000;Songer & Ruiz-Primo, 2012).These more authentic forms of assessment include labs and experiments (e.g., Hofstein & Lunetta, 2004), but also extend to classroom discussion, scientific explanations, and argumentation (Berland & McNeill, 2009;Duschl & Osborne, 2002;Duschl, Schweingruber, & Shouse, 2007;Krajcik, McNeill, & Reiser, 2008;McNeill & Krajcik, 2008;Songer & Gotwals, 2012); performance assessments such as problem-or project-based learning assignments (Kolodner et al., 2003); writing assignments (Wiggins, 1990); and classroom presentations.Utilizing a variety of assessment that is specifically designed to measure student mastery of scientific practices provides teachers with more actionable data on student mastery of both science content and scientific practices (Black & William, 1998;Buck & Trauth-Nare, 2009).
Using the research questions presented above, this study investigated the data use practices of fifth-eighth grade science teachers.Next, we explain the context of the study and present the methods used to collect and analyze the data.Then, we present the findings from the exploratory investigation.We conclude with a discussion of the findings and outline implications for research, policy, and practice in science education.

Methods
The results presented here are drawn from a broader study that focused on the development of the user interface of a PreK-12 online science curriculum (referred to below as "the study curriculum"). 2In the first year of data collection, however, the focus was on teacher data use broadly defined and the focus of the survey and our conversations with teachers was not on the study curriculum.The participating science teachers were asked about all of the different kinds of data they used to plan their instruction and assessments, including from other science curricula and materials.In this way, the results reported here are not about one particular curriculum, but about data use generally among science teachers.Data were collected to answer exploratory questions: Which data the science teachers use on a daily basis to plan, teach, and assess their students, and which data they would like to use to complete that same work.

Participants
During the summer of 2014, we recruited six suburban school districts in southeast Texas to partner on the project.Three were charter management organizations (17 schools), and three were traditional public school districts (TPS; 19 schools).The six districts constituted a convenience sample and they were chosen because they all had some experience using the study curriculum.The six districts were quite diverse (see Table 1).For example, District B, a TPS, is a suburban district that serves a majority White student population with few economically disadvantaged students and high levels of student performance in science. 3Conversely, District A, also a suburban TPS, serves large African American and Latino student populations, with a very high percentage of their students qualifying as economically disadvantaged and lower performance on the state science assessment.The three charter school organizations (Districts D, E, and F) serve predominantly Latino and African American students, and large numbers of economically disadvantaged students.District C had the lowest passing rate in 2014 of all six of the districts, while District D had a higher passing rate than District A but a lower one than District B.
From these districts, we recruited 73 teachers, 71 of whom completed all of the research activities; 45 of them were from public schools, and 26 were from charter schools.We recruited teachers by working with the district science coordinators to email all teachers about the study; participation was voluntary and therefore the distribution of teachers was not even across the districts, and participants cannot be considered representative of other teachers in the districts.According to responses from our teacher survey (described in greater detail below), over half of the participating teachers had five years of experience or fewer, and over three-quarters of participating teachers had fewer than 10 years of experience (Table 2).Compared to the state average, teachers participating in this study were, on average, less experienced.Teacher experience also varied across districts: Teachers in District A had significantly more experience than those in Districts E and F, while those in District C significantly more experience than teachers in District F. 4 The vast majority of teachers participating had at least some previous experience teaching science (58%), and 66.2% of all of the teachers only taught science; 25.4% taught other subjects as well, including English language arts, math, or social studies.Participating teachers were relatively evenly distributed across the four grade levels (Table 3), though there were relatively fewer teachers from sixth grade and there was one fourth grade teacher.

Data Collection
We collected data from participating teachers in three ways: A teacher survey, teacher observations during planning periods or professional learning community meetings, and interviews.Here, we only report on results derived from the survey and the interviews.The survey was administered to all of the 73 teachers we recruited; 71 teachers completed the survey, for a response rate of 97%.The survey primarily drew items from existing surveys, including the Study Curriculum Evaluation Survey (Snodgrass Rangel, Bell, Monroy, & Whitaker, 2012), the Survey of Educator Data Use (Wayman, Cho, & Shaw, 2009), the School and Staffing Survey (demographic items; NCES, 2014), and the TPACK Survey (used as a measure of science content knowledge; Koehler, 2009).The survey, referenced in Appendix B and available as a supplementary file, focused primarily on general types of data that the teachers used, the frequency with which they used them, and the purposes of their data use.It included several scales, though only the data use practice scale (Cronbach's alpha = 0.95)5 will be analyzed here.The survey did not ask the teachers what sciencespecific data they used, though it did provide them with opportunities to answer open-ended questions about the data and tools that would be useful for planning and collaboration.Teachers' responses to these items focused almost exclusively on the data tools they wanted to use, and not on the kinds of data they wanted to use; their responses are provided in Appendix C.
We conducted two kinds of interviews with the teachers: One-on-one interviews and focus group interviews.In total, we conducted 12 focus groups on those campuses where more than one teacher had volunteered to participate in the study; each of which included two to four teachers.The remaining teachers were interviewed one-on-one because, in all of these cases, they were the only participating teacher on their campus and the only teacher at that grade level.The interviews were semi-structured, which allowed the researchers some flexibility to address additional issues that arose during an interview that were not in the protocol (e.g., Merriam, 2009; protocol included in Appendix A).During the interviews, we were able to ask teachers about their science-specific data use.The interview protocol was developed together with the Study Curriculum developers as well as with the project's advisory board.All interviews were audio-recorded and transcribed by the research team.

Analysis of Data
We analyzed our data descriptively and analytically, both across and within district cases.Survey data were analyzed descriptively and also using independent sample t tests and one-way and two-way ANOVAs in order to tease out any differences across the different types of districts, the different districts, and different grade levels.The interview data were coded using a list of a priori codes that were generated from the interview protocol.All interviews were coded at least twice by different members of the research team in order to ensure reliability.Once we had coded the data, we compared results from the interviews both across grade levels and across districts and district types to identify similarities and differences in data use practices.

Limitations
As with any study, the findings presented here are limited.The first is that the focus of our data collection efforts was exploratory, with an eye towards understanding the different kinds of data that the teachers were using, as well as the kinds of data they wanted to use; the question of how teachers used data and why came up naturally as part of our investigation, but not equally across all interviews or observations.Furthermore, our sample of districts and teachers was a convenience sample and therefore are not necessarily representative of other teachers in the districts, or of other districts.As a result, we are limited in our ability to make generalizations from our data to other teachers, schools, and districts.Furthermore, the number of teachers were able to recruit from each district was not equal, which meant that in our group comparisons, the Type I error rate may have been affected.As an example, the number of teachers per grade level was unequal as more teachers were in either fifth or eighth grades as compared to sixth or seventh grades.Another limitation inherent in any study of data use is that what counts as 'data' varies across districts and even schools.While we think that is a finding in and of itself since it reflects the way data use is framed and defined in their school and district, teachers may use data, such as from observations, that they might not consider 'data' in the formal sense and therefore may not have mentioned when prompted to elaborate on their data use.Finally, our findings are limited because they are derived from teacher self-report and from limited observations that cannot be considered representative of teachers' planning or data use practices.

Results
In this section, we present the results of our analysis of teacher data use logics.First, we examine the data teachers used in their classrooms, and then we move on to the data teachers wanted to use in their classrooms.

Data Science Teachers Used in their Classrooms
Here, we draw on data from the survey and interviews to describe the kinds of data the teachers used in their classrooms.In general, we found that teachers use a wide range of data to plan both their lessons and classroom assessments.We present the results both overall and also broken down by district type (charter vs. traditional public school), district, and grade level.

State assessment data.
Students in Texas public schools take a science assessment three times for the purposes of both state and federal accountability, in fifth and eighth grades, and in high school for biology.Across all districts, the teachers reported relatively infrequent use of state achievement data (see Table 4; 'never' was coded as 1, and 'daily' was coded as 7).Indeed, it was the second least-frequently used kind of data and relative to the teachers' other responses, there was a fair amount of variability in responses (M = 3.28, SD = 1.74).There also were no significant differences in reported frequency of use of state assessment data between the charter school and traditional public school teachers.There were, however, some differences among grade levels in frequency of use of state assessment data (F=5.93[4, 62], p < 0.05).Specifically, fifth grade and eighth grade teachers reported using state achievement test data significantly more frequently than seventh grade teachers.These two differences make sense given that fifth and eighth grade science teachers must prepare their students for the end of year state assessment.There also were significant differences across districts in teachers' reported use of state assessment data.A one-way ANOVA analysis revealed that group differences existed between the following districts (F=5.41 [5,62], p < 0.05): Teachers in District B reported using state achievement data more frequently than teachers in District A, and teachers in District D reported using state achievement data more frequently than teachers in Districts A, C, E, and F. In District D, both the fifth and eighth grade teachers reported more frequent use of state assessment data than the teachers in the four districts named.There was, however, no significant interaction between district and grade level, suggesting that while there were differences in the frequency of use of state assessment data across districts and across grade levels, frequency of use did not depend on both district and grade level.Data from the interviews confirmed what the survey uncovered, that teacher use of state achievement data was not as common as the use of other, more timely and local sources of data.Importantly, we did not ask in the interviews how frequently the teachers used each kind of data, but rather we asked them about all of the different kinds of data they used at all, and in what ways.The teachers who mentioned using state achievement data were from across all four grade levels, though most were fifth-and eighth-grade teachers, which mirrors use of these data as reported in the survey.Based on the interviews, the traditional public school teachers appeared to use state achievement data more than the teachers from the charter schools, but this difference was not supported by the survey data.
In the interviews, teachers reported using students' state assessment data to reflect, to create local assessments, and to plan long-term.For example, two fifth grade teachers at the same campus in District B described how they utilized item analyses from past state achievement tests to see what their students had struggled with in the past and to use that information to make changes to their planning with the next group of students.They also described focusing on how the questions of the state achievement tests were worded so that they could prepare future students for similar items.They told us, [T2]: Because it's a general trend of the same things are being missed.
[T1]: And we look at those past STAAR questions, how are they asked, and we kind of use that to know kind of where we need to go on things that we need to make sure that we cover with them too.
Similarly, teachers reported utilizing their current students' previous state assessment data in math and reading to try to identify where students might struggle with science content and with the science assessment.For instance, an eighth grade teacher from District A told us, "we also look at their seventh grade, we talk to their seventh grade teacher to see, and we get their data to see how they did on reading and math in STAAR."This practice was particularly useful, and common, in science due to the fact that in fifth, seventh, and eighth grades, most of the teachers did not have end of year science data for their incoming students.
Teachers also described using state achievement test data as part of 'big picture' planning discussions with others in their building.These conversations often happened prior to the school year or during special planning days, when teachers, instructional coaches, and even administrators would sit and plan out whole units at a time, including how many days to dedicate to teaching each learning standard.A fifth grade teacher in District B reported that, …at the beginning of the school year, in some of our meetings that first week, the administrators sat with us and they had all this data, we used Lead4ward [a data reporting tool the district uses] that had compiled some of the information and we had information from our district and state, and you plug it all in to see which [Texas standards] were more challenging.
Several eighth grade teachers in District B noted that at the beginning of the school year, the past year's results were helpful for big-picture planning and looking for trends over time in student achievement, but ultimately using the previous year's eighth grade results had them comparing apples to oranges since the new students could be quite different.If the teachers wanted to look at the students' old state achievement scores, they reported that this "is hard in science because the STAAR [state achievement] data is from three years ago", and therefore not necessarily an accurate reflection of what that student has learned since fifth grade.Similarly, a sixth grade teacher in District B noted that she did not spend much time looking at students' fifth grade STAAR results because of a lack of alignment in the standards from one grade to the next.
Use of STAAR data was concentrated among the eighth grade teachers and their primary use of these data was for planning out the whole year.For example, an eighth grade teacher in District A told us that, "based on previous years, on what students struggled with the most [on the STAAR], so what they will need more time on vs. what they can pick up easily, we would dedicate a fewer number of days on that."Similarly, an eighth grade teacher from District B reported that, "I also have the released STAAR, and just an idea of how we did, what was weak, what was strong.What we need to focus on."District assessments.Results regarding teacher use of district-wide assessments, such as interim and benchmark assessments, were consistent across both methods of data collection.In the survey, teachers responded that on average they used data from district-wide assessments between once a month and two to three times a month, and there was less variability in teachers' responses than there was for state achievement tests, suggesting that the frequency of use among teachers was more consistent (M = 3.38, SD = 1.68).When the results were disaggregated, a significant difference in the frequency of use of district-wide assessment data emerged between teachers at the charter schools and the teachers at the traditional public schools: Teachers at the charter schools on average used these data more frequently than the traditional public schools teachers (mean difference = 0.94, t(67) = , p < 0.05).There also were significant differences across districts in the frequency of districtwide data use (F=5.68 [5,62], p < 0.05): Teachers in District D on average reported using these data more frequently than teachers in Districts A, B, and E; and teachers in District F reported using them more frequently than teachers in Districts A, B, and E. There were no differences in the frequency of district-wide data use across grade levels.
According to our interviews with teachers, data from district-wide assessments were primarily used to to gauge students' preparation for the fifth and eighth grade STAAR assessments, and to provide interventions for those students struggling on any learning objective.It is important to note that the districts used different kinds of district-wide assessments.The traditional public school districts (A, B, and C) and one of the charter schools (District F) created their own district benchmark science assessments to test students' understanding of two to three months' worth of content.Districts A, B, and C created assessments similar to the state science assessment, while District F created its district 'common assessments' to prepare its students for the Advanced Placement exams.The traditional public schools administered the benchmarks twice a year (state law prohibits more than two administrations) to help teachers gauge whether students had learned the content during the previous months.It is likely, therefore, that data from these assessments were used more frequently than state achievement data because the district-wide assessments were administered more regularly and in every grade, so the results were more timely and relevant to all of the participating teachers.
In contrast, the three charter school districts (Districts D, E, and F) utilized the Measures of Academic Progress (MAP) interim assessments created by the Northwest Evaluation Association (NWEA).The MAP is not aligned to the Texas state standards, though it is a multiple choice assessment.Unlike the district-created assessments, the MAP assessments measure students' mastery of all of the content knowledge and skills they are expected to learn in a particular grade level.As such, the results can be used as predictive tool or an "early warning summative" tool (Wiliam & Thompson, 2007, as cited in Horn et al., 2015, p. 3).
Despite the differences in the actual tests, teachers reported using data from district-wide interim assessments in similar ways: To gauge students' preparation for end-of-year testing and to identify struggling students for interventions.For example, an eighth grade teacher in District B noted that the results from the district benchmarks "are helpful for our STAAR tutorial groups, we use those again at the end of the year."Similarly, in District A, two teams of eighth grade teachers reported using the district benchmark to track their students' progress, and also to compare their own students to those from other middle schools in the district.Several sixth and seventh grade teachers (non-tested years) told us they relied on the district benchmark data to understand their students' past performance and to do longer-term planning in lieu of using STAAR data.A fifth grade teachers in District C told us, "we need the information, I don't want to wait until the released test to see what we skipped or missed."6Finally, an eighth grade teacher from District F recounted, "Once we get the common assessment data, I will use that to first, see who needs a lot of help so they can do better on the next common assessment and, second, use that data towards preparation for STAAR and what groups will be separated." We did hear about one difference from teachers in District F. They described utilizing their students' MAP scores in some cases to prepare their students for the STAAR, but in most cases to see how students were progressing towards each student's improvement goals, which the teachers created by correlating students' MAP scores to equivalent Advanced Placement (AP) scores.A seventh grade teacher described this to us, saying, "Every kid has a growth goal for the year, which is one number higher than where they were last year, so all my 1's, my goal is to get them to be 2's this year.Is that something that realistically would happen, no but we will try to push as many as we can, and our 5's we want to stay 5's."Teachers used the correlations to predict how students might do on an AP test and to help students set their own improvement goals in relation to their readiness to take an AP test.
Local assessment data.Teachers reported using data from their own assessments more frequently than other kinds of data.In the survey, teachers responded that on average they used data from their own assessments between two to three times a week and daily; this form of data use generated the least amount of variability in teachers' responses, suggesting that the frequency of use among teachers was quite consistent (M = 5.38, SD = 1.51).There was no significant variation across grade levels, across districts, or across district types (charter vs. traditional public school districts).
Data from local assessments helped teachers in five of the six of the districts make week-toweek and even day-to-day planning decisions for their science instruction. 7For example, teachers reported using the data from local assessments on a regular basis to create intervention or tutorial groups, particularly in the eighth grade.An eighth grade teacher in District A told us that they sat down as a team to review student scores on their twice-monthly campus assessments, nothing that these data "play a big role.We look at what our kids are struggling with the most, and like I said, we try to put those one or two questions back on the new [campus assessment] that we have, and we'll try to hit re-teaching points".Aseventh grade teacher from District F told us that at her school, "quizzes are standards-based, so I can tell which objectives we did well on or didn't do well on.That leads me to whether or not we do a full group re-take or do we do individual re-takes.Who do I pull for tutorials?Who do I pull for one-on-ones or small groups?"Teacher use of "other data".Based on survey responses and interviews with teachers, we learned of other kinds of data and data use.Among these other kinds of data were non-academic data, which the survey defined as including student demographic data, special programmatic data (e.g., student enrolled in special education programs or is categorized as an ELL), and whether students have had disciplinary issues.On average, teachers utilized 'other' forms of data frequently, just under two to three times a week, though there was substantial variability around the mean (M = 4.94, SD = 2.1).There were no significant differences in the frequency of use of 'other' data between charter and traditional public school teachers, among grade levels, or among districts.Teachers also were encouraged to describe the other forms of data they used to understand their students and improve their instruction.Not all teachers responded, but for those who did, their responses are summarized below in Table 5.As can be seen, the teachers use a wide variety of student data in their work, some of which were science-specific (e.g., observations during in-class labs).
In the following sections, we draw exclusively on our interviews with teachers to briefly describe how teachers used 'other' data from formative assessments, classroom observations, science labs, in-class science labs, student writing, classroom discussion and questioning, performance tasks, and argumentation exercises.As is depicted in Figure 1, which summarizes the number of teachers who mentioned each data source in an interview, some of these forms of data were identified more frequently than others.For example, data from observations and exit tickets (formative assessment) were mentioned by more teachers than were classroom questioning/discussion, which were mentioned more frequently than student writing.Of these forms of 'other' data, some were specific to science, while others are common across subject areas.
Exit ticket data.Many teachers reported using data from a specific form of formative assessment: 'Exit tickets'.As teachers explained to us, unlike many of the other forms of assessment from which they drew data, exit tickets were not required, but rather were strongly encouraged by teachers' instructional coaches as a way to gather daily formative data on their students' learning.Exit tickets are short formative assessments that usually are administered at the end of class and ask about the concepts covered in that day's class; exit tickets would not be considered science-specific and are commonly used across subject areas.In general, teachers reported using the exit ticket data immediately, such as to make changes for the following day or week.Several teachers from across the districts told us that they use the exit ticket data to follow up in small groups to work on concepts they did not yet understand.Other teachers described using the data exit ticket data to create the next day's 'warm-up', which usually consisted of additional practice on a particular concept.
We heard about the use of exit tickets use in five of the six districts; only teachers in District E appeared not to use them.One eighth grade teacher in District F told us, "Here we use exit tickets, we are supposed to.It's a short time, usually 5 minutes of class I used exit tickets with about 3 quick questions about what we did in the class."An eighth grade teacher in District A used data from her exit tickets to make immediate adjustments to her lesson plans: "I like the exit ticket type things.Formative because then I can-Oh yes! Lesson plans have to be changed tomorrow because they didn't get this!"  Science teachers' data use practices Observation data.In our interviews, several teachers discussed using their in-class observations of student work as a source of data.As with exit tickets, data from in-class observations are commonly used across subject or content areas.Many teachers from District B and District C in particular discussed using informal classroom observations to learn about what their students understood.This school district was, on average, substantially wealthier and higher performing than four of the five other districts.For example, teachers in District B described using their observations formatively to ensure that their students were understanding the material or the application of new material, and to quickly decide whether to pause a lesson to follow up with students or a certain group of students.A sixth and eighth grade teacher also in District B described observing her students and, if she saw them struggling with something, using that as a jumping off point for talking with that student's other teachers.Teacher observations appeared to go hand-inhand with hands-on student work, such as labs and investigations.No teachers in either of the charter school districts reported use of observation data in our conversations.
The teachers also explained that they used their informal observations as a way to triangulate with other formative and summative data they already used as well as a way to see if they students really 'got it'.For instance, two fifth grade teachers recounted that, "I think we use observation a lot.Because when we do our Explore [hands-on activity], we walk around and…we get a really good sense of if they know what they're doing or if they don't."They described what they look for when they walk around and observe their students, saying, "It could just be like a deer caught in the headlights", a look that betrays a student's lack of understanding.Similarly, in a middle school, the teachers agreed that they used "a lot of it is observations" in addition to other kinds of data.Yet another team of fifth grade teachers told us they used "basic classroom observations" of students as they worked to help them decide, "what else do we need to work on, where do we need to put in more information and do things like do an extra lab." Triangulation allowed the teachers to develop a more complete understanding of student comprehension since they were relying not on one or one type of measure, but rather two or more types of measures.Two of the fifth grade teachers in District B explained that in many cases different sources of data provided them with different, often contradictory, information about student learning.For example, one of them told us, "I think my observation, especially in science, a lot of kids who seem like they're on the ball, when you put it in a chart like this, they don't get it.Some of the kids whom I think, wow they have it, when I'm watching them, and I see it like this, and I'm shocked at some of them on the lower end of the scale [on quizzes]."As a result of the dissonance that often came of teachers' data triangulation, several of the teachers described how they had to dig deep to uncover where students' misunderstandings were-whether a student was struggling with the concept itself, or perhaps was struggling with the way a question asked about the concept.
Data from science labs.There was relatively little mention of teacher use of data derived from in-class labs or experiments, which do generate science-specific data.Indeed, in District E, the teachers did not make almost any mention of science labs, either as part of their instruction or as a source of useful data, and in District A, teachers explained that only their pre-AP students completed in-class labs.In the remaining four districts teachers reported regularly implementing science labs, though there was little to no discussion of using the students' work from the science labs as a source of data.In Districts B and F, teachers described using data from lab assessments or homework assignments that the teachers administered after the lab was completed.An eighth grade teacher in District B reported that he carefully watched students as they worked and listened to their conversations to gauge the extent to which they understood the concepts they were working with and their application.In contrast, a teacher in District F, where all science teachers were required to complete at least one lab per month, told us that district's lab assessments were not always aligned to the rest of the curriculum and therefore were not all useful as sources of data.
Data from student writing.A handful of teachers from across all four districts discussed using data from writing assignments to deepen their own understanding of what students were learning.Data derived from student writing assignments is common across other subjects and is not science-specific.The teachers who described using student writing as a source of data explained that it provided a window into students' understanding of a concept because it required them to articulate their thinking.It was not a common practice, though, in large part because of a lack of time or, as one eighth grade teacher from District F recounted, "I don't like to read a lot of writing." An eighth grade teacher from District A explained the value that writing assignments provide to both the students and the teachers, noting that, More understanding of what they do and don't know from reading an essay than from a multiple choice question.Because when they have to explain it, you can tell if they really know it or not… And not just the answer, because I see that they can write the answer, but I can see that they don't understand it.If you ask them to write a sentence, you say, oh no, they know the word, but they don't really know what it is.
Similarly, an eighth grade teacher from District C explained that having his students write because writing required them to use their knowledge and therefore was a good indication of whether they had really understood the content.Finally, a teacher from District D explained that she had her students write out detailed explanations of their quiz corrections, asking them to explain "why the correct answer was correct," adding, "that helps me to know that they really got it, and put it in my notes."Data from classroom discussions and questioning.Another informal but not sciencespecific way that some of the teachers described using data was through classroom discussions and questioning.Three teachers from the charter school districts D (one) and F (two) reported using both to determine if students understood the content.For instance, one middle school teacher at District F recounted how she used questioning as a way to probe for deeper understanding, describing how, "…a lot of my checks for understanding are just asking, what is your evidence for that, what is your reasoning for that?"Similarly, a middle school teacher in District D told us how she made certain to follow up with students during class as they worked on problems and she looked over their shoulders.In these cases, discussions and observations went hand-in-hand in that teachers, as they observed students work, might pose questions to students as a way to address potential student misunderstanding.In this way, interestingly, discussions often played a dual role of instructional strategy as well as formative data collection and use.
Data from performance tasks.Performance tasks require students to "actually perform, demonstrate, construct, develop a product or a solution under defined conditions and standards" (Khattri & Sweet, 1996, p. 3).While they are not unique to science, they are one way that students can demonstrate that they have mastered not just content knowledge but also scientific practices.Only two of the teachers we spoke to, one in District A and one in District B, reported using performance tasks in their classes.The eighth grade teacher in District A recalled that she had her pre-AP students complete performance tasks relatively regularly, telling us, "when we get to physics, pre-AP, they're going to do an independent project, which is to build a rollercoaster.They love it.It takes about a good day and a half.I think for regular for potential kinetic energy, we used some string and balloon and they did that activity."A seventh grade teacher in District B described one performance task that her students completed to help them understand and differentiate animal and plant cells saying, "I think we hit that, well, I know 7 th grade hit that with the performance task.We did the Martian cell and they decided what kind of cell it is and explain why."The teachers did not elaborate on how they used data from these activities.Interestingly, teachers from District E, which was implementing a new project-based learning (PBL) initiative district-wide, did not discuss working on or using data from in-class projects.Data from argumentation.Argumentation is the process by which students construct and evaluate scientific explanations for the phenomena they observe using evidence and scientific reasoning (McNeill & Krajcik, 2008), and could be considered science-specific.We only heard about the use of argumentation as an instructional strategy and as a source of data from teachers in District D. The teachers explained that, similar to student writing, argumentation was a good way to gauge the depth of students' understanding of a concept.For instance, one teacher at District D probed for deeper understanding, describing how, "…a lot of my checks for understanding are just asking, what is your evidence for that, what is your reasoning for that?"Another teacher said she was using the claim-evidence-reasoning (CER) model (Krajcik, McNeill, & Reiser, 2008;McNeill & Krajcik, 2008), which is a model for teaching students argumentation, in her classroom as a way to encourage argumentation and to ask her students to complete more rigorous work.The district was just beginning to encourage the use of argumentation and CER and this teacher told us it was hard for the students, noting, "we are now working on making claim-evidence-response/argument, but they are still struggling with that."

Data Science Teachers Want to Use in their Classrooms
While most teachers reported having the data they needed, in two districts we heard from a limited number of teachers about other kinds of data they wanted to use, or wanted to use more frequently. 8Here we report on what those teachers told us.
Performance assessments.The teachers described performance assessments as way for students to demonstrate and even extend what they had learned in class.An eighth grade teacher from District A told us that, "In the perfect no-STAAR world, we would do instruction where [students] are actively engaged in finding the information and then have them do projects, engineering-based projects, to demonstrate their knowledge and mastery of the concept."Similarly, another group of seventh grade teachers in District A described how they wanted to give their students more projects as assessments in order to have their students apply their knowledge in a new context.Teachers also pointed out that performance assessments were more useful and engaging than multiple choice assessments because the task was more likely to resemble the real-world experiences.A teacher from District F elaborated on this feature, describing an ideal assessment as "…projects where they're building something, creating something.Something that is more real world-applicable.Because to me, when you get a job, you're not taking a test all day, you're working".Several District A teachers came back to this point, noting that students should be 'trying out' the work of scientists and engineers, applying and extending their knowledge, and not just relaying it on paper.
Science labs.In addition to the performance assessments, teachers also told us they wanted to assess their students more regularly through lab assignments.The teachers saw labs as a way for students to demonstrate their knowledge as well as to 'work like scientists'.While many of the teachers we visited and spoke to described doing some kind of lab, at least a few times a semester, many teachers still felt they did not do enough labs and that they did not use the data from the labs.An eighth grade teacher in District B described the challenge of fitting in more labs because of the number of instructional days lost to local and state testing: And so if we could take all those days and combine them up, we'd probably get about 30 days back out of the year, 30 instructional days, not just calendar days.Thirty instructional days that we could get back, and we'd be doing some fun stuff.Using more of our chemicals that we have in the back for chemistry labs, or doing different physics projects.…Several teachers also noted that if they had more science labs, they would have more time to observe and listen to their students, as well as ask them probing questions.
Writing assignments.Several teachers also told us that if they had more time, they would give their students more science writing assignments.The most common explanation for why teachers wanted their students to complete more writing assignments was that writing was a better source of data than most of what they were using.For example, an eighth grade teacher in District A told us that he found his students' writing revealed much about what they knew and did not know, noting that, "I'm reading carefully, if the students write really well, that means the students learned, because students are using their knowledge..." Similarly, a fifth grade teacher in District B explained why she wanted her students to write more, noting that, "Even just reading their journals and their writing you kind of can tell what they're understanding or if they're making connections with what you've been teaching or if they're not making connections." A handful of teachers described writing in science as a way for students to both reflect and develop their own voices, and lamented that they did not have enough time to have their students write more.A 7 th grade teacher from District A described this ideal, saying, I think if I could, I'd try to give more opportunities for story-telling, writing, because, as I'm continuing what I was saying before, they have stories to tell, but without the experience, they don't have the background knowledge to say, OK, I'm going to write about what I've done or what I know.
Another eighth grade teacher from District A noted that students in more advanced science classes were required to complete writing assignments in preparation for Advanced Placement courses in high school, but that in the 'regular' level classes, students rarely had the opportunity to write about what they were doing in class.She said, "Whereas, we've done away with essays, with the exception of Pre-AP.Where being able to express their thoughts, which is a lot more important than a multiple guess test, I guess."

Discussion
In this paper, we have described the kinds of data science teachers in grades 5 through 8 use, the ways in which they use them, and the kinds of data they would like to use more frequently.
Below, we summarize our findings and then discuss implications of the results for research and practice.

Summary of Findings
Based on the data we collected through teacher surveys and interviews, we found that this sample of science teachers used a wide range of data to plan and learn about their students.The most common data they reported using were from local assessments, including campus-and teacher-made quizzes, tests, and homework assignments.This finding was consistent across all of the schools and the grade levels.Teachers described using these local assessment data to make weekto-week and even day-to-day planning decisions for their science instruction.Teachers also told us they used data from district-wide interim or benchmark assessments.Use of these data was more frequent than data from the state assessments, though teachers at the charter schools on average used these data more frequently than the traditional public schools teachers.Teachers primarily used these data to gauge students' preparation for the fifth and eighth grade state science assessments and to provide interventions for those students struggling on any learning objective.
Teachers did use other kinds of data, and wanted to use other kinds of data, in addition to the three described in the paragraph above.According to the survey, use of other kinds of data was quite frequent, but in the survey these data were defined quite broadly.In the interviews, we did hear about other kinds of data use, including from what could be considered science-specific assessments, but it was less frequent.Most teachers we spoke to also seemed satisfied with the data they already were using, and only teachers in Districts A and B discussed other kinds of data they would like to use, such as data from science writing and labs.

Implications and Future Research
Our findings contribute to research on data use by providing additional support for how we currently understand teachers' data use, by describing the ways in which data use might be specific to a particular subject area, and by highlighting where additional research is needed.The teachers who participated in this study used data frequently in their work and they use several different kinds of data.Moreover, their use was sophisticated in that they used different kinds of data to ask and answer different questions.For example, they did not use state achievement data for daily planning, but rather for long-term planning and for reflection on their own teaching.Similarly, they used short, formative assessments for daily planning and student grouping, and not for longer-term planning.This finding supports research into how teachers alter their task or purpose according to the data they are using (e.g., Beaver & Weinbaum, 2015;Marsh et al., 2006).In other words, the teachers we surveyed and interviewed appear to use those data that are appropriate to the particular task.Future research should probe the extent to which this finding is generalizable across other teachers in other contexts, as well as whether what teachers report is represents their actual practice.
The finding that teachers use multiple sources of data also is interesting in light of the data systems to which they have access.Some research around technology suggests that it is deterministic, that is, that its design determines its use (Leonardi, 2012;Orlikowski & Iacono, 2001).The varied data use we observed, however, suggests that teachers' data use has a complex relationship with the technology meant to support it: On one hand, teachers did not rely on data from a single data management system, or even on data from any system, but on the other hand, they did not seem to make much use of data that could not be easily entered into a data system, such as an online grade book.For example, teachers did not talk to us about using rubrics from a performance-based task as a form of data for planning.In other words, we did not hear about data use that was determined by technology, but rather facilitated by it.Future research should explore more carefully the evolving relationship between teachers' data use and the technology that is meant to support it.
In contrast to accounts of data use where teachers reported frustration with the push to use what many considered to be untimely or limited state achievement data (e.g., Beaver & Weinbaum, 2015;Ingram et al., 2004), we did not hear consistent frustration from teachers.Rather, all of the teachers demonstrated an understanding that state achievement data were one part of their broader data use practices, which also included district and local assessments.This is not to say that some teachers were not frustrated with what in some districts appeared to be a narrow focus on using data from the state achievement tests: Teachers in District A in particular expressed disapproval for how the achievement tests and preparation or the tests had crowded other kinds of assessments that might yield richer data about their students' learning in science.Additional research will be needed to understand the extent of this frustration among teachers, as well as the ways in which this frustration is distributed across teachers, schools, and districts.Given that District A had the lowest pass rates during the two years prior to this study, it is plausible to hypothesize that they are under more or different pressure than the teachers in the other districts where students had higher performance; the design of this study did not allow us to link student achievement to data use practices directly, but the data suggest that this hypothesis is worth testing.
We found relatively limited use of science-specific assessment data.As we outlined in our literature review, we expected that we would observe data use that extended beyond what the literature had identified as common 'generic' data use practices across different teachers.Among the assessments from which we hypothesized teachers would use data were hands-on activities and labs, performance tasks, and argumentation activities.In the interviews, a small number of teachers described completing these activities, but either did not mention using data from them or explicitly noted that they did not use data from these activities.It is not clear from this study why the teachers either did not implement these science-specific assessments that are considered best practices within the science education community or did not use data from the science-specific assessments they did administer in their classrooms.Based on previous research into the implementation of inquiry-based science (e.g., Crawford, 2007;Davis et al., 2006;Johnson, 2006), it is possible to speculate that teachers continue to struggle with inquiry in their classrooms and, for this reason, do not have data from inquiry-based assessments, or perhaps are not sure about how to use data from inquiry-based assessments.Other research suggests that novice teachers in particular have relatively unsophisticated knowledge of science content and processes as well as the nature of science, which perhaps would make them uncomfortable with or unprepared for hands-on, inquiry-based instruction and or for anything but multiple choice assessments (Davis et al., 2006).There also is research that would support speculation that a narrow focus on using data from the state assessment and other assessments designed to prepare for the state assessment simply may crowd out more authentic assessment, whether in science (Marx & Harris, 2006) or in other areas (e.g., Palmer & Snodgrass Rangel, 2011).Future research will need to examine more closely the full chain of teacher data use, including assessment, data analysis, and instructional planning and decision making to understand how teachers utilize science-specific assessments and activities and, for those who do not, why.
While the nature of our sample makes it difficult to generalize to other teachers, our findings do point to potential areas of improvement for practitioners.The results suggest that teachers take seriously the mandate to use data regularly to improve their instruction and long-term planning, but they also indicate that their data use could be broadened to incorporate data from science-specific assessments.On one hand, this is a problem of teachers not administering these assessments on a regular basis, but on the other hand our findings suggest that it may be a problem of helping teachers incorporate data from these assessments when they are administered.
Finally, future research on data use among science teachers should address the central limitation of this study: The research design.The teachers who participated in this study did not comprise a random sample and therefore cannot be considered representative of the population of teachers in any of the six districts.For this reason, it is not possible to generalize the findings beyond the teachers we interviewed.A future study would benefit from a comparative design that investigates the extent to which practices are specific to districts.Further, the districts should be selected with an eye towards representing diverse contexts: Rural, urban, suburban, traditional public, public charter, and private schools.From there, a stratified random sampling approach would be a more fruitful strategy for selecting specific schools and teachers.Finally, a more comprehensive data collection strategy combining survey responses, one-on-one interviews, planning observations, and classroom observations would yield richer data.Such a design would permit an investigation of the extent to which the key findings presented here are unique to the contexts and teachers we studied, or are common to science teachers more broadly.
One central place that allows me to keep track of all of my data and also ties into the gradebook so that I'm not exporting data all the time.
The item analysis feature of Eduphoria is useful in assessing what question was missed by most students and which TEKS need to be re-taught in order to reinforce the concepts.Simple practice questions and guided practice activities Sometimes it would help to have an idea of how many minutes/periods (of 45 minutes) a particular activity may take Stemscopes, videos, ppt STAAR data, district curriculum resources

[
T1]: … we use past STAAR [state achievement test data] too to kind of see what the kids have done.

Figure 1 .
Figure 1.The number of teachers citing use of each data source (teacher interviews)

Table 1
Note: Data from Texas Education Agency, 2014 Performance Reports.

Table 2
Participants' teaching experience *Data from Texas Education Agency, 2010.

Table 3
Grades taught *In this category, one teacher was in 4th grade, and two taught high school in addition to middle school.

Table 4
Kinds of data teachers used