Reforms, Research and Variability:
A Reply to Lois Weiner
Lauren B. Resnick
University of Pittsburgh
Citation: Resnick, L. B.
(2003, August 7). Reforms, research and variability:
A reply to Lois Weiner. Education Policy Analysis Archives,
11(28).
Retrieved [Date] from http://epaa.asu.edu/epaa/v11n28/.
Abstract
Lois Weiner (2003) argues that the research reports from High
Performance Learning Communities (HPLC) were biased because of
the close working relationships between the researchers and the
leaders of the Community School District Two (CSD2) reform.
Contrary to any claims otherwise, this
relationship was quite open and acknowledged. The intent of the
HPLC investigation was always to link scholars and practitioners
in a new form of research and development in which scholars
became problem-solving partners with practitioners. There are
important issues about how to profitably conduct such
“problem-solving” research. These issues are worth
substantial attention from the communities of researchers and
practitioners as collaborative research/practice partnerships
proliferate. Serious studies of such partnerships are needed,
going well beyond the anecdotal attacks offered by Weiner in her
article.
|
Dr. Weiner’s (2003) article is at once an analysis of
data on demographics and achievement in Community School District
Two (CSD2) in New York City and an attack on the research
strategy (and by implication the research ethics) of the High
Performance Learning Communities (HPLC) project that I
co-directed, along with Richard Elmore and Anthony Alvarado. Her
paper begins with what can only be construed as a personal attack
on the researchers and practitioners of the HPLC project. The
attack is inherent in the title of the paper, in the way quotes
are used and in the personal story of Weiner’s own interest
that threads through the introduction but is never fully
documented. Nevertheless, I welcome Dr. Weiner’s effort to
provide new data and a fresh perspective on work to which we
devoted substantial professional effort during the period
1996-2001.
Dr. Weiner points out that CSD2 was not a typical urban
district in terms of its demography. She is absolutely right
about this. As we have noted in most reports on HPLC research,
the district sits in the midst of some of the greatest
concentrations of wealth in the nation, and a noticeable
(although minority) portion of middle class parents send their
children to CSD2 schools. At the same time, the district has
large numbers of students of color and families of poverty (as
measured by eligibility for free and reduced lunch), as well as
immigrant students who are in the process of learning
English.
Dr. Weiner’s reports of the demographics of the district
as a whole and schools within the district roughly match the data
we have collected and reported in several papers over the course
of our five-year study. Perhaps most important to her argument
is that schools in CSD2 did not uniformly represent the
demographics of the district as a whole. That is correct. CSD2
contained “rich” schools and “poor”
schools, schools with very few children of color and others
almost entirely filled with minority students. The variability
among schools was never in question. What is important to ask is
whether CSD’s unique (at the time) system of curriculum
leadership and professional development within schools led to
learning gains—especially in “high need”
schools.
Dr. Weiner addresses this question by comparing CSD2 schools
with relatively high need ratings (7 schools in all) with a
single high need school in District 25 in a single year on
a single test in one subject matter. Overall, the CSD2 schools
did not outperform the District 25 school. Dr. Weiner seems to
imply that we should therefore conclude that CSD2’s program
of curriculum and professional development was not effective.
Possibly—but it pays to look at more evidence than she
provides. HPLC conducted a number of analyses of CSD2 academic
performance, both for the district as a whole and
school-by-school. Summaries of most of these analyses appear in
the 2001 Final Report of the project.(Note 1)
One HPLC analysis examined changes over time in reading and
mathematics during the period 1992 to 1998—a period in
which the CSD2 curriculum and professional program was being put
into place and expanded, and during which a stable test in each
subject was being used in New York City. In 1993—the first
year of New York’s renormed math achievement test, just
under 70% of CSD2 students were at or above grade level in math;
in 1998 , about 82% of CSD2 students were at or above grade
level. The story is similar for reading: Scores rose from just
under 60% at or above grade level in 1992 to about 72% in
1998.
This gradual rise in overall achievement could have resulted
from a change in overall district demographics resulting from
more middle class students attending CSD schools. But it did
not. According to our data, during this period the percentage of
students in the district eligible for free or reduced lunch
remained stable at about 53%.
The overall improvement also might have resulted mainly from
nudging students already near “grade level” over the
mark into performance level 3, leaving the students in greatest
need behind. To check this, we analyzed achievement
quartile-by-quartile wherever such data was available in several
successive years on the same test. In the period between 1996 and
2000, the proportion of CSD2 students testing in the bottom
quartile in reading fell a bit every year—to a low of just
over10% by 2000. Math drops were smaller, perhaps because the
CSD2 math curriculum and professional development system was
introduced later and might not yet have fully taken root.
Unfortunately, we were not able to obtain detailed enough data
on other districts in New York to make comparisons with them.
But we were able to use the variability among schools within CSD2
to examine whether the leadership’s curriculum and
professional development system can be credited with raising
achievement, especially for children with the greatest academic
need. Using questionnaires and ratings of classroom
instructional quality to assess the extent of engagement in the
CSD2 program, our studies showed that deep teacher engagement in
professional development and faithful implementation of the
district’s literacy and math programs both raised overall
achievement and reduced the connection between achievement and
socioeconomic status. It is interesting that a similar finding
for mathematics, using a different curriculum but a similar
professional development system, was reported for the Pittsburgh
Public Schools during the period 1996-98. (Note 2)
We are still left with the question of whether there was
something special about the mix of students and schools in CSD2
that might have made it easier than elsewhere to effect the kinds
of learning changes that the leadership sought. Here Dr. Weiner
makes an important contribution in calling to attention the fact
that there are important cultural differences among minority and
English learning groups. As she points out, several of the high
poverty schools in CSD2 were Chinatown schools. She suggests
that Chinese immigrants are, in Ogbu’s terminology,
”voluntary” immigrants, and their children perhaps
more likely to participate actively in the opportunities offered
by schools. This and other possible cultural difference between
the Chinatown schools and other high poverty and minority schools
certainly warrant further investigation. Meanwhile, however,
Weiner’s data make it clear that Asian students and schools
were not the only minorities who did well in the CSD2 reform
effort. Note, for example that the school with the highest
overall academic performance in Dr. Weiner’s Table 5 (PS
198 in CSD2), had a population of 52% Hispanic and 26% Black
students. Thus, interesting as the “Asian question”
is, does not call the overall record of CSD2 into question.
This brings me back to what appears to be Dr. Weiner’s
main point: That the research reports from HPLC were biased
because of the close working relationships between the
investigators and the leaders of the CSD2 reform. There was
never a secret about this relationship. Indeed the intent of the
HPLC investigation from the start was to link scholars and
practitioners in a (then) new form of research and development in
which scholars became problem-solving partners with practitioners
and practitioners accepted the responsibility of collecting
evidence in as unbiased a manner as possible, using it to refine
and—when necessary—alter their theories of action.(Note 3) Our goal
was to deeply document, analyze and understand the actual
practices of the CSD2 reform. We conducted extensive interview
and observation studies of professional development and classroom
practice as well as the studies discussed here that examined
impact on student learning. Variability—among students,
teachers and schools—was a central object of investigation
and analysis throughout.
HPLC was not a simple undertaking and there were difficulties
encountered along the way. Dr. Weiner points to one of
them—an unwillingness of many teachers to return a
questionnaire when they felt that anonymity could not be
guaranteed because of the relationship between the researchers
and the district leaders. Problems of that kind are easy to
recognize and to address. There are in addition deeper issues
about how to profitably conduct such
“problem-solving” research, however—for
example, the subtle ways in which question are formulated, or
avoided, because of the common perspectives that emerge in long
collaborations. These issues are worth substantial attention
from the research and practice communities as collaborative
research/practice partnerships proliferate. Serious studies of
such partnerships are needed, going well beyond the anecdotal
attacks offered by Dr. Weiner in the opening sections of her
paper.
At the same time, it is critically important to conduct more
“arm's length” research on reform programs or other
interventions that appear to be succeeding. The HPLC project did
not claim to be an arm’s length investigation, but all
parties to it would welcome such investigations. CSD2 as such no
longer exists (having been absorbed into a much larger
instructional Region in New York’s recent school
reorganization). But many of the ideas pioneered in CSD2 are now
being tried in districts across the country. There is, thus,
plenty of opportunity both for collaborative problem-solving
research of the HPLC variety and more arm's length evaluations.
Such studies will tell us what aspects of the CSD2 effort can
“travel” well to other environments, what effects
they have on various populations of students and educators,
and—of utmost importance—what revised or totally new
theories of action are likely to meet the demands for increased
academic achievement across a broad spectrum of the school
population.
Notes
1. High
Performance Learning Communities Project: Final Report
(September 15, 2001). Learning Research and Development Center,
University of Pittsburgh, September 15, 2001. (Available on the
HPLC website: http://www.lrdc.pitt.edu/hplc)
2. Briars, D.,
& Resnick, L. B.. Standards, assessments – and what
else? The essential elements of standards-based school
improvement. CRESST Technical Report, 2000.
3. This form of
Problem Solving Research and Development was recommended
in a National Academy of Education report to OERI as one
important means of bringing research and practice into closer
interaction See Brown, A.L. & Greeno, J.G. (Eds.)
Recommendations regarding research priorities: An advisory
report to the National Educational Research Policy and Priorities
Board. National Academy of Education. Spring, 1999.
Reference
Weiner, L. (2003, August 7). Research or
“cheerleading”? Scholarship on Community School
District 2, New York City. Education Policy Analysis Archives,
11(27). Retrieved [date] from
http://epaa.asu.edu/epaa/v11n27/.
About the Author
Lauren B. Resnick
Director and Senior Scientist
Learning Research and Development Center
University of Pittsburgh
3939 O'Hara Street
Pittsburgh, PA 15260
Phone: 412-624-7020
Fax: 412-624-9149
Email: resnick+@pitt.edu
Lauren B. Resnick is an internationally known scholar in the
cognitive science of learning and instruction. Her recent
research has focused on school reform, assessment, effort-based
education, the nature and development of thinking abilities, and
the relation between school learning and everyday competence. Her
current work lies at the intersection of cognitive science and
policy for education. Dr. Resnick founded and directs the
Institute for Learning, which focuses on professional development
based on cognitive learning principles and effort-oriented
education. She is co-founder and co-director of the New Standards
Project, which has developed standards and assessments that have
widely influenced state and school district practice. Resnick was
a member of the Commission on the Skills of the American
Workforce and served as chair of the assessment committee of the
SCANS Commission and of the Resource Group on Student Achievement
of the National Education Goals Panel. She has served on the
Commission on Behavioral and Social Sciences and Education and on
the Mathematical Sciences Education Board at the National
Research Council. Her National Academy of Sciences monograph,
Education and Learning to Think, has been influential in
school reform efforts, and her widely circulated Presidential
Address to the American Educational Research Association,
"Learning In School and Out," has shaped thinking about youth
apprenticeship and school-to-work transition. Dr. Resnick is
Professor of Psychology at the University of Pittsburgh, where
she directs the Learning Research and Development Center.
Educated at Radcliffe and Harvard, she received the 1998 E. L.
Thorndike Award from the American Psychological Association and
the 1999 Oeuvre Award from the European Association for Research
on Learning and Instruction.
|
