Perspectives on evaluation from curricular contexts

Educational issues in nations throughout the world center on teacher and student evaluation, leading to much consternation. The root of debate stretches to curriculum questions that directly address matters of worth or value. Examining evaluation of education at any level through curricular lenses increases its complexity. First, complexity is approached by focusing on three broad categories that can be framed as questions: What are value considerations in foundations of curriculum? What is worthwhile for subsequent generations? How should we evaluate extant impacts of curricular venues? Second, perspectives on the complexity of evaluation based in curriculum studies are advanced through responses from five orientations to curriculum: Intellectual Traditionalist, Social Behaviorist, Experientialist, Critical Reconstructionist, and Postmodern Global Anti-Imperialist. Each offers a perspective worth considering by those who wish to improve curriculum and its evaluation.

The tack I take here, in presenting curricular categories and orientations, will involve roleplaying as guest commentators, characters, or guest speakers, an approach based on my historical work (Schubert & Lopez Schubert, 1980;Schubert, Lopez Schubert, Thomas, & Carroll, 2002).I have used such role-playing on many occasions in an attempt to clarify topics of concern through lectures (e.g., Schubert 2004Schubert , 2015)), articles (Schubert 1996(Schubert , 1997)), and self-critiques at the end of each chapter of my synoptic curriculum text, Curriculum: Perspective, Paradigm, and Possibility (Schubert, 1986(Schubert, /1997)).In each lecture or written version I conjure guest speakers through whom I strive to deliver a fair version of each of the categories and orientations.I do not intend the role-played characters to be caricatures.Instead, I attempt to have each speaker represent the best of each position.Some key figures, such as John Dewey or Ralph Tyler, fit in more than one category or orientation.I have found the speakers to be helpful in dealing with many audiences, from beginning graduate students (even undergraduates) to practicing teachers, school leaders, educational evaluators, policy makers, teacher educators, curriculum scholars or researchers, and local school council members (governing boards of schools in Chicago).Although I adapt the rendition for different audiences, I use essentially the same routine, because I try to uphold a common faith (Dewey, 1934) that enables concerned individuals and groups, including teachers and students, to understand curricular issues and complexities.

Addressing Three Primary Curricula Questions: Explaining the Complexity of Evaluation
What are Value Considerations in Foundations of Curriculum?
From a curriculum standpoint evaluation must be a central thread of the entire process of curriculum development, design, implementation-and the whole realm of curriculum studies (e.g., Caswell & Campbell, 1935, 1937;Connelly et al., 2008;Eisner, 1979;He et al, 2015;Jackson, 1994;Kridel, 2010;Marsh & Willis, 2007;Marshall et al., 2007;Molnar & Zahorik, 1977;Pinar, 2006Pinar, , 2012;;Pinar et al., 1995;Schubert & Lopez Schubert, 1980;Schubert, 1986Schubert, /1997;;Skilbeck, 1984b;Smith et al., 1957;Tanner & Tanner, 1975;Tyler, 1949;Walker, 1990).The intersecting concerns of curriculum studies and social, historical, and philosophical foundations of education are palpable (Phillips, 2014;Provenzo, Renaud, & Provenzo, 2009;Stanley, Smith, Benne, & Anderson, 1956;Tozer, Gallegos, & Henry, 2011).These concerns pertain to all choices about assumptions (historical, philosophical, social, cultural, political, economic, and psychological) regarding decisions about the diverse range of ways to achieve purposes or objectives through curriculum development (selection of learning activities, experiences, or opportunities, and their organization, sequence, learning environments, and instructional strategies).All choices one makes, every action engaged, involve values; thus, they pertain to evaluation.Even if the values are not explicitly addressed, they govern by default, chance, or expediency.How can evaluation at the level of grading and testing be pursued defensibly if it is not based on conscious consideration of questions about purposes, learning experiences, and organization-key elements of the Tyler's (1949) rationale for curriculum development?
What is Worthwhile for Subsequent Generations?
Through provision of key philosophical writings on education in Robert Ulich's (1954) surveys of literature, and those of others, I deduce that the primary curricular question is: What is worthwhile?I suggest that to conscientiously address questions about what is worthwhile was the central concern of those who became the first curriculum scholars.Such questioning can be traced back to Herbert Spencer (1861) who asked, "What knowledge is of most worth?"Of course, his social Darwinist perspective made many dissenting scholars modify this question by asking: What is worth knowing?They did not want to be forced to decide the most worthwhile, and they surely did not want to promote the self-preservation answer in Spencer's distortion of Darwin.Others have asked, for instance: What is worth needing, experiencing, doing, being, becoming, overcoming, contributing, sharing, just plain wondering, and more (see Schubert, 2008Schubert, , 2009.These topics prompt consideration of much broader images of what should and can be informed by assessment and evaluation.Questions of worth are deeply connected with considerations of the meanings of good education, as Gert Biesta (2010) contends, in an age of measurement, especially regarding matters of ethics, politics, and democracy.These points of inquiry evoke questions that are metaphysical, epistemological, axiological, ethical, aesthetic, political, and more.To what extent should those who make choices about testing and grading explicitly address such philosophical matters, since those matters undergird decisions made and actions taken?How should we evaluate diverse extant curricular venues?
Curriculum is often equated with subject matter areas of study, textbooks, or purposes set forth in policy statements as objectives (i.e., the intended curriculum), or more recently the narrower strand of that which is tested.As worthwhile as it is to evaluate the intended curriculum, other curricular venues must be addressed (Schubert, 2008a, pp. 407-412).For example, there are the taught curriculum, the hidden curriculum, the experienced curriculum, the embodied curriculum, the null curriculum, the extra-curriculum, the tested curriculum, and the outside curriculum.While the intended curriculum might be explicitly stated in national or state mandates, it is not the same as the taught curriculum, or that to which students are exposed by teaching.Should not the differences between the intended and taught curricula be evaluated?Moreover, another curriculum is enacted via the mannerisms and personalities of teachers; it is often referred to as the hidden curriculum, and as Philip Jackson (1968) noted, it develops through the routines and procedures of the daily grind of classroom existence and school organization.Should evaluations address what students learn from this image of hidden curriculum?Further, at a deep level, the hidden curricula of structural aspects of any society's economic, ideological, racial or ethnic, political, and cultural life are explicitly and implicitly mediated through all institutions of that society, including school (e.g., Anyon, 1980;Apple, 1979Apple, /2004;;Giroux & Purpel, 1983).How can evaluation take these powerful forces into account, and not simply accept the dominant view of the oppressors, colonialists, or conquerors who often make curriculum to enhance their interests?Too, there is the null curriculum (Eisner, 1979) or that which is simply not designated to be taught or is given short shrift through cut-backs in times of financial difficulty or ideological challenge.Should not evaluators want to identify null curricula in order to broaden their portrayals?Together, these curricular venues amass to create the experienced curriculum or all that happens in a student's life under the auspices of the school (Smith et al., 1957).Whatever students retain in their cognitive, affective, social, and psychomotor repertoires that can be recalled from the experienced curriculum might be called the learned curriculum.Further, the embodied curriculum refers to the perspectives, life decisions, and subsequent actions that constitute growing identities of students as they move through life.What, then, would it require in terms of finance, time, and effort to adequately portray the kind and quality of curricular influence that schools provide?Curriculum is not merely a set of knowledge and skills to be covered, and the phrase cover the curriculum is so frequently uttered.Curriculum, when considering this diversity of venues, is something to be carefully uncovered as well.It is curious that the long-term embodied curriculum (knowledge, ideas, skills, and dispositions, which are lived) is rarely studied.What if a poem learned in school suddenly becomes meaningful to a life experience 20 or more years later?Is that captured by our evaluations?Should it be?
Making this even more complex, all of this gestalt of curricular venues is continuously mediated by a host of outside curricula (Schubert, 1981(Schubert, , 2010) ) or curricula of homes, families, neighborhoods, communities, cultures, languages, ethnicities, non-school organizations (e.g., churches, museums, scouts, dance and music studios, gangs, clubs), and mass media (television, popular music and art, movies, plays, videogames, the Internet, and a host of social media).The whole realm of public pedagogy (Giroux, 2000;Sandlin, Schultz, & Burdick, 2010) exposes the centrality of the outside curriculum, and implores curriculum evaluators to include it as they determine student needs and interests.As educators design curriculum, and as students are exposed to curriculum, teaching, and schooling situations, all of these outside curricula continuously influence the lived interpretations of what is experienced.This is why Joseph Schwab (1970Schwab ( , 1973) ) argues that curriculum needs to be seen through a dynamic interaction of commonplaces (teachers, learners, subject matter, and milieu) that continuously need to be monitored and evaluated in the course of curriculum enactment.So, what then should be included in evaluations of the experience of this myriad ecology of curriculum in our lives?What of it does current evaluation or assessment address?Who benefits from that which is evaluated or left out, and who is harmed?What does evaluation miss, that should be evaluated?

Guest Speakers from Diverse Curriculum Orientations
As the moderator, I have asked each of the following speakers to address: How do you characterize your orientation to curriculum studies, and thus to evaluation?What traditions do you draw upon?How do you build upon foundations of education?What is worthwhile?What evaluation do you use to determine what you deem worthwhile?Who, from the legacy of curriculum studies, do you draw upon and recommend to others?What kinds of curriculum do you recognize as important to evaluate?To what should evaluators attend?Of course, the speakers will not necessarily discuss these questions in systematic order.Nor will they explicitly address all of them.Instead, they are encouraged to work responses to them into their presentations.(I will play the role of a representative of each orientation.In so doing I try to exemplify objectivity, since (through me) each speaker will look and sound alike!Since I act as a character, speaking, the reader might imagine the comments are in quotations.The reader should also note that the written version has more references to pursue for elaboration, only a few of which are weaved into the acted version to remain in harmony with usual spoken commentary.

Speaker 1: Intellectual Traditionalist
I characterize my curricular orientation as an advocacy of liberal arts and sciences, derived from traditions espoused by such scholars as William T. Harris, William C. Bagley, Robert M. Hutchins, Mortimer Adler, and Robert Ulich.The most worthwhile contribution of this form of curriculum is acquiring insight into or understanding of great ideas and mysteries and events of life (Ulich, 1955) through great works.After considering over 100 great ideas, Adler (1981) synthesizes them into truth, beauty, goodness, liberty, equality, and justice.Stated another way, in a response to Ralph Harper's (1955) pioneering article on existentialism in education, Ulich (1955) says that education should be fundamentally about the "great mysteries and events of life: birth, death, love, tradition, society and the crowd, success and failure, salvation, and anxiety" (p.255).Thus, a key issue in evaluation is the initial determination of what constitutes great works, ideas, mysteries, and events that should be made accessible to students.Do evaluators typically address the reasons for determining curricular purposes?Does evaluation address why, how, and by whom determinations are made?I am convinced that experts in each field should make decisions about what and why, and that the teachers imbued with the art of teaching (Broudy, Smith, & Burnett, 1964;Broudy & Palmer, 1965;Highet, 1950;Rubin, 1984) should assess and implement the how.Moreover, I also agree with William C. Bagley (1934) who advocated that education provide a core of cultural knowledge that provides subsequent generations of human beings with common interests and concerns.Relative to evaluating student learning, teachers should become connoisseurs of student discussions, presentations, performances, and compositions in an effort to judge the ways in which they incorporate the ideas deemed worthy of teaching and learning.Outside judges from state and corporate sectors may offer useful topics, via curriculum materials and tests, but these should not overpower the judgment of good teachers.This is because the greatest outcomes of philosophy (meaning inquiry generally), as Whitehead (1938, p. 168) asserted, begin and end in wonder, though the hope is that it is a deeper wonder; or as Huebner (1975, p. 219) put it, the best educators stand in a cloud of unknowing, since the most insightful humans rarely claim certain knowledge.Good teachers are in a position to judge if students are achieving productive uncertainty (Dewey, 1929), cultivating humanity (Nussbaum, 1997), and composing a life (Bateson, 1989).
I have found that good practice for appreciating this orientation is for participants (students or educators) to share with each other a work that they consider to be great and to portray how it has had profound impact on their outlook.I ask them to explain how a great work's author or artist reaches them with meaning and inspiration when the author or artist does not know them personally.How is an author or artist a teacher, I challenge students, and how is their work a curriculum?How are the consequences of artists' works evaluated, and how should we evaluate work of teachers who are deemed artists?How can teachers come to know the impact of a work or educational experience on a student?How can they perceive students as their works of art?Moreover, could students see themselves as works of art continuously created and recreated throughout lifetimes?How might evaluation that addresses this further the educative process?Speaker 2: Social Behaviorist I am sometime called a conceptual empiricist (Pinar, 1975) or a proponent of social efficiency (Kliebard, 1986).I am convinced that science and objective analysis are the most important inventions of the past several centuries.The great advances in medicine, architecture, engineering, communication, transportation, and other spheres of life are results of scientific inquiry and analytic thought.Thus, it was thought in the early 20 th century that this new thinking would contribute advancements in education, as well.Evidence around the turn of the 20 th century was provided by William James and Edward L. Thorndike, who published research to empirically discredit faculty psychology that previously dominated 19 th century educational thought and paved the way for a social efficiency movement.Robert W. M. Travers (1980) documented many ways in which research contributed to clarification of needed directions in educational policy and practice.This scientific approach to curriculum held that subject matters derived from disciplines of knowledge cannot be accepted as beneficial cart blanche unless their benefits can be defended with reason and evidence.
With an emergent faith in I.Q.and related testing as a veritable blood test of educational health or illness, my social efficiency ancestors, Franklin Bobbitt (1918Bobbitt ( , 1924) ) and W. W. Charters (1923), called for curriculum design that worked toward creating skills, knowledge, and behaviors that explicitly forged better members of society.Bobbitt, for instance, developed activity analysis in which successful persons were surveyed and their capabilities catalogued; for him curriculum-making became translation of knowledge and skills of successful persons into objectives for students, so that they, too, could become successful individuals.Similarly, Charters derived purposes from study of ideals of successful citizens, and then constructed curriculum to arrange learning activities to develop students who could live such ideals.
Although today an emphasis on purposes or objectives seems outmoded to some critics, Ralph Tyler (1987) reflected historically that his call for clarifying curricular purposes was novel in his early consulting ventures in the 1920s and 1930s, when educators could not give a rationale for perpetuating subject matter content, other than that it stood the test of time.Tyler's Basic Principles of Curriculum and Instruction (Tyler, 1949), known as The Tyler Rationale, was fashioned as a result of consultation, study of many curriculum orientations, and work in the renowned Eight Year Study (Aikin, 1942), a longitudinal comparison of progressive and traditional curriculum relative to student outcomes.This study showed superiority of progressively taught students, by using a diverse array of evaluation methods.Arguably the Tyler Rationale had greater influence on curriculum policy, curriculum guides, lesson plan structures, and teacher education than any other book from the emergent curriculum field.After all, Tyler was an educational advisor to six U.S. presidents, and he was an exemplar among educational researchers and evaluators.Today's widely used manuals for curriculum development by Grant Wiggins and Jay McTighe (2005), promulgated by the Association for Supervision and Curriculum Development and their so-called backwards design are variations on Tyler's rationale for curriculum, though many users are not fully aware of this history.
Returning for a moment to the Eight Year Study (1933-41), especially the third volume (Smith & Tyler, 1942), a pioneering work on evaluation, we can see one of the most comprehensive compendia of evaluation procedures for complex purposes.In efforts to appraise student progress the evaluation team developed a host of qualitative and affective measures.Embedded in the Eight Year Study are levels of cognitive and affective objectives later developed by Benjamin Bloom (a renowned student of Tyler who worked with him on the Eight Year Study), David Krathwohl, and others (Anderson & Sosniak, 1994).This early work broadens perspective and looks at both intended and unintended outcomes, as does other early work on curriculum evaluation, for example: formative and summative evaluation (Bloom, Hastings & Madaus, 1971Scriven, 1967); a call by Robert Stake (1967) to look at a multifaceted countenance of evaluation's content that includes antecedents, transactions, and outcomes; Daniel Stufflebeam's (1971) entreaty to evaluators to address context, input, process, and product; and a host of affective evaluation materials and approaches collected by Beatty (1969).Such work reflects insights from a recent portrayal of contributions through stories about the Eight Year Study by Craig Kridel and Robert Bullough (2007) wherein they recognize that curriculum evaluation entails experimental work of educators and collaboration of all engaged as stakeholders in deeply contextualized educational settings.
Even though there is much emphasis on evaluation in educational policy and practice today, the theoretical constructs of these authors from a half century or more ago deals with much greater complexity than does much of the most popular and superficial means-ends evaluation of today.Today's highly politicized evaluations, wherein evaluation instruments are too often surrogates for purposes, need to be contrasted with Ralph Tyler's (1949Tyler's ( , 1987) ) frequent admonition that evaluation should be done to improve curriculum and instruction.Moreover, as mentioned in the previous chapter, another of Tyler's renowned students, Lee J. Cronbach (1963), proposed that evaluation should be for course improvement.Some would consider Cronbach and Associates (1980) as the definitive statement that integrated and expanded the best of this early phase of curriculum evaluation.In the decades that followed, many continued to build on this work, especially Australian scholars, such as Malcolm Skilbeck (1984aSkilbeck ( , 1990) ) and Stephen Kemmis (1982;Kemmis & Stake, 1988).The work of these scholars has enabled me to suggest that the fullness of a curriculum perspective on evaluation means that it is necessary to perceive evaluation as being part of every aspect of determining purposes, and selecting experiences that facilitate acquisition of purposes, creation of organizational patterns (classroom environments, sequential patterns, school plant design, grouping arrangements, curricular structures, and more).By comparison, the evaluation of students, teachers, schools, and nations using only external tests seems narrow indeed, even though I am an advocate of beneficial uses of testing.Quite clearly, I am declaring that testing needs to be seen as part of a larger picture of evaluation.Testing is not to be equated with evaluation or even assessment.
To encourage educators to think as Social Behaviorists, I ask them to select a skill or disposition that they would like to see fostered among students and to then frame it, based on the best research available, in terms of purposes and objectives, selection of learning experiences or opportunities that help foster the purposes, organizational patterns through which to convey purposes via instructional strategies and materials, and then to develop modes of evaluation (marshalling evidence) to determine the extent to which purposes are obtained.The evaluation must account for all of the purposes as well as for unintended outcomes, and must be consistent with learning experiences selected, organizational patterns, instructional strategies, and learning environments.

Speaker 3: Experientialist
As long as you were speaking of Ralph Tyler, I say he was as much connected with the Experientialist orientation as with the Social Behaviorists.While Tyler (1949) frequently used the term behavior, he was not a behaviorist in the sense that B. F. Skinner was.Tyler said that his use of behavior was a brief substitute for saying think, feel, and do each time.Moreover, Tyler's use of the term experiences bespeaks a Deweyan (John Dewey, 1859Dewey, -1952) ) orientation, one that differs from activities (Social Behaviorist) and content or subject matter (Intellectual Traditionalist).Dewey (1916) defined education as reconstruction of experience to guide subsequent experience (p.76) and later elaborated the intricate interface of education and experience (Dewey, 1938).Tyler studied Dewey preparatory to working as Director of Evaluation for the Eight Year Study, in which the experimental group (experiencing Deweyan progressivism) advocated learning that built on student needs and interests through continuous collaboration and experimentation (Aikin, 1942;Kridel & Bullough, 2007).Tyler's experience on the Eight Year Study Evaluation Team provided a foundation in his Basic Principles of Curriculum and Instruction (Tyler, 1949) for emphasis on two highly progressive threads woven throughout his rationale, which (like the Eight Year Study itself) are too often forgotten.Tyler (1977) re-elaborated them in a reflective retrospective, referring to them as: i) the active role of learners, and ii) the multiple non-school curricular areas.The latter was quite similar to the areas of education that influence students outside of school as characterized as dimensions of public education by Lawrence Cremin (1976), and elaborated as the outside curriculum, noted above (Schubert, 1981(Schubert, , 2010)).Some argue that other intellectual ancestors of the experientialist orientation include Francis Parker (1837-1902), Maria Montessori (1870-1952), Jane Addams (1860-1935), Ella Flagg Young (1845-1918), with roots to Froebel (1782-1852), Herbart (1776-1841), Pestalozzi (1746-1827), and Rousseau (1712-1778).Of Tyler's generation, L. Thomas Hopkins (1929) developed Experientialist curriculum principles through reflection on questions of practicing educators in schools where he consulted throughout the USA, and advocated (1) curriculum integration of subjects around student interests and concerns (Hopkins, 1937), (2) expanded integration based on embryonic research to embrace the democratic process (Hopkins 1941), and (3) that education should be focused on a more integrated self, facilitated by organic groups (as opposed to mere aggregates) in homes as well as in schools (Hopkins, 1954).The emphasis on student interests as a key to needs and a spur to effort (which Dewey, 1913, originally wanted to call will) was consistently a thread of Hopkins's work.Too, it was a primary dimension of the experimental or progressive schools in the Eight Year Study, as Kridel and Bullough (2007) also note the centrality of needs in the educative process.Dewey, Tyler, and Hopkins realized that interests of the moment can be valuable clues to deep and pervasive human interests, the pursuit of which helps students meet needs to consciously develop the project of composing their lives, as phrased so well by Mary Catherine Bateson (1989Bateson ( , 2010)).This hearkens back to the Intellectual Traditionalist focus on great ideas (Adler, 1981(Adler, , 1982)), especially life's great mysteries and events (Ulich, 1955).The original core curriculum, as developed by Harold Alberty (1947), ironically differs diametrically from the imposed and predetermined common core of recent years.Alberty's core, derived from Boyd's (1937) philosophy of living a democratic theory of education (which was close to Dewey's participatory democracy, 1916Dewey's participatory democracy, , 1938)), perceived study of key social interests or problems as the core from which myriad realms of knowledge could be accessed and learned.Beane (1997) revived and renewed this orientation to integrated curriculum decades later.
To help educators become acclimated to the Experientialist orientation, I often ask them to construct a pathway that visually represents their life journey thus far, and to indicate turning points or milestones in the development of personal perspectives that guide their lives.In reflecting on milestones of their pathway, they begin to see education, as expressed by Ronald Swartz (2016), as an informal "philosophy of education for personal responsibility."Such a pathway, I suggest, is a student's most valuable curriculum, derived from both inside and outside of school.As educators discuss their journeys, they begin to see the need to view their students as continuously influenced by multiple curricula simultaneously.Educators need, therefore, to evaluate curricula they intend to teach in light of the diversity of other curricula of student lives.Then educators need to address what it would mean to evaluate that curriculum relative to short-and long-term consequences of enacting it.Many of the evaluation instruments (qualitative and quantitative) developed in the Eight Year Study by Tyler and his evaluation team (Smith & Tyler, 1942) could serve as prototypes for complex evaluation lenses needed to understand experienced curricula today.An evaluation question that cannot be neglected, as I see it, is how can evaluators provide insight for teachers and students to continuously re-create their lives, since composing one's life should be the key work of education?Speaker 4: Critical Reconstructionist My primary focus is on inequities and injustices perpetrated through educational institutions to those who live in poverty or suffer from persecution or oppression due to prejudice relative to race, class, gender, place, heritage, ethnicity, sexual orientation, cultural practices, religion or beliefs, language, appearance, credential, (dis)ability, or other markers of diversity.My point is that the Intellectual Traditionalist and the Social Behaviorist may differ from each other in what they choose to advocate, but neither of them engages students, parents, and communities in serious evaluation of curriculum matters.They remain essentially authoritarian.The Experientialist wants students, parents, and communities to be involved in curriculum development and to participate in processes of setting purposes; however, they rarely are consulted during curricular planning or enactment.
They definitely are not involved in establishing endpoints that could be a basis for summative evaluation.Further, decision and action during enactment seldom allow students and practicing educators to interpret formative evaluation as changing direction rather than getting back on a preordained track, when it is clear that forging ahead in a detrimental direction is not educationally sound.While understanding and agreeing with this, the Experientialist is too naïve to realize that educators in an acquisitive society (Dewey, 1933), cannot simply decide to build on student experiences and interests, or those of their parents and communities, when governmental and corporate greed is the ideological and economic power behind schooling which purports to be education (Counts, 1932, Schubert, 2009b).
The disparity between haves and have-nots must be exposed; the chasm is immense between those who are deemed worthy of being authoritative in creation and development of curriculum, i.e., those whom Paulo Freire (1970) calls oppressors and oppressed local administrators, teachers, parents, and community members who are expected only to implement or support it, or the even more oppressed students who are tricked and coerced to receive and internalize it.Influenced by Karl Marx (1818-1883), Erich Fromm (1900-1980), Antonio Gramsci (1891-1937), and Latin American liberation theologians, among others (see Lake & Kress, 2013), Paulo Freire (1970Freire ( , 1998) ) called for teachers to be cultural workers who build curricular or pedagogical experiences through a dialogic exchange with students.Thus, evaluation becomes the evaluation of artifacts and events from student experience in an effort to enable them to become literate by transforming the word and the world through praxis (politicized theorizing in action) and by striving to overcome the kinds of inequities noted above.
What would evaluators and evaluation have to do and be in order to facilitate what I am advocating?Evaluation in the Critical Reconstructionist orientation is an effort to discover radical possibilities as characterized by Jean Anyon (2004) and to challenge societal disparities leading to hidden curriculum identified in her earlier work.In the early 1980s Anyon (1980) found that students from lower socioeconomic classes were taught to follow the rules, middle-class students learned to give answers wanted by superiors, professional-class students were taught to be creative without disturbing ruling class authority, and upper-class students were facilitated in their longstanding role of manipulating other classes and the system itself to their will.Freire (1970) criticized this banking system of education, and called for problem posing pedagogy as an alternative.Put another way by Michael Apple (1979Apple ( /2004), educators need to address ways in which conventional curriculum reproduces ideological structures; thus, they must re-evaluate official knowledge (Apple, 2000) and seek democratic alternatives (Apple & Beane, 2007).
In an effort to develop empathy of educators with the plight of the oppressed, I often ask participants to reflect on or share instances of discrimination that they have faced, how they felt and acted or performed.I strive to help them frame the curriculum questions about what is worthwhile by bringing to the fore questions such as: Whose notion of worth is promoted?Who benefits?Whose image of worth should be promoted?Who should benefit?I sometimes ask educators to make a list of six things they hate to do and cannot do well; then I ask them to imagine having to go to a place where they are judged on their capacity to do those things-five days a week, about 200 days per year, for 13 years, equivalent to K-12 schooling.I follow by asking how they would change that distressing situation to maximize the experience of fairness and capacity building for all or most students and teachers involved.I also ask them to consider instances when they experienced or witnessed discrimination or inequity and how such situations could be prevented or overcome.With James B. Macdonald (1977) I implore them to address: "What is, or should be, curriculum talkers' and workers' idea of goodness?What fundamental values inform our own activity, arise out of that activity?"(p.19), and … "What is the meaning of human life" … "and how shall we live together?"(p.20).

Speaker 5: Postmodern and Global Anti-Imperialist
All previous speakers share a common flaw; they wittingly or unwittingly see their own position as the master narrative.They are also often centric in favor of a region, nation, or epistemological orientation.This is why I draw on postmodern perspectives and combine them with anti-imperialism.I urge us to evaluate curriculum, teachers, students, and all of educational experience relative to imperialism in global proportions.By this, I mean a global imperialism of a kind of coalescing of conquerors-governmental, corporate, and other bastions of oppressive wealth.One of the key admonitions I take from postmodern perspectives is that there should be no master narrative that dominates, and that we should evaluate with the goal of fair representation of narratives from all stakeholders in providing perspectives on what is or ought to be in any given educational situation.In curriculum work, the often dominant Western, white, male educational ideals need to be balanced with a host of choices as evaluated by non-dominant groups (Doll, 1993;Slattery, 2006).The latter reap consequences of any given educational experience, especially in diverse nations.A prime question for evaluation, then, is how to plan, act, and evaluate consequences of action on insights of all stakeholders in any educational situation.All involved should decide together.The anti-imperialist global perspective that I advocate is based on my observations that the viewpoints of conquerors dominate all spheres of life, including and especially educational institutions.Let's begin to look at this through the eyes of one dominant (and often dominating) nation, the United States, even though many others can be criticized from a similar vantage point.
Responding to this situation, Joel Spring (2010) has critiqued the master narrative of a white supremacist corporate state in the United States that deculturalizes, dominates, and even erases cultures and ways of life of Native Americans, African Americans, Puerto Ricans, Mexican Americans, and Asian Americans.Clearly, this critique is not new, though it needs to be exposed frequently.Carter G. Woodson (1933), for instance, persuasively advanced it in the 1930s, as did others in decades that followed.Ronald Takaki (1989Takaki ( , 1993) ) focused on multicultural, especially Asian communities, whereas Ming Fang He (2003He ( , 2010;;He & Phillion, 2008), addressed the treatment of the issue in curriculum studies.Molefi Asante (1991) and bell hooks (1994) provide insights about the immense African American omissions from educative life outside of institutionalized curriculum studies; inside the curriculum field, I recommend examples by Cameron McCarthy (1990), McCarthy and Warren Crichlow (1993) and William Watkins (1993Watkins ( , 2001Watkins ( , 2015)).So pronounced has been the disregard for African American insights, that as highly regarded as the Eight Year Study has been in scholarly circles, a principal offshoot study known as the Black High School Study (1940Study ( -1946)), led by William A. Robinson, has been scarcely known until studied extensively by Craig Kridel (2015).Sandy Grande's (2005Grande's ( /2015) ) work illuminates red pedagogy or educational approaches of Native Americans, which finally has been accepted in U.S. curriculum discourse after long ignoring earlier Native Americans, many of whom remain unnamed or unknown.The plight of Latino/a education in the USA is vividly captured by Angela Valenzuela's (1999) term subtractive schooling, a force that diminishes or erases cultural knowledge and community understandings of Latino/a populations, while Gonzalez, Moll, and Amanti (2005) build on the corpus of work by Luis Moll, which acknowledges the legitimacy of formerly discredited funds of knowledge in repressed and suppressed Latina/o families and communities.Building on funds of knowledge could partially prevent or reinstate that which is lost by subtractive schooling.Dealing with such issues involves a highly complex endeavor that Bernardo Gallegos (in press) shows to involve colonized, indigenous performances of persons or groups cast into subaltern experiences.
In the realm of recent curriculum history, scholars (e.g., Au, Brown, & Calderon, 2016Baker, 2001, 2009;Grant, Brown, & Brown, 2016;Hendry, 2011;Miller, 2005;Munro, 1998;Watkins, 1993Watkins, , 2005Watkins, , 2015) ) admonish us not to neglect the contributions of scholars of color who were often left out of most of 20 th century curriculum discourse by scholars who have created the academic field of curriculum studies.Along with the aforementioned courageous and insightful pioneers of diverse U.S. minority groups, I suggest that evaluation should entail seeking and acknowledging those from the past (as well as the present) who offer curriculum views that rarely have been recognized for doing so.
To begin, I advocate evaluating the contemporary and past relevance of leaders such as Chief Seattle (1786-1866), Cordero y Molina (1790-1868), Sojourner Truth (1797-1883), Frederick Douglass (1818-1895), Harriet Tubman (1822-1913), Sarah Winnemucca (1844-1910), Booker T. Washington (1856-1915), Ida B. Wells (1862-1931), Jane Addams (1860-1935), Black Elk (1863-1950, Anna Julia Cooper (1868-1964), W. E. B. DuBois (1868-1963), Carter G. Woodson (1875-1950), Helen Keller (1880-1968), Alain Locke (1885-1954), Septima Clark (1898-1987), Ella Baker (1903-1986), and George Sanchez (1906-1972).In these and other educators of color one could see examples of resistance to education.For example, in the mid-20 th century, I suggest as exemplary the freedom schools of the U.S. Civil Rights Movement as developed by Septima Clark, Ella Baker, Charles Cobb, Myles Horton, and many others, as depicted by Charles Payne (1996).So, too, we should revisit critiques of U.S. history textbooks by James Loewen (1999Loewen ( /2007) ) in which he exposes lies that obfuscate evils such as conquest, slavery, and genocide in U.S. history curriculum.Earlier, Ruth Elson (1964) exposed many deleterious values taught by 19 th century U.S. textbooks.Moreover, lies have not been limited to what appears in textbooks for students.Work of recent curriculum scholars whose roots or interests reside in the well-being of non-dominant groups has recently questioned and critiqued the domination of Western white (some say white supremacist) curriculum history and development in the United States.This critique is part of a larger criticism of the Western white orientation of academe.In curriculum studies, recent work by Anthony Brown and Wayne Au (Brown & Au, 2014;Au, Brown, & Calderon, 2016) point to the need to expand curriculum literature in the U.S. to be inclusive of perspectives that lie outside the hegemonic power structure.Similar critiques are set forth by many others in a new generation of curriculum scholars (e.g.Hongyu Wang, Danise Taliaferro Baszile, Isabel Nunez, Min Yu, Nathalia Jaramillo, Lasana Kazembe, Elaine Chan, Valerie Kinloch, Maria Botelho, Kenneth Saltman, Jason Gulah, Jinting Wu, Erica Meiners, Therese Quinn, Sabrina Ross, Brad Porfilio, Derek Ford, Debbie Sonu, Sherick Hughes, Cynthia Cole Robinson, Robert Helfenbein, Ruben Gaztambide-Fernandez, Jason Lukasik, Nina Asher, Eve Tuck, and Sonia Janis).There are many more; however, all of these are included with others in The SAGE Guide to Curriculum in Education (He et al, 2015).
Furthermore, misinformation has been rampant in how the U.S. corporate state advances "knowledge" about schooling.For example, Seymour Sarason (1990) predicted the failure of neoliberal and neo-conservative public school reform, for which Myron Lieberman (1995) declared an autopsy, and David Berliner and Bruce Biddle (1995 unearthed research that showed the U.S. educational crisis had been manufactured by governmental and corporate collaboration to sustain their own greed.A most brilliant critique of the lack of sound basis in research for educational policy, as well as the derailing of it by privatized socio-economic interests is found in Gene Glass's (2008) exposure of the fate of education in the wake of fertilizers, pills, magnetic strips, and more.Peter Taubman (2009) insightfully deconstructed the uncritical uses of standards, testing, and accountability-emphasizing implications of doing so.Berliner & Glass (2014) have continued with colleagues to show that the real crisis in education is constructed by a flat-out assault that seeks to privatize U.S. schools through dissemination of misinformation.This is consistent with the images of assault on public education in work provided by William Watkins (2011), which points to profound consequences of policies and practices on persons of color.
As a subset of curriculum studies, we need to address the extent to which curriculum evaluation is susceptible to similar critique.Again, how can or should orientations to evaluation be revised to focus more fully on questions about whose conceptions of knowledge count, whose do not, and whose should count?Clearly, this question raises more questions that are prerequisites to usual treatments of evaluation, assessment, and grading.Basically, as noted earlier by the Critical Reconstructionist, they challenge the field of education with questions about diversity, relative to race, class, gender, ethnicity, sexuality, (dis)ability, language, culture, tradition, place, and more.Whose voices should be heard?Whose values should be a basis for curriculum and evaluation?Who benefits from and who is harmed by past and present practices, who profits from the harm, how should the harm be overcome, and do some policy makers not want the harm to be overcome?Such questions are obviously matters for revision of assessment and evaluation of what we do as educators and evaluators.They bear heavily upon evaluation of schools, teachers, and students; however, they are too seldom considered evaluation questions.Thus, I move and second the motion, here, to see evaluation through lenses of the continuously expanding domain of curriculum studies.
It is increasingly clear to me that there is need for a more global or worldwide critique, similar to that emerging without much power or policy influence in the USA and in other dominant world powers.Critiques of oppression and neglect of diverse groups scattered throughout the world are not powerful enough without the global emphasis noted in my title.I concur with the centrality of the question of Noam Chomsky's (2016) most recent book: Who Rules the World?If a hybrid of governmental-private-corporate-military power and wealth rule the world, then it strives to influence education so as to sustain that rule and obedience to it.As laudable as it is, then, to critique U.S. curriculum, or any other national curriculum, and to include more diverse perspectives, there is a need to move past concern for any one nation or culture, no matter how pervasive it is or how sordid the injustices are within it.As evaluators we needed to evaluate whether we are playing into the hand of such powers.We need to evaluate ourselves and the evaluation structures and instruments we use.I, of course, advocate asking educators to address such issues and questions that follow.How, for instance, would they propose to study greed and injustice on a global scale?
At the same time, I am convinced that we need to search for possibilities from around the world-contemporary and historical.We need to evaluate or assess the myriad anti-imperialist or anti-colonial struggles and resistances to education created by conquerors or by generations of their descendants.As we do this, we should give serious attention to indigenous modes of research or inquiry (Denzin, Lincoln, & Tuhiwai Smith, 2008' Tuhiwai Smith, 2001/2012).For theoretical frameworks to guide such global inquiry, I suggest that we turn back to Frantz Fanon's (1963) devastating critique of imperialist forces that created persons he characterized as wretched, as in his book, The Wretched of the Earth.Too, they should move to Edward Said's (1993) continuation of Fanon's critique as it applies to nationalism and many forms of colonialism that followed the ages of empire, noting Rashid Khalidi's (2004) resurrection of empire as a revival of new kinds of colonialism in a globalized world.It is interesting to see this global threat to oppressed cultures in light of longstanding work of aforementioned Chomsky (2003Chomsky ( , 2005) ) and its myriad implications for education (Chomsky, 2000).John Willinsky (1998) built on central features of Fanon and Said, as well as Chomsky, to argue that the current instantiation of empire teaches students benefits of nefarious ways to divide the world, defining the conquerors as good and right and the conquered as lesser beings without heritage or contribution.It is important to realize that every region of the world (country, culture, or constellation of countries or cultures)-contemporary or historical, and probably future-is beset with its own special version of conqueror and conquered (vanquished, enslaved, wretched).This being the case, one should realize with William Pinar (2008) that the next reconceptualization of curriculum studies is international: "Internationalization promises a … paradigmatic shift" (p.501).
There are, of course, many possible ways of conceiving this.Maybe national and international are metaphoric of transnational entities or even flowing realms of power and capital that shape and reshape a global elite, a manageable middle or subservient class kept moderately content, and an increasing wretched class constantly controlled.Or the expectation might be essentially for nations to be more collaborative while remaining constituted much as they are.Nonetheless, we need to investigate our own work as evaluators with focus on the potential peril in exporting U.S. curriculum or that of other powerful nations in ways that facilitate military and economic conquest.Thus, evaluation is necessary to determine how to learn from resistance to conquest in many different places in the world.To exemplify a move from emphasis on a single nation state to the flow of power on a global scale, we could consider the contributions of Howard Zinn, whose A People's History of the United States (Zinn, 1980) is a critique of conquest of dominant groups in the USA, as an example of the kind of critique that could be developed in other regions, cultures, or places.Zinn's work is exemplary in this regard, and he also has provided global perspective with colleagues (Zinn, Konopacki, & Buhle, 2008) to portray, through a graphic novel format, consequences of American empire.Along with allies (former conquerors and colonialists) a supremacy of rich and powerful forces can be seen to devolve into their own sordid conjuring of manifest destiny for middle class, working class, and wretched peoples worldwide.
Thus, in the early 21 st century there exists need to include more worldwide emphasis on curriculum studies, not merely the old approaches of comparative curricula which often magnified superiority of dominant cultures.In what may be responses to that need, I briefly note several responses from within the curriculum field.Pinar (2003), for example, and colleagues established the International Association for the Advancement of Curriculum Studies.Other examples include John Miller's (1988) long-time emphasis on exemplars of holistic education in many cultures, David Hansen's (2007) portrayal of ethical visions of education from many parts of the world, and Peter McLaren's and Nathalia Jaramillo's (2007) pedagogy offered particularly for what they call the new age of empire.Thomas Popkewitz (2013) pioneered relevant work on the intercontinental, and has provided international perspectives on the questions, methods, and knowledge of schools.McLaren (2015) expanded a pedagogy of revolution, a critical rage pedagogy for self and social transformation, while Noah De Lissovoy (2008,2014) encourages me to hope that pedagogical communities and sharing of embodied theory offer possible challenges to colonialism in the globalized, neo-liberal world.Introduced by William Reynolds who called for expanded curriculum theory (Reynolds & Webber, 2004/2016), examples of expansion are provided in a book edited by Joao Paraskeva and Shirley Steinberg (2016) along with caveats about accepting a canonization of curriculum sources, since it could perpetuate another version of colonized knowledge.Paraskeva (2011Paraskeva ( , 2016) ) continues to provide provocative work on curriculum theory and epistemicide.
I continue to search for instances of resistance to imperial, nationalist, and colonial oppression, especially those neglected in curriculum history.I hope that communities that Fanon (1963) has called wretched can marshal challenges that bring at least a modicum of liberation.In most cases, I have found evidence in situations that have achieved a separation of school and corporate state.For instance, I note educational work by Princess Kartina (1879-2004) and Ki Hadjar Dewantara in Indonesia (1889-1959), Rabindranath Tagore (1861-1941) and Mahatma Gandhi (1869-1948) in India, Jose Marti (1853-1995) in Cuba, Maria Montessori (1870-1952) in Italy, Tsunesaburo Makiguchi (1871-1944) and Daisaku Ikeda (b. 1928) in Japan, and of course, Paulo Freire (1921Freire ( -1997) ) in Brazil, as already noted.Tagore, Montessori, and Ki Hadjar of Indonesia returned to their home countries from work within a small coterie of educators in the Netherlands who lived in self-imposed or forced exile.They created ideas together about alternatives to schools incubated within ideologies of conquest and colonialism.Some of the best of these alternatives embraced play, love, and peace in lived curricula.That is another story, though one well worth telling, enacting, and evaluating, revising, enacting, evaluation-many times over.

Conclusion
In conclusion, my Guest Speakers and I have characterized how Curriculum Studies has been and continues to be a source for expanding conceptualization of evaluation.Curriculum Studies, as a field, began with a strong focus on Foundations of Education; thus, my first Speaker cautioned that we need to be aware of the values that lie behind the major categories of curriculum development: purposes, learning experiences, and organization of learning experiences.This is a central and often neglected task of evaluation.The second Speaker magnified the central foundational question of education (in or out of school), asking: What is worthwhile?If evaluations bypass this question, then the rest of their effort is baseless.The third Speaker warned that evaluators dare not merely attend to the intended curriculum or to the curricular purposes implicit in standardized tests without seeing them in a large ecological web with the other powerful and often neglected curricula: taught, hidden, null, outside, learned, and embodied.
Looking again at Gert Biesta's (2010) challenge that educators not focus so fully on intricate techniques of measurement that they displace the above fundamental considerations about what constitutes good education.As Bertrand Russell (1926) noted long ago, striving to create good education is intricately connected to the quest for the good life.Accepting the importance of the notion of good education and education for the good life as basic is only a beginning.Alternative meanings of what good education and good life mean must also be addressed by evaluators.To illustrate this, I introduced the curriculum orientations of five Speakers: the Intellectual Traditionalist, the Social Behaviorist, the Experientalist, the Critical Reconstructionist, and the Postmodern and Global Anti-Imperialist.Each spoke to problems of evaluating teachers, students, and educational programs/systems.I might be asked why I devoted more space to the latter two speakers, especially the last one.Was it because I see their positions as more relevant?No! It is mainly because I see their positions as less emphasized and therefore in need of greater characterization.Also, the fifth is the newest, so it receives the greatest emphasis.Another question often arises: Which orientation do I adhere to most?I am not an official actor, so I am in a sense channeling different persona within me, a matter for which I should claim to be in therapy, but I am not.I think it is possible that each Speaker expands the complex portrayal of extant curriculum; therefore, together all of the speakers bring a more enhanced perspective of the complexity of education that evaluators must address.
Finally, however, I do hold to a tenet valued by the Postmodern and Global Anti-Imperialist and well expressed by Maxine Greene (1995, p. 197), namely, that we turn our attention to those made wretched by conquerors.She asks, "How can we reconcile the multiple realities of human lives with shared commitment to communities infused once again with principles?How can we do it without regressing, without mythicizing?How, like Tarrou in [Camus, 1948, pp. 229-230] The Plague can we move ourselves and others to affirm that "on this earth there are pestilences and there are victims, and it's up to us so far as possible, not to join forces with the pestilences?How can we, in every predicament, take the victims' sides, so as to reduce the damage done?" Returning to Macdonald (1977), noted above, in each situation we should ask: What are our guiding values?What should they be?How should we live together?And who should be the evaluators who ask such questions?Should it be all who are involved?Evaluation from a curriculum perspective is keeping the curriculum questions alive, i.e., asking in each situation, "What is worthwhile?Who benefits?What should determine better situations for those who do not benefit?Perhaps asking such questions is also the purpose of evaluation.I suggest that they should always be asked, acted on, and asked again-continuously.Is that not a central calling of education itself?
.phillips@gmail.comD. C. Phillips was born, educated, and began his professional life in Australia; he holds a B.Sc., B.Ed., M. Ed., and Ph.D. from the University of Melbourne.After teaching in high schools and at Monash University, he moved to Stanford University in the USA in 1974, where for a period he served as Associate Dean and later as Interim Dean of the School of Education, and where he is currently Professor Emeritus of Education and Philosophy.He is a philosopher of education and of social science, and has taught courses and also has published widely on the philosophers of science Popper, Kuhn and Lakatos; on philosophical issues in educational research and in program evaluation; on John Dewey and William James; and on social and psychological constructivism.For several years at Stanford he directed the Evaluation Training Program, and he also chaired a national Task Force representing eleven prominent Schools of Education that had received Spencer Foundation grants to make innovations to their doctoral-level research training programs.He is a Fellow of the IAE, and a member of the U.S. National Academy of Education, and has been a Fellow at the Center for Advanced Study in the Behavioral Sciences.Among his most recent publications are the Encyclopedia of Educational Theory and Philosophy (Sage; editor) and A Companion to John Dewey's "Democracy and Education" (University of Chicago Press).