This article has been retrieved
times since August 1, 2000
This article has been retrieved
| ||
Volume 8 Number 40 |
August 1, 2000 |
ISSN 1068-2341 |
|
Editor: Gene V Glass, College of Education Arizona State University
Copyright 2000, the
EDUCATION POLICY ANALYSIS ARCHIVES. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education. |
Advanced Placement: Access Not ExclusionWayne Camara
|
|
Abstract
Lichten (2000) argues that increased access to AP courses in high schools has led to a decline in AP quality. He uses a mix of actual data, inaccurate data, and fabricated data to support this hypothesis. A logical consequence of his argument is that a reduction in the availability of AP courses will lead to an improvement in AP quality. In this paper, we maintain that his thesis is flawed because he confounds quality with scarcity. In contrast to his narrow conception of quality, quality in the AP context is subject- specific and multifaceted, embracing course content, the teacher, the student as well as the exam. Increased access will not diminish quality. Instead, increased access exposes students to college-level course material, encourages teachers to expand their knowledge domains, serves as a lever for lifting curriculum rigor, and provides students with the opportunity to experience the challenges associated with advanced placement in college. |
|
Quality. What is quality? How do we measure quality? How do we improve quality? Lichten (2000), in his study "Whither Advanced Placement?," attempts to assess the quality of the Advanced Placement Program®. We believe he fails for several reasons, many of which revolve around his narrow, simplistic definition of quality. We address these concerns in the following section, entitled "Quality."; Then we point out the many "Inaccuracies, Fabrications, and Leaps of Logic" in Lichten's study; indeed, he seems to use data the way an impassioned partisan would in fashioning an opinion piece for an op-ed page. We then explain in the section "AP® Grades" how AP grade levels are set, since Lichten's lack of understanding of the linkage between AP grades and college standards may have confused readers. Finally, we address the issue of ";Access and Elitism," contrasting Lichten's exclusionary ideal with the College Board's goal of widening the circle of students who have access to AP and its challenging curriculum. QualityAny effort to assess the quality of the AP Program must recognize its diversity and complexity, and the fact that each discipline has unique characteristics that must be taken into account. One size does not fit all. Some disciplines are more constant and well defined, which makes it easier to shape AP course descriptions and assess student capability. Other disciplines (such as computer science, for example) are continually evolving; the challenge is to be responsive to anticipated developments in an ever-changing field.The diversity of students taking AP also adds to the complexity. They do not enter a course with the same level of preparedness for undertaking rigorous college-level course work. Some exam-takers come to the AP course with a head start. The advantage that native speakers of Spanish have in the AP Spanish Language and AP Spanish Literature courses is obvious. A similar, yet less apparent, advantage might be possessed by the children of physicists who might receive preparation for science courses through home-based experiences, when it comes to science courses. As AP offers opportunities to more and more students, the range of backgrounds of these students will increase commensurately. Lichten ignores this diversity and complexity to promote his viewpoint. To him, quality can be captured in a simple operational definition: the ratio of the number of advanced placements made by colleges to the number of AP examinations taken, regardless of the subject area or the preparation of the students. By this standard, AP Spanish Language is a high quality examination because its many native Spanish speakers are very likely to receive advanced placement credit. Conversely, the AP Chemistry exam is lower in quality because the corresponding ratio is not as high as for AP Spanish Language. This narrow, simplistic definition of quality is flawed for several reasons. First, the ratio is subject to many factors that have little or nothing to do with quality. For example, students vary with respect to the preparation they bring to the AP course, and their performance on the exam may reflect their varied backgrounds. This affects the top part of the ratio. External factors, such as certain legislative initiatives that provide payment for students' AP Examination fees, will increase the number of students who take AP exams, which in turn affects the bottom part of the ratio. Neither preexisting differences in preparation nor external initiatives affect the quality of the AP course or its examination (or the scoring or grade standards for the exams), yet they affect the ratio definition of quality Lichten uses. Second, Lichten ignores the distinct nature of each AP course by aggregating results across all courses; for example, treating a 3 on the AP Spanish Language exam as if it means the same thing as a 3 on the AP Chemistry exam. Quality is a complex concept. Ignoring the fact that each course and exam is unique is akin to treating all elements as if they had the same atomic weight. Any serious scholarly treatment of the AP Program must recognize the uniqueness of each course. Third, and most critical, Lichten's definition confuses quality with scarcity. Scarcity does not improve quality; it merely alters the context from which we judge it. He argues that access to AP must be restricted or limited in order to restore AP quality. This sounds like an OPEC argument with respect to oil production. Limit oil production (access to AP courses), and the price of oil will rise (Lichten's quality index will increase). Certainly, the price of oil will increase. But will its quality increase? Of course not. Likewise, restricting access to AP courses will make the number of qualified candidates smaller. But will it increase the quality of the AP courses and examinations? Instead of viewing knowledge in disciplines as the exclusive domain of a selected few, the AP Program employs a model based on access. The more people know about math and the sciences, music and the arts, and languages, the more they and society will profit from this knowledge. AP is rooted in the meritocratic principles that led to the foundation of ETS by the College Board and other parties interested in tapping the potential that lay within America (Lemann, 1999). AP was never to be a barrier to access. Instead it should serve as an avenue for access. Students should be encouraged to maximize their capabilities. Quality, as AP defines it, should be measured by the number of students who have been positively influenced by taking AP courses, rather than by the ratio of the number of advanced placements to the number of exams administered. The College Board states in its publication A Guide to the Advanced Placement Program (The College Board, 1999), “There are many benefits for students who take AP courses. They can study subjects they are interested in and challenge themselves with students who are similarly motivated. AP often helps steer students who are unsure about future plans toward college or advanced studies…AP prepares students for the future by giving them tools that will serve them well throughout their college career (p. 6).” The quality of the AP Program is multidimensional and rests on three pillars of quality: fair, valid, and reliable assessments; rigorous introductory college-level curricula; and exemplary teacher professional development. AP strives to ensure that the exam scoring and scaling are accurate and of high quality (as measured by statistical/psychometric indices of accuracy, reliability, and validity). Teacher quality and student preparedness are important factors that also influence quality. Quality also manifests itself in the effects that AP has on students who take the courses but do not take the exam or who do take the exam but do not seek or receive college credit or advanced placement. By Lichten's standards, a student appears on the quality side of the ledger only if she receives advanced placement at the university she attends. Therefore a student who has a 3 on an exam will not receive advanced placement at a college that requires a 4, but will receive it at a college requiring a 3. If the student goes to the college requiring the 4, she is a debit on the quality ledger; if she goes to the other college, she is a plus on the Lichten index. From the AP perspective, the in-depth exposure to the discipline and quality instruction that the student received are the same regardless of which college she attends. She learned from the course; the existence of the course at her school enhanced the overall value of education at that school. While difficult to quantify, it is hard to argue that the existence of AP courses at more schools hurts quality, unless the definition of quality that one adopts confounds scarcity with quality. Finally, AP quality is carefully monitored within each subject domain. AP, as a matter of course, strives to ensure that the exam, grading, scaling, and scoring are accurate and of high quality (as measured by statistical/psychometric indices of accuracy, reliability, and validity). Enhancing course quality is an important component of the AP process as well. Teacher professional development and student preparedness are important factors that also influence quality. Inaccuracies, Fabrications, and Leaps of LogicIn addition to using a narrow, simplistic definition of quality, Lichten (2000) commits several serious errors in scholarship and makes erroneous assumptions about the use and utility of AP.Table 6 is filled with inaccuracies. The number of exams is misreported by 10,000 in 1980 and by over 100,000 in the speculation for 2000. The basis for the percent of qualifying grades is never stated for any year and is thus left to the imagination of the reader. If one assumes that the author is using the percent of AP grades of 3 or higher, the percentage for 1960 is 49% rather than 75%. In 1970, 66% of AP grades were 3 or higher rather than the 75% Lichten reported. Likewise, the percentage for 1980 is off by 1% and the actual percentage for 1990 differs by 4%. The basis for any of the entries for 2000 and 2010 appears to be pure speculation, as are the percentages qualifying for earlier years. Due to the inaccuracies in the left-hand side of the table, the right-hand side errors are substantial (10% inaccuracy in the last column for 1980). The fabrications in the data throughout the entire paper call to question the quality of the scholarship of the document and the inferences made from them. Lichten creates a table of SAT and AP data from ETS and College Board sources. In preparing this table, he assumed that the college associated with each examinee was the college that the student attended. This is correct for students who sent grades to only one college. For those who sent grades to multiple colleges, the college in the Lichten data was the last one on the student's list of colleges. This reality calls into question the validity of his assumption (which would hold true only if every student went to the college that was last on their lists), and any inferences that depend on the validity of the assumption. Table 2 is not only based on a questionable assumption, it also appears to involve unacknowledged estimation on the part of the author. He states that “55% of 3s pass.” Unless Lichten contacted every college for their numbers of AP grades of 3, numbers of AP 4s, and their numbers of AP grades of 5 received, he is stating as fact something that he is fabricating. As discussed earlier, Table 6 shows that his estimations are often quite inaccurate. The text indicates that the data in Table 5 were obtained from ETS. Standard practice is to cite where the data have been published before, and which colleges supplied data. It addition, it would have been helpful to know what constituted remedial classes to calculus. While focusing on the 24% (the paper incorrectly states 22%) of students with AP grades of 3 who took the second or third calculus as their first mathematics course, Lichten again misses the point about the benefits of AP. Exposing students to a rigorous college-level course at high school surely has many benefits. It is clear that the study is unbalanced in its treatment of the issues. When there is competing evidence that refutes his assumptions, Lichten chooses not to cite it. Likewise, when there are alternative explanations for the findings he cites, those interpretations are not posited, even in a footnote. Selective citation may be acceptable in op-ed pieces, but it has no place in a scientific journal. Some examples follow:
AP GradesLichten contends that the College Board's grade equivalents for AP courses are misleading because colleges use different standards for awarding college credit or advanced placement. There are flaws in this argument.The AP grade equivalents are empirically established through research that compares student performance on AP Examinations with the grades students achieve in comparable introductory courses at college. Such grade equivalency studies are conducted with college students attending a range of colleges. Typically, instructors at the 200 colleges receiving the largest number of AP grades for the AP Exam under evaluation are asked to have their students take portions of the appropriate AP Exam under motivated conditions. The lowest composite score that earns an AP grade of 5 is set to represent the average performance equivalent of college students who earn grades of A from their instructor on the AP Exam. The lowest composite score that earns an AP grade of 4 represents the average performance level equivalent of college students who earn grades of B from their instructor on the AP Exam. The lowest composite score that earns AP grades of 3 and 2 represents those college students receiving grades of C and D, respectively, on the AP Exam. Thus, the AP grade scale reflects a consistent standard of student performance that is empirically related to college grades. Lichten asserts that the AP grade scale is misleading and that a “yawning gap” is created between AP grades and college grading policies because some colleges and departments reject the AP recommendation for awarding credit and/or advanced placement to students with an AP grade of 3 as evidence that AP grades are misleading. Individual colleges, and often individual academic departments, establish their own policies for awarding college credit and/or advanced placement for a particular AP grade. It is the specific AP grades that individual colleges use and the course grades at these colleges that differ widely, as perhaps they should. The standard embodied in an AP grade level on a particular exam, e.g., AP Calculus, is the same across institutions; institutional use of AP grades varies across institutions. Access and ElitismThe most disturbing aspects of the Lichten report are the repeated statements and inferences that the quality of the AP Program could only be maintained “as long as AP served a small, elite population chosen from selective schools (p.13).” Additional statements that minority students are not likely to succeed in AP and that better selection of students into AP courses is required to reestablish AP quality are equally troubling. AP data do illustrate that African-American students and Hispanic students generally perform less well on AP Exams than do Asian-American students and White students. Nevertheless, African-American students and Hispanic students can and do succeed in AP. For example, in the last year, there was a 23% increase over the previous year in the number of African-American students who received AP grades of 3 or higher in Charlotte-Mecklenburg, North Carolina.In the 1999-2000 academic year, the AP Program consisted of 32 college-level courses delivered in approximately 13,000 schools to over 700,000 students who completed more than 1.25 million exams. The net impact of AP is that many more students are taking rigorous and challenging introductory college-level courses while in high school. Some of these students may elect not to take the AP Examination, others may take the Examination but not meet an individual college's requirement for advanced placement, and others may be entitled to advanced placement in a subject but not elect to place out of the introductory course. Yet most, if not all, of these students will have benefited from participating in AP. And, as more students complete AP courses, more teachers are completing AP professional development and mastering the teaching of challenging courses and preparing students in earlier grades to be ready for AP-level work in high school. The net effect is to raise academic standards throughout middle and high school and greatly expand the pool and diversity of students exposed to challenging AP courses. In 1979, only 485 African-American and Hispanic students took Calculus AB. Forty-eight percent (236 of 495) of those students earned grades of 3 or higher. In 1999, the number of African-American and Hispanic students earning grades of 3 on the Calculus AB exam increased to 4,889 (a 2072% increase). Lichten may point out that the percentage of AP grades of 3 for these students decreased from 48% to 41%, but one should also note the increase in opportunity for African-American and Hispanic students. Nearly ten times more African-American and Hispanic students received AP grades of 3 or higher in 1999 than even took the AP Calculus AB Exam in 1979. In fact, in a recent publication, Lichten and Wainer (2000) state “…the PSAT-AP relation tells us that a major expansion of advanced placement achievement is possible in this country in all types of schools: inner city, high-performing suburbs, and just garden-variety schools. A doubling of the number of AP students is not only possible, but is likely within the next decade or so (p. 223).” Yet in his study, the same author recommends reducing access to challenging courses such as AP to “only a small minority of above average high school students.” The author is opposed to legislative efforts to prepare more students for success in AP and other rigorous courses through expanded teacher development and initiatives in the middle schools. Restricting access to only the highest ability students attending the most selective high schools is elitist and runs counter to the goals and mission of AP and the College Board. The author attempts to construct a rationale for restricting access to AP and turning back the clock, based on half-truths, constructed data, and selective citations. He does not cite his sources and ignores research suggestive of alternatives. We believe his study does not meet even the minimal scholarly standards for a scientific publication and we reject the unsupported assertions made throughout. NoteThe order of authorship is alphabetical. The work was a collaboration. The views in this article represent the opinions of the authors and not those of the College Board or the Educational Testing Service. The paper was enhanced significantly by the authors following suggestions from Janet Cook, Drew Gitomer, Lee Jones, and Walter MacDonald.ReferencesLemann, N. (1999). The big test: The secret history of the American meritocracy . New York, NY: Farrar, Straus, and Giroux. Lichten, W. (2000, June 24). “Whither Advanced Placement?” Education Policy Analysis Archives, 8, (29). [Online] Available http://epaa.asu.edu/eppa/v8n29.html [Augiust 1, 2000] Lichten, W. & Wainer, H. (2000). The aptitude-achievement function: An aid for allocating educational resources with an Advanced Placement Example. Educational Psychology Review 12 - (2), 201-228. Morgan, R. & Ramist, L. (1998). Advanced Placement students in college: An investigation of course grades at 21 colleges. (Statistical Report 98-13). Princeton, NJ: Educational Testing Service. Available http://collegeboard.org/ap/colleges/sr-98- 13.pdf. About the AuthorsWayne J. CamaraOffice of Research and Development The College Board 45 Columbus Ave. New York, NY 10023 212-713-8069 fax 212-649-8427 Email: wcamara@collegeboard.org
Wayne J. Camara is
the Vice President for Research and Development at The
College Board. He is responsible for monitoring,
coordinating and conducting all research and product
development associated with the range of College Board
assessments, services, and programs. He has served as the
Assistant Executive Director of Science at the American
Psychological Association (APA) directing scientific
involvement in policy and research activities. His principle
areas of research are test validity, selection and
admissions testing, standards and professional practice in
testing, legal and regulatory issues relating to assessment,
and public policy issues in assessment. Dr. Camara completed
a Ph.D. in industrial-organizational psychology at the
University of Illinois at Champaign-Urbana.
Neil J. Dorans is a Principal Measurement
Statistician at Educational Testing Service. He is
currently the statistical coordinator for the Advanced
Placement Program. He has extensive experience in the
statistical work associated with large-scale high-stakes
testing programs, such as the SAT I. Dr. Dorans was the
architect for the recentered SAT I and II scales. He also
developed a flexible, easy-to-use method for assessing
differential item functioning for selected choice and
constructed response items. Dr. Dorans completed a Ph. D. in
quantitative psychology at the University of Illinois at
Champaign-Urbana.
Rick Morgan is a Program Administrator at Educational
Testing Service for the Advanced Placement Program. During
the 1990s he served as the statistical coordinator for
several testing programs including AP. He has published
research in the areas of exam validity, constructed response
testing, and the impact of allowing examinee choice. Dr.
Morgan completed his Ph. D at The Ohio State University in
quantitative psychology and later was a post-doctoral fellow
in measurement at Indiana University.
Carol Myford is a Senior
Research Scientist in the Center for Measurement Models at
Educational Testing Service. Her program of research at ETS
focuses on scoring issues in performance and portfolio
assessments. She has conducted studies related to rater
training, designing scoring rubrics, quality control
monitoring, improving rater performance, and detecting
different types of rater errors. Dr. Myford received her
doctoral degree from the University of Chicago. |
Copyright 2000 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu . EPAA Editorial Board
|