This article has been retrieved  times since August 1, 1999

Education Policy Analysis Archives
Volume 7 Number 23	August 1, 1999	ISSN 1068-2341

---

A peer-reviewed scholarly electronic journal Editor: Gene V Glass, College of Education Arizona State University Copyright 1999, the EDUCATION POLICY ANALYSIS ARCHIVES. Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education.

---

Academic Program Approval and Review Practices in the United State And Selected Foreign Countries Don G. Creamer Steven M. Janosik Virginia Polytechnic Institute and State University Abstract This report outlines general and specific processes for both program approval and program review practices found in 50 states and eight foreign countries and regions.  Models that depict these procedures are defined and the strengths and weakness of each are discussed.  Alternatives to current practice by state agencies in the U.S. are described that might provide for greater decentralization of these practices while maintaining institutional accountability. Introduction         Responding to multiple challenges in the governance and coordination of higher education, state agencies increasingly are examining their structures for carrying out their mandates. Writing for the Education Commission of the States (ECS), McGuinness (1997) predicted that changes would be necessary to correct some structures that were designed for earlier times. Challenges such as the integration of technology into delivery systems for higher education, market pressures, instability in state government leadership, and growing political involvement in state coordination and governance are among the most compelling forces that make these changes likely.         One of the most common responsibilities of state coordinating or governing agencies for higher education is academic program approval and program review. Program approval refers to the process for approval of new academic programs by state higher education agencies or boards and generally is done to curb unnecessary duplication of programs among public institutions. Program review refers to the process of critique of existing academic programs and generally is seen as a strategy for quality and productivity improvements. According to Barak (1991), 45 state agencies undertake some form of program approval and 34 state agencies review at least some existing programs (some reviews are conducted at the state level).         Concerns for quality and for accountability in higher education is increasing in state governments and agencies, putting pressure on traditional statewide coordination functions of these agencies such as program approval and review, budgeting, planning, and monitoring quality. Increasingly, state agencies are seeking ways to decentralize certain functions while, at the same time, increasing accountability. McGuinnes (1997) predicted that states would turn to two quality assurance mechanisms to accommodate these trends. First, he saw a reliance shift from regulatory controls to incentives to ensure public interests. Second, he surmised more coordinated tactics among state and federal governments, accrediting agencies, institutional governing boards, and disciplinary and professional organizations.         This study was undertaken at the request of the State Council of Higher Education for Virginia (SCHEV) that sought policy alternatives to their current academic program approval and review practices to enable a more decentralized approach to the process in an environment of greater accountability. The study sought to provide these policy alternatives by first collecting base-line information about current academic program approval and review practices in all 50 states and in selected foreign countries and, second, to formulate policy alternatives based upon a synthesis of these findings and upon reasoned judgment about such practices in higher education.

Methodology         All 50 states in the U.S. and eight foreign countries and regions, including Australia, Canada, England, Germany (Lower Saxony), Hong Kong, the Netherlands, New Zealand, and Scotland, were selected for study. Data for this study were obtained in two forms: (a) documents obtained from web sites or direct mail and (b) interview results with academic officers of state agencies (no interviews were conducted with agency representatives from foreign countries or regions). Documents were analyzed to illuminate their academic program approval and review policies and practices. Semi-structured interviews with agency academic officers were conducted to determine perceived strengths and weaknesses of current practices and future plans for change in these procedures. Useful data in one or both forms were obtained from all foreign countries and regions and from 46 states. Information from these sources varied in content. Some written policies were explicit and detailed; others were vague and confined in scope. Likewise, interview results varied from expansive and illuminating to narrow and sketchy. Most data were instructive, however, clearly revealing current practice. Inquiries about future plans for change were met with limited success. Either the officers did not know what changes might occur in their agencies or were cautious in their comments for a variety of reasons.          Information from these sources was analyzed for patterns in responses. These synthesized patterns of practice were used to report the findings from the study. Discovered patterns were more normative among program approval practices than from program review practices; therefore, variations in review practices also were synthesized from the data and reported as substratum patterns.   Findings         Findings are presented in four parts: (a) generalized findings about program approval processes, (b) generalized findings about program review processes, (c) summary of program approval and review practices, and (d) generalized program approval and review findings from foreign countries and regions. Generalized findings about program approval and program review are followed next by steps in the processes and, finally, by strengths and weaknesses of the generalized model presented. Distinctive features of the practices from foreign counties and regions follows their generalized findings.   Program Approval Practices in the States         Practices regarding state agency program approval can be summarized and displayed in a generalized model. This model is depicted in Figure 1 and shows widely accepted practices at the institution and at the state agency level where multiple decisions and actions are possible. Notes: Early screening is used by some states to save time and resources. Good proposals are helped. Poor proposals are discouraged. The process used by state agencies may include internal review by staff, external reviewers, or peer reviews. Criteria include need, demand, duplication, cost, ability to deliver, etc. Some states will aid institutions to revise their proposals even after a program has been disapproved. Review at this stage comes as part of a conditional approval or as part of the criteria for full approval. It may involve rigorous review and include accreditation agencies, outside consultants, or agency staff. Figure 1. Typical State Approval Process

When proposals are disapproved at the state agency level, some agencies may help institutions improve their proposals and encourage them to resubmit. When proposals are approved at the state agency level, some agencies schedule a subsequent review as part of the approval process while others grant automatic continuation unless a review is triggered by productivity concerns. Steps in Program Approval by State Agencies Program approval procedures by state agencies generally followed these steps: 1. Institution determines the feasibility of its intent to plan a new program. 2. Institution notifies the state agency of intent. 3. Institution prepares a draft proposal containing a brief statement identifying the program and addressing the following issues: Relation to institutional mission, strategic plan, goals and objectives; Projected source of resources (reallocation, external funds, request for new dollars); Student need; Relationship to other programs in the system and region. 4. The state agency distributes the proposal to other affected institutions to elicit comments and recommendations. 5. State agency staff comments and makes recommendations on the draft proposal. 6. Institution submits the full proposal addressing some or all of the following issues: Centrality to institutional mission and planning Need for the proposed program Societal need Occupational need Student availability and demand (Enrollment level) Reasonableness of program duplication, if any (not including general education programs) Adequacy of curriculum design and related learning outcomes Adequacy of resources to support the program Adequacy of finances Faculty resources Library resources Student affairs services Physical facilities and instructional equipment Adequacy of program administration Adequacy of the plan for evaluation and assessment of the program Diversity plan for increasing the number of students from underrepresented populations Accreditation (Is there a recognized accreditation agency for the program? Will accreditation be pursued?) Use of technology 7. The full proposal is reviewed by one or a combination of appropriate governance bodies, external consultants, and/or program review committees consisting of a representative(s) of the program proposing unit, state agency staff, and/or external experts in the area. 8. State agency takes action to: Approve (provisional approval or full approval) Disapprove Defer 9. If provisionally approved, the institution will address the issues raised by the state agency. The state agency reviews the program after a relatively short period (e.g., for one year). 10. If fully approved, the institution will develop and implement the program. 11. If disapproved, the institution may have the right to appeal. 12. After the graduation of the first class of the new program, the program may receive an in-depth comprehensive review. 13. Change to current program status. Summary of Strengths and Weaknesses of Program Approval Processes by State Agencies Certain strengths and weaknesses in current were identified as follows:         Strengths of Current Practice  Tends to improve the quality of the academic program  Increases interinstitutional communication and collaboration  Incorporates future assessment criteria and accountability measures  Ensures demand and need  Reduces duplication  Conserves resources  Stresses application of state planning priorities         Weaknesses of Current Practice  Reduces autonomy of the institutions  Can  delay the initiation of needed academic programs  Decision making may be politicized or arbitrary  Staffing requirements may be excessive Generalized Patterns of Program Review Practices in States         All states do not conduct program reviews. Where they are conducted, as occurs in a majority of states, they are conducted in differentiated or even idiosyncratic patterns. Though practiced in a variety of approaches, program review procedures in state agencies can be normatively represented in a conceptual scheme. This arrangement is depicted in Figure 2. Notes: Programs are selected for review on a cyclical or "triggered" basis. Cyclical patterns are based on varying recurring time frames. "Triggered" reviews occur due to response to results from productivity measures or interest on the part of the state legislature. As programs are selected, some states use a peer review process as a precursor to the full review process. This process helps in the data collection phase. Program reviews take a variety of forms. They may be done in conjunction with self-studies or accreditation visits. The institution, state agency, or external consultants may conduct them. May be formal or informal. Programs that are approved conditionally are usually given a specific period of time to correct shortcomings. The programs are monitored and additional reviews may be conducted to determine the program's fate. May lead to modification, consolidation, or elimination. Figure 2. State Agency Academic Program Review Process Model

This conceptual scheme suggests three generalizations about academic program review processes. First, some external agent such as the state legislature or state agency selects programs for review. The review may be triggered by a concern such as productivity or mission-related matters. Second, institutions are requested to take certain actions such as conducting self-studies of program effectiveness. Third, state agencies take certain actions such as forming agency review committees or other structures which may include internal or external consultants and/or representatives from accrediting agencies to determine a program’s approval status. Reviews, where conducted, often are focused on disciplines (or discipline clusters) or on broad categories such as degree level programs.         Academic program review practices can be placed in one of three general approaches:         Independent Institutional Review. In this approach, the state agency delegates the authority to conduct program reviews to the institution.  The state agency does not exercise any supervision or audit of the processes (e.g., Michigan, Minnesota, Nevada, and New Jersey).         Interdependent Institutional Review. In this approach, the institution conducts the program review on a regular basis but does so under the guidance and audit of the state agency. The institution determines the review processes and criteria to be used consistent with the context and characteristics of the institution. The institution submits its program review report to the state agency according to an annual or cyclical state-determined plan. Program review reports conducted in this manner often include:  Descriptive program information,  Year of last program review,  Documentation of continuing need,  Assessment information related to expected student learning outcomes and the achievement of the program’s objectives,  Plans to improve the quality and productivity of the program, and  Program productivity indicators.         Based on the information that the institution provides the state agency will make recommendations to modify, consolidate, or eliminate the program(s) (e.g., Hawaii, Kansas, and Montana).         State-Mandated Review. In this approach, the state agency determines the procedures and criteria of the program review, and conducts or commissions the review of the selected programs within the state system. The state agency staff will participate in the review process. System wide (lateral) program review of similar programs within the state may be carried out at the same time as can be seen in Illinois. The state agency also may conduct post-audit reviews of new programs following the graduation of the first class using pre-determined criteria (e.g., Georgia and North Dakota).         Variations on these program review approaches include the use of productivity reviews (normally triggered by evidence of below standard efficacy) and cyclical reviews. When productivity reviews are incorporated into the process, productivity indicators (such as credit hours, course enrollments, number of majors, number of degrees awarded, cost, and related information) are examined annually as reported by the institution. The state agency identifies low productivity and/or duplicative programs and takes action based on their determinations (e.g., Virginia and New Hampshire). Sometimes when reviews are triggered in this manner, the state agency reviews all similar programs in the state (e.g., Montana). When cyclical reviews are conducted, all programs are examined on some pre-determined schedule such as once each 3, 5, 7, or 10 years (e.g., South Carolina and Illinois).         External consultants may be used as a complement with any of the generalized approaches to program review. External consultants form an advisory committee to participate directly in the program review process. On-site visitations may be performed. Most states require the use of external consultants. External consultants may be selected from several groups of experts: External evaluators: Qualified professionals selected from in-state or out-of state to provide objectivity and expertise. Representatives from peer institutions with similar programs: Selected from similar institutions with similar programs to permit informed exchange and to establish comparable standards (e.g., Georgia and Wisconsin). Accreditation agencies: Representatives from specialized and regional accreditation recognized by the state agency may be used in the reviews (e.g., Montana and Georgia). Representatives from state agencies of elementary and secondary education: Selected to achieve better linkage among the different educational levels. Local lay people and other interested parties: Selected to address societal and occupational needs.        The consultants and/or representatives comment upon the quality of the program, resources available to the program, outcomes of the program, program costs, and other factors. An external review report is provided on the findings and each institution may have the opportunity to review the report and make comments. The final report and comments of the institution are reviewed by the state agency where further action may be taken.  The generalized academic program review approaches may occur in combination with one another and may be combined with the use of external consultants. Some of these combinations may be described as follows:  Example 1 features interdependent and state-mandated reviews with the use of external consultants (e.g., Arizona, Wisconsin, and Idaho).  Example 2 features interdependent review and the use of consultants (e.g., Washington and Georgia).  Example 3 features independent review and the use of external consultants (e.g., Michigan, Minnesota, Nevada, and New Jersey).  Example 4 features state-mandated review characterized by productivity review approaches or cyclical state-mandated reviews in combination with the use of external consultants (e.g., Virginia and West Virginia).  Example 5 features independent review under state agency guidelines (e.g., New Hampshire). Summary of Strengths and Weaknesses of Program Review Processes by States         Strengths and weaknesses of program review practices may vary according to the model or approach chosen; however, they may be characterized generally as follows: Strengths of Program Review Practices  Provides an on-going quality assurance check  Even when done on an irregular basis, the process serves as an incentive to ensure quality at the institutional level  When outside reviewers are used, a greater measure of objectivity can be obtained Weaknesses of Program Review Practices  Institutions may focus on the review process and do little with the results  Reviews are not done with great enough frequency to provide real quality control  Process is time consuming  Process is expensive Summary of Program Approval and Review Practices by States         Program approval and program review can be seen as part of the integrated components of quality assurance practices within a state system of higher education. In this view, program approval is the initial and authorizing stage of program quality assurance and program review is a continuation and revalidation of the approval process. The objectives of program approval and program review are the same: ensure mission compatibility, maintain academic standards, assure continuing improvement of academic programs, and guarantee accountability of academic programs. Issues in both program approval and program review also are the same: mission compatibility, need, program structure, availability of resources (financing, faculty and staff, facilities, technology, etc.), and quality assurance.         Program approval and program review processes can be both internal and external, that is, they can be carried out both within the institutions themselves and/or by external agents. External agents may include the state agency, external consultants, peer institutions, accreditation agencies, and other interested parties.         Internal program approval and program review can best safeguard the institution’s autonomy, integrate the processes with the institutional self-improvement efforts, be more flexible, and boost the morale of the faculty and administrators of institutions. However, internal program approval tends not to provide sufficient stimulation and motivation for improvement. External program approval and review procedures are part of the internal program operating processes, exercise outside monitoring, challenge existing program development notions, ensure maximum objectivity and expertise, and encourage the exchange of good practices. However, external review approaches may intrude on institutional autonomy and bring extra financial and reporting burdens to the institutions. Distinctions between program approval and review practices between undergraduate and graduate programs cannot be clearly drawn from this study. Some states clearly are more concerned with one level of academic program than the other, but no systematic pattern in these concerns was evident from the data. Program Approval and Review Practices by Foreign Counties and Regions         Program approval and review practices of Australia, Canada, England, Germany (Lower Saxony), Hong Kong, the Netherlands, New Zealand, and Scotland are summarized as a single practice. In these international practices, program approval and program review often are intertwined and are called quality assurance.         Quality assurance approaches in international locations are similar to practices in the United States in many respects. Three general models are evident: Self-regulating (regulation by the institution or provider of the educational program), as seen in Canada where universities have the authority and responsibility for quality assurance. Externally regulated (regulation by an external agency), as seen in Australia. The federal government of Australia plays a direct and intrusive role in educational policy. A combination of the two (mixed or collaborative regulation), as seen in most of the countries and regions, such as in England, Scotland, the Netherlands, Hong Kong, Germany (Lower Saxony), and New Zealand, though the degree of the external control varies to a great extent. For example, in England and Scotland, quality assurance is more government-driven than in the Netherlands where the institutions are delegated more autonomy. This approach features institutional self-evaluation and cyclical review conducted by a quality assurance agency. Distinctive Features of Program Approval and Review Practices in Foreign Counties and Regions  Institutional self-regulation (self-study) is combined with external quality assurance agency review or audit. The quality assurance agency ensures that the institutions implement their own quality assurance procedures effectively.  The institution may either design its own quality assurance procedures or adopt a formal quality assurance policy determined by the quality assurance agency or by the government. Adopting the formal quality assurance policy helps to emphasize system priorities and ensures consistency and comprehensiveness of comments and judgement of external reviewers across the system.  External reviewers (assessors) play a very important role to ensure objectivity and expertise, promote the exchange of good practices, and respond to the needs of the society. In some countries, external reviewers are draw from foreign countries (e.g., Hong Kong and the Netherlands), from industry (e.g., the Netherlands), and from the local lay people (e.g., Hong Kong). External reviewers may receive training from the quality assurance agency before visiting institutions under review (e.g., Scotland).  In some countries, quality assurance initiatives are very extensive, including an assessment of institutional teaching and learning practices of all academic programs and an assessment of the research skills and training of junior academic staff (e.g., United Kingdom and Germany).  Quality assurance results are scored (e.g., United Kingdom), ranked (e.g., the Netherlands) or published (e.g., United Kingdom) in some countries. Decision-making, such as funding and program elimination, is based on these scores or ranks.  On-site visits involve meetings with groups of faculty, students, administrative staff, and those responsible for running support services. Time is spent in direct observation of teaching and learning.  To reduce the administration burden, participants are encouraged to share proposals, databases, and trend analyses electronically (e.g., New Zealand).

Discussion         According to Barak (1998), more than half of the 50 states are considering deregulation or decentralization of program approval and review practices. This study confirmed a widespread interest in finding alternatives to current practices that still meet statutory or policy requirements.         Academic program approval and review procedures generally are conducted to address program quality and program productivity at the institutional level. Statewide concerns include access and capacity, quality, occupational supply and demand, and program costs and institutional productivity. Interest in decentralization or deregulation of academic program approval and review policies in a context of accountability was evident in state agencies though most demonstrated this interest only in their future plans.         Evidence from this study suggested that overall program approval and review practice in 50 states and eight foreign countries and regions can be distilled into three conceptual models for practice:  State Regulatory Model. A centralized model for quality control characterized by development and application of centralized regulatory requirements for program approval and review by state-level agency.   Collaboration Model. A consolidated model for institution and state agency cooperation characterized by jointly developed and administered program approval and review procedures by institution and state agency.  Accreditation Model. A decentralized standards-based model characterized by the development and application of standards and guidelines for program approval and review and by cyclical audit by state and consulting agents from outside the institution.         The State Regulatory Model and  Collaboration Model are derived from practices in the 50 states. The Accreditation Model primarily is used in foreign countries and regions, though aspects of accreditation are used in program review practices in this country.These models can be depicted along a continuum of state control as shown in Figure 3. Figure 3. Relationship of Program Evaluation Models to State Agency Control         Analysis of information from this study was used to formulate two suggested alternative models for consideration by SCHEV--the Quality Assurance Audit Model (see Figure 4) and the Modified  Collaboration Model (see Figures 5 and 6).         The Quality Assurance Audit Model is a decentralized model of program approval and review characterized by:   Figure 4. Quality Assurance Audit Model of Program Approval and Review     Figure 5. Modified Institution/State Collaboration Model  Delegation of appropriate state agency authority to institutional governing boards,  Development and application of institutional-level quality  assurance policies and procedures (referring to policies and practices that include quality, duplication, and productivity issues), and  Cyclical or triggered state-level audit of these policies and procedures. The Modified Collaboration Model is a centralized model of program approval and review characterized by:  Shared institution and state-level oversight authority,  Institutional-level program approval by classification according to mission relatedness (within mission, related to mission, outside of mission) and the requirement for new  resources, and  Cyclical reviews by state-level agency (for example, at five-year intervals) depending upon classification of initial approval. Figure 6. Program Approval and Review Status within the Modified Institution/State Collaboration Model         The degree of centralization among the five models, those that represent current practice and those proposed as alternatives, can be depicted as shown in Figure 7 on the continuum of state agency control.         Both alternative models are attractive for different reasons relative to state interests in deregulation or decentralization of program approval and review practices.  The Quality Assurance Audit Model places the agency in a policy/coordination role that enables the agency staff to provide broad oversight for the process of quality assurance.  The state agency would be integrally involved in process development and management but would leave the implementation of the process to its respective institutions.   Figure 7. Level of State Agency Control in Five Program Evaluation Models         The most apparent disadvantages of the Quality Assurance Audit Model is that too much authority and control are delegated to the institutions (although this runs counter to stated interests in deregulation or decentralization).  However, using a periodic system-wide audit of program offerings noting year-to-year changes might serve as an excellent way to monitor institutional activity.  Self-study reports and accreditation visits, processes already in place in most public institutions, would provide additional information on institutional decision making in the area of program approval and review.         The Modified Collaboration Model is attractive because it stratifies the approval and review process based on two critical factors--mission and cost.  The model prescribes that additional attention be given to programs that require supplementary resources and fall outside an institution's current mission--the areas of greatest risk to the institution and the state.  At the same time, however, institutions building new mission-related programs by reallocating existing resources receive additional control and authority.  The disadvantage in this process is that risk-taking and innovation may be reduced if institutions act to avoid the more rigorous reviews that come with programs that may fall outside their current mission or require new resources. Implications         State agencies that wish to modify their current academic program approval and program review practices to accomplish goals of deregulation or decentralization in an environment of accountability may find policy alternatives suitable to accomplish the goal. Most current practices currently are reasonably well portrayed in either the State Regulatory Model or the  Collaboration Model. As currently practiced, however, neither of these models accomplish the goals of deregulation or decentralization very well.         Two alternative models to current practice were developed as part of this study.  These new models, when appropriately constructed on policies consistent with the applicable statutory requirements, can release state agencies from burdensome practices without relinquishing responsibility or diminishing accountability. Both the Quality Assurance Audit Model and the Modified Collaboration Model may serve this purpose although clearly the Quality Assurance Audit Model moves the agencies further from current practice than does the Modified  Collaboration Model.         States coordinating and governing boards across the country are struggling to find new, more effective ways of dealing with program approval and program review.  This synthesis of current practice, along with the two alternative models suggested here, may prove helpful as these discussions continue. Notes 1. Special thanks are due to Virginia Tech doctoral students Chunmei Zhao, Michael Perry, and Miya Simpson who assisted in all phases of the research for this project. 2.  A copy of the complete study report and a complete bibliography of materials used for the original  study is available at: http://epaa.asu.edu/epaa/v7n23/v7n23.pdf   References Barak, R. J. (1991). Program review and new program approval of state education boards. (Report to State Higher Education Executive Officers) Denver, CO: SHEEO. Barak, R. J. (September, 1998). Personal communication. Creamer, D. G., Janosik, S. M., Zhao, C., Simpson, M., & Perry M. (1998). Academic program approval and review practices in states and selected foreign countries. (Special Study Report to the State Council of Higher Education for Virginia). Blacksburg, VA: Virginia Tech. McGuinness, A. C., Jr. (1997). Essay: The functions and evolution of state coordination and governance in postsecondary education. In Education Commission of the States, 1997 state postsecondary education structures sourcebook: State coordinating and governing boards (pp. 1-48). Denver, CO: Education Commission of the States.     About the Authors Don G. Creamer dgc2@vt.edu Don G. Creamer is Professor and Coordinator of Higher Education and Student Affairs in the Department of Educational Leadership and Policy Studies at Virginia Polytechnic Institute and State University (Virginia Tech). He is Director of the Educatonal Policy Institute of Virginia Tech. Steven M. Janosik Steven M. Janosik is Associate Professor and Senior Policy Analyst in the Department of Educational Leadership and Policy Studies at Virginia Polytechnic Institute and State University (Virginia Tech).  He is Associate Director of the Educational Policy Institute of Virginia Tech and recently served as Deputy Secretary of Education for the Commonwealth of Virginia.

Copyright 1999 by the Education Policy Analysis Archives The World Wide Web address for the Education Policy Analysis Archives is http://epaa.asu.edu General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Book Review Editor is Walter E. Shepherd: shepherd@asu.edu . The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu . EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Andrew Coulson a_coulson@msn.com Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov hmwkhelp@scott.net Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Richard M. Jaeger University of North Carolina— Greensboro Daniel Kallós Umeå University Benjamin Levin University of Manitoba Thomas Mauhs- Pugh Green Mountain College Dewayne Matthews Western Interstate Commission for Higher Education William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton apembert@pen.k12.va.us Hugh G. Petrie SUNY Buffalo Richard C. Richardson New York University Anthony G. Rud Jr. Purdue University Dennis Sayers Ann Leavenworth Center for Accelerated Learning Jay D. Scribner University of Texas at Austin Michael Scriven scriven@aol.com Robert E. Stake University of Illinois—UC Robert Stonehill U.S. Department of Education Robert T. Stout Arizona State University David D. Williams Brigham Young University   EPAA Spanish Language Editorial Board Associate Editor for Spanish Language Roberto Rodríguez Gómez Universidad Nacional Autónoma de México roberto@servidor.unam.mx Adrián Acosta (México) Universidad de Guadalajara adrianacosta@compuserve.com J. Félix Angulo Rasco (Spain) Universidad de Cádiz felix.angulo@uca.es Teresa Bracho (México) Centro de Investigación y Docencia Económica-CIDE bracho dis1.cide.mx Alejandro Canales (México) Universidad Nacional Autónoma de México canalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State University casanova@asu.edu José Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of Chicago Eepstein@luc.edu Josué González (U.S.A.) Arizona State University josue@asu.edu Rollin Kent (México) Departamento de Investigación Educativa- DIE/CINVESTAV rkent@gemtel.com.mx       kentr@data.net.mx María Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Sul- UFRGS lucemb@orion.ufrgs.br Javier Mendoza Rojas (México) Universidad Nacional Autónoma de México javiermr@servidor.unam.mx Marcela Mollis (Argentina) Universidad de Buenos Aires mmollis@filo.uba.ar Humberto Muñoz García (México) Universidad Nacional Autónoma de México humberto@servidor.unam.mx Angel Ignacio Pérez Gómez (Spain) Universidad de Málaga aiperez@uma.es Daniel Schugurensky (Argentina-Canadá) OISE/UT, Canada dschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil) Fundação Instituto Brasileiro e Geografia e Estatística simon@openlink.com.br Jurjo Torres Santomé (Spain) Universidad de A Coruña jurjo@udc.es Carlos Alberto Torres (U.S.A.) University of California, Los Angeles torres@gseisucla.edu    archives   |   abstracts   |   editors   |   board   |   submit   |   comment   |   subscribe   |   search