~ EPAA Vol. 7 No. 4: Haney, Fowler, Wheelock, Bebell & Malec "Massachusetts Teacher Test" ~
page 1 | introduction | background | reliability & validity | interviews | conclusions | references

Appendix 1
The Massachusetts Teacher Tests: A Chronology

  • 7/85       The Massachusetts legislature passed a bill that, among other things, required candidates for teacher certification to pass a "standardized exam in his or her subject field [and] a standardized exam of communication and language skill" (MGL, 1985, Chapter 188, Section 15).

  • 6/93       The Massachusetts legislature passed the Education Reform Act of 1993 which included language that required candidates to pass tests, specifically "a writing and subject matter test," in order to be certified.

  • 10/96       The Board of Education had "an initial discussion about implementing ... a two-part test for teacher certification .... The Commissioner [Robert Antonucci] recommended that the Board set October 1, 1997 as the implementation date. The Board agreed to discuss this further and take action at the November meeting" (Massachusetts Department of Education, Board in Brief, 10/24/96).

  • 11/96       The Massachusetts Board of Education "voted to endorse a recommendation by Commissioner Antonucci to require all candidates for teacher certification as of January 1, 1998 to pass a standardized test in communications and literacy skills and subject matter knowledge" (Massachusetts Department of Education, Board in Brief, 11/21/96). The Board also voted that Commissioner Robert Antonucci should "proceed at once with the selection of a test vendor with the aim of having the test available for review by the Board no later than October 1, 1997" (Massachusetts Department of Education, Videotape of Board Meeting of 11/18/96).

  • 2/97       The DOE issued on 2/24/97 a Request for Responses (RFR) from prospective teacher certification test contractors. The RFR stated that vendors should describe in their bid 1) how they would deliver a "technical report to the Department of Education following the use of each new form of the tests with a summary for public dissemination" (p. 15); and 2) their "plan for consultation with a technical committee of nationally recognized experts recommended by the Contractor (external to the Contractor's organization) ... The technical committee will review the test items, test administration, and scoring procedures for validity and reliability and report its findings to the Department of Education" (p. 11). The RFR also asked bidding vendors to meet a timetable that included the following critical dates and events:

  • 5/97-11/97       Advisory committees review sample test questions, scoring and evaluation criteria, and plan for quality control. Pilot tests conducted.

  • 11/97       Test materials submitted to DOE for approval.

  • 1/98-6/98       Monitoring of test quality and standardization; reliability study due.

  • 12/97       Commissioner Antonucci reported to the Board on 12/18/97 that he had selected National Evaluation Systems (NES) to develop and administer the new tests, but some issues needed to be settled before executing a contract: "Issues still under discussion with the contractor include the test administration schedule and, more importantly, ensuring that the test will be rigorous and of high quality, based on college-level content" (Massachusetts Department of Education, Board in Brief, 12/15/97).

  • 1/98       The DOE released a registration bulletin and an informational packet concerning these tests. The informational packet, titled "Massachusetts Teacher Tests, Questions and Answers, January, 1998" stated the following:
    "[Question]: If I am now enrolled in a teacher preparation program, when should I take the tests?
    [Answer]: Candidates who expect to complete their teacher preparation programs by August 31, 1998 are encouraged to take the teacher tests on either April 4 or July 11, 1998. Candidates who take the tests on these dates will satisfy the testing requirement automatically. Candidates who take the tests beginning with the October 3, 1998 administration will be required to achieve a qualifying score in order to be certified" (p. 3).
    The official 1998-1999 Registration Bulletin also informed candidates that "no qualifying score will be established until after the first two administrations of the tests ... Candidates who must take the tests and are eligible to participate in those first two administrations will satisfy the testing requirement by completing the tests" (p. 2).

  • 1/98       The DOE mailed content validation surveys to school districts and teacher preparation programs asking teachers and professors to review and comment on test objectives for 31 different tests. Respondents were asked to complete and return these surveys by January 31, 1998. 4,300 eligible respondents replied.

  • 2/98       Educators participated in test validation conferences for the MTT held on 2/10 and 2/12/98. Participants reviewed test bank items for: 1) match of item to test objective, 2) accuracy, 3) freedom from bias, and 4) job-relatedness (for Massachusetts teachers).

  • 2/98       Robert Antonucci resigned the position of Commissioner and was later replaced by Frank Haydu as Interim Commissioner.

  • 2/98       Approximately 1,500 college juniors and seniors in the state's teacher preparation programs participated in pilot testing of new open-ended and multiple-choice questions for selected tests during the early part of February.

  • 2/98       The DOE and NES signed a contract to have NES develop the MTT. William Gorth signed for NES on 2/23/98; exiting Commissioner Robert Antonucci signed for the DOE on 2/26/98.

  • 3/98       The DOE issued a "Study Guide" that stated the following: "This is a preliminary edition of the Massachusetts Teacher Tests study guide. An expanded set of study guides, including sample questions and open-ended questions from each test field, will be available beginning in August 1998."

  • 3/98       The DOE withdrew the study guide because the Written Mechanics Exercise, designed to assess candidates' knowledge of spelling, punctuation and capitalization, was changed. Originally, candidates were to be asked to fill in (three to five) missing portions of six different sentences that were printed in the test booklet. They were to do this as an audiotaped narrator read each sentence, several times over, in its entirety. For the April exam, however, candidates were asked to transcribe a 156-word text written by James Madison in 1787. This text, part of the Federalist Papers, was read three times by an audiotaped narrator. (See copy of this text at the end of this chronology.)

  • 3/98       In a reversal of previous policy, the DOE announced on 3/25/98 that eligible candidates taking the April and July tests would no longer qualify automatically; instead, they would have to achieve a passing score to be provisionally certified.

  • 4/98       The first round of tests were administered on 4/4/98.

  • 4/98-7/98       Scoring panels met to recommend cut scores for every test.

  • 6/98       The Board of Education voted on 6/22/98 to set the cut score at 1 standard error of measurement below the scores recommended by the scoring panels.

  • 7/98       The Board met again on 7/1/98, at the request of Acting Governor Cellucci, and voted to raise the cut score to the level originally recommended by the scoring panels. During this meeting, Frank Haydu resigned the position of Commissioner of Education. David Driscoll was later named Acting Commissioner.

  • 7/98       NES mailed score reports on 7/6/98 to April test-takers.

  • 7/98       The second round of tests were administered on 7/11/98. The Ad Hoc Committee distributed flyers at five of the six test sites.

  • 9/98       The DOE released a "Test Information Booklet" that contained a) sample questions from each section of the communications and literacy test; b) one sample multiple-choice question for just thirteen of the (43) subject exams; c) one sample open-response item for just one subject exam; and d) one sample oral expression and one sample written expression item from one (of the 7) foreign language exams.

  • 9/98       The DOE announced that as of 9/30/98 the title of the educator certification testing program changed from Massachusetts Teacher Tests to Massachusetts Educator Certification Tests.

  • 10/98       The third round of tests were administered on 10/3/98. The Ad Hoc Committee distributed flyers at all six test sites.

  • 1/99       The fourth round of tests were administered on 1/9/99. The Ad Hoc Committee distributed flyers at all six test sites.

      Candidates who sat for the first administration of the MTT on April 4, 1998, were asked to transcribe the following text written by James Madison in 1787. This exercise, which constituted the Written Mechanics section of the communications and literacy test, is intended to assess test-takers' knowledge of capitalization, punctuation, and spelling. An audiotaped narrator read the full text three times:

"No man is allowed to be a judge in his own cause because his interest would certainly bias his judgment and, not improbably, corrupt his integrity. With equal, no, with greater reason, a body of men are unfit to be both judges and parties at the same time. Yet, what are many of the most important acts of legislation but so many judicial determinations, not indeed concerning the rights of single persons, but concerning the rights of large bodies of citizens? And what are the different classes of legislators but advocates and parties to the causes which they determine? It is in vain to say that enlightened statesmen will be able to adjust these clashing interests and render them all subservient to the public good. The inference to which we are brought is that the causes of faction cannot be removed, and the relief is only to be sought in the means of controlling its effects."

page 1 | introduction | background | reliability & validity | interviews | conclusions | references