~ EPAA Vol. 7 No. 4: Haney, Fowler, Wheelock, Bebell & Malec
"Massachusetts Teacher Test" ~
page 1 |
introduction |
background |
reliability & validity |
interviews |
conclusions |
references
Appendix 1 The Massachusetts Teacher Tests: A Chronology
- 7/85
The Massachusetts legislature passed a bill that, among
other things, required candidates for teacher
certification to pass a "standardized exam in his or
her subject field [and] a standardized exam of
communication and language skill" (MGL, 1985, Chapter
188, Section 15).
- 6/93
The Massachusetts legislature passed the Education
Reform Act of 1993 which included language that
required candidates to pass tests, specifically "a
writing and subject matter test," in order to be
certified.
- 10/96
The Board of Education had "an initial discussion about
implementing ... a two-part test for teacher
certification .... The Commissioner [Robert Antonucci]
recommended that the Board set October 1, 1997 as the
implementation date. The Board agreed to discuss this
further and take action at the November meeting"
(Massachusetts Department of Education, Board in Brief,
10/24/96).
- 11/96
The Massachusetts Board of Education "voted to endorse
a recommendation by Commissioner Antonucci to require
all candidates for teacher certification as of January
1, 1998 to pass a standardized test in communications
and literacy skills and subject matter knowledge"
(Massachusetts Department of Education, Board in Brief,
11/21/96). The Board also voted that Commissioner
Robert Antonucci should "proceed at once with the
selection of a test vendor with the aim of having the
test available for review by the Board no later than
October 1, 1997" (Massachusetts Department of
Education, Videotape of Board Meeting of 11/18/96).
- 2/97
The DOE issued on 2/24/97 a Request for Responses (RFR)
from prospective teacher certification test
contractors. The RFR stated that vendors should
describe in their bid 1) how they would deliver a
"technical report to the Department of Education
following the use of each new form of the tests with a
summary for public dissemination" (p. 15); and 2) their
"plan for consultation with a technical committee of
nationally recognized experts recommended by the
Contractor (external to the Contractor's organization)
... The technical committee will review the test items,
test administration, and scoring procedures for
validity and reliability and report its findings to the
Department of Education" (p. 11). The RFR also asked
bidding vendors to meet a timetable that included the
following critical dates and events:
- 5/97-11/97
Advisory committees review sample test
questions, scoring and evaluation
criteria, and plan for quality control.
Pilot tests conducted.
- 11/97
Test materials submitted to DOE for approval.
- 1/98-6/98
Monitoring of test quality and
standardization; reliability study due.
- 12/97
Commissioner Antonucci reported to the Board on
12/18/97 that he had selected National Evaluation
Systems (NES) to develop and administer the new tests,
but some issues needed to be settled before executing a
contract: "Issues still under discussion with the
contractor include the test administration schedule
and, more importantly, ensuring that the test will be
rigorous and of high quality, based on college-level
content" (Massachusetts Department of Education, Board
in Brief, 12/15/97).
- 1/98
The DOE released a registration bulletin and an
informational packet concerning these tests. The
informational packet, titled "Massachusetts Teacher
Tests, Questions and Answers, January, 1998" stated the
following:
"[Question]: If I am now enrolled in a teacher
preparation program, when should I take
the tests?
[Answer]: Candidates who expect to complete their
teacher preparation programs by August
31, 1998 are encouraged to take the
teacher tests on either April 4 or July
11, 1998. Candidates who take the tests
on these dates will satisfy the testing
requirement automatically. Candidates
who take the tests beginning with the
October 3, 1998 administration will be
required to achieve a qualifying score
in order to be certified" (p. 3).
The official 1998-1999 Registration Bulletin also
informed candidates that "no qualifying score will be
established until after the first two administrations
of the tests ... Candidates who must take the tests and
are eligible to participate in those first two
administrations will satisfy the testing requirement by
completing the tests" (p. 2).
- 1/98
The DOE mailed content validation surveys to school
districts and teacher preparation programs asking
teachers and professors to review and comment on test
objectives for 31 different tests. Respondents were
asked to complete and return these surveys by January
31, 1998. 4,300 eligible respondents replied.
- 2/98
Educators participated in test validation conferences
for the MTT held on 2/10 and 2/12/98. Participants
reviewed test bank items for: 1) match of item to test
objective, 2) accuracy, 3) freedom from bias, and 4)
job-relatedness (for Massachusetts teachers).
- 2/98
Robert Antonucci resigned the position of Commissioner
and was later replaced by Frank Haydu as Interim
Commissioner.
- 2/98
Approximately 1,500 college juniors and seniors in the
state's teacher preparation programs participated in
pilot testing of new open-ended and multiple-choice
questions for selected tests during the early part of
February.
- 2/98
The DOE and NES signed a contract to have NES develop
the MTT. William Gorth signed for NES on 2/23/98;
exiting Commissioner Robert Antonucci signed for the
DOE on 2/26/98.
- 3/98
The DOE issued a "Study Guide" that stated the
following: "This is a preliminary edition of the
Massachusetts Teacher Tests study guide. An expanded
set of study guides, including sample questions and
open-ended questions from each test field, will be
available beginning in August 1998."
- 3/98
The DOE withdrew the study guide because the Written
Mechanics Exercise, designed to assess candidates'
knowledge of spelling, punctuation and capitalization,
was changed. Originally, candidates were to be asked to
fill in (three to five) missing portions of six
different sentences that were printed in the test
booklet. They were to do this as an audiotaped narrator
read each sentence, several times over, in its
entirety. For the April exam, however, candidates were
asked to transcribe a 156-word text written by James
Madison in 1787. This text, part of the Federalist
Papers, was read three times by an audiotaped narrator.
(See copy of this text at the end of this chronology.)
- 3/98
In a reversal of previous policy, the DOE announced on
3/25/98 that eligible candidates taking the April and
July tests would no longer qualify automatically;
instead, they would have to achieve a passing score to
be provisionally certified.
- 4/98
The first round of tests were administered on 4/4/98.
- 4/98-7/98
Scoring panels met to recommend cut scores for every
test.
- 6/98
The Board of Education voted on 6/22/98 to set the cut
score at 1 standard error of measurement below the
scores recommended by the scoring panels.
- 7/98
The Board met again on 7/1/98, at the request of Acting
Governor Cellucci, and voted to raise the cut score to
the level originally recommended by the scoring panels.
During this meeting, Frank Haydu resigned the position
of Commissioner of Education. David Driscoll was later
named Acting Commissioner.
- 7/98
NES mailed score reports on 7/6/98 to April
test-takers.
- 7/98
The second round of tests were administered on 7/11/98.
The Ad Hoc Committee distributed flyers at five of the
six test sites.
- 9/98
The DOE released a "Test Information Booklet" that
contained a) sample questions from each section of the
communications and literacy test; b) one sample
multiple-choice question for just thirteen of the (43)
subject exams; c) one sample open-response item for
just one subject exam; and d) one sample oral
expression and one sample written expression item from
one (of the 7) foreign language exams.
- 9/98
The DOE announced that as of 9/30/98 the title of the
educator certification testing program changed from
Massachusetts Teacher Tests to Massachusetts Educator
Certification Tests.
- 10/98
The third round of tests were administered on 10/3/98.
The Ad Hoc Committee distributed flyers at all six test
sites.
- 1/99
The fourth round of tests were administered on 1/9/99.
The Ad Hoc Committee distributed flyers at all six test
sites.
Candidates who sat for the first administration of the MTT
on April 4, 1998, were asked to transcribe the following text
written by James Madison in 1787. This exercise, which
constituted the Written Mechanics section of the communications
and literacy test, is intended to assess test-takers' knowledge
of capitalization, punctuation, and spelling. An audiotaped
narrator read the full text three times:
"No man is allowed to be a judge in his own cause because his
interest would certainly bias his judgment and, not
improbably, corrupt his integrity. With equal, no, with
greater reason, a body of men are unfit to be both judges
and parties at the same time. Yet, what are many of the most
important acts of legislation but so many judicial
determinations, not indeed concerning the rights of single
persons, but concerning the rights of large bodies of
citizens? And what are the different classes of legislators
but advocates and parties to the causes which they
determine? It is in vain to say that enlightened statesmen
will be able to adjust these clashing interests and render
them all subservient to the public good. The inference to
which we are brought is that the causes of faction cannot be
removed, and the relief is only to be sought in the means of
controlling its effects."
page 1 |
introduction |
background |
reliability & validity |
interviews |
conclusions |
references
|