This article has been retrieved times since September 23, 1997.

Comprehensive Study of Factors Impacting Perceived Quality in School Organizations:
Findings from research on quality assessment in Iowa school districts

William K. Poston Jr.
Associate Professor
Department of Professional Studies
Iowa State University

 
 

Abstract

            This paper presents the findings of studies conducted at Iowa State University of public schools in Iowa in the area of perceived quality assessment.  Demographic characteristics of the respondents on the Perceived Quality Assessment Instrument from forty-four school districts were described by position, home annual income, gender, age, level of education, and years experience in current or a similar job.
            The research project undertaken incorporated several studies of quality improvement characteristics of public schools.  The project resulted in a compendium of coordinated research aimed at learning more about the relationships and effects of quality improvement efforts with other factors of school district operations.
            The individual study components (doctoral dissertations) focused on the following issues:

• Assessment of quality improvement climate in community colleges (Bax, 1994).
• Teachers' perceptions of training programs and their relationships to total district  perceptions of quality management (Johnson, 1995).
• Performance-based pay of chief executive officers and effects upon quality improvement processes in school organizations (Behounek, 1996).
• Financial characteristics of school organizations and relationships to quality management factors (Kirchoff, 1996).

            This series of studies conducted over a period beginning in 1993 and culminating in 1996, was designed to assess the perceptions of school district stakeholders about the quality of their school district in terms of the seven quality dimensions of the Baldrige Award criteria.  The instrument, in two versions, focused on the following Baldrige Award areas:

1.  Leadership
2.  Information and Analysis
3.  Strategic Quality Planning
4.  Human Resource Development and Management
5.  Management of Process Quality
6.  Quality and Operational Results
7.  Client Focus and Satisfaction

            Two other instruments were developed for use as a part of the -- the Staff Development and the Executive Compensation Questionnaire was developed and used in the study.  Financial information was obtained from government records.
            A determination was also sought to establish if the PQAI  differentiated in terms of quality between high ranking and low ranking school districts.  Inferential statistics established not only a significant difference between the high and low groups' quality effectiveness index, but there were also significant differences between the groups in each of the seven dimensions or sub-areas of the PQAI instrument.
            A significant positive relationship was found between the perceived quality of district staff development and the perceived quality effectiveness index of the districts.   Also, differences between performance-related and situational-related (nonperformance)  factors were evidenced between board presidents and superintendents and a weak inverse relationship was found between performance-based compensation support and the perceived quality of the systems.   Significant differences were found between the high and low QEI groups in two areas -- revenues per pupil in the leadership sub-area, and transportation cost per pupil in the information and analysis category.  No correlation was found between the sample schools financial characteristics and their any PQAI  rating area with one exception -- transportation cost per pupil and information and analysis.

Introduction

Fragmentation is the process by which we take things apart and study the parts as a means to  understanding the whole.  This is an appropriate model for mechanical systems, but for organizations, it is the interaction among the parts that makes things happen (Graff, 1995).

            The quality improvement emphasis of the private sector has been permeating gradually into the public schools for over a decade. The direction school organizations have been heading has been modified during this period in attempts to improve the quality of schooling, the value of educational programs, and the accomplishments of  learners.  Many changes were effected in schools in America during the aforementioned period, and support for sustained efforts to reform education has come from federal, state, and local levels of  government (Lezotte, 1994).  Once focused exclusively on manufacturing industries, quality improvement strategies have been developed and implemented in many service organizations, including school systems.
            Quality improvement has become nearly a standard fixture of organizational development and staff training in schools, and the importance given to various principles of quality improvement has spawned considerable interest in the suitability of such strategies for school systems.  Suddenly, policy officials are asking whether  quality improvement strategies make sense for school organizations, in which terms such as "product" or "customer" have imprecise definition.
            School organizations' focus on quality improvement strategies is an outgrowth of several policy efforts in the United States and other educationally progressive countries.  These policy efforts indicate that everyone must become "champions of change" (Rowe, 1994).  Old  leadership ideas won't work anymore, and new leadership principles must recognize a number of factors:

•   Organizational hierarchies don't work
•   Vision has power
•   Shared values create alignment
•   Successful management incorporates coaching, vision, and facilitation
•   Feedback on performance is crucial for improvement over time

            In order to foster organizational change, educational institutions have gathered in ideas from systems theory, psychology, management theory, human-resource and organizational development, statistical process control, and human synergy.  All of these ideas, in many guises and combinations, aim to remake organizations so they become more focused, disciplined, quick-footed, humane and competitive (Marchese, 1993).  Many of these efforts are aimed at reducing the level of criticism toward schools and traditional types of organizational operations  (Bradley, 1993)
            Regardless of motivation, school systems have undertaken a number of initiatives to improve classroom instruction, academic units, staff attitudes, and financial costs (Teeter and Lozier, 1993).  A plethora of school improvements and reforms have been designed and implemented with little examination of the net results or effect.  Often, school reforms and improvements are focused on a subordinate part of the system rather than on system-wide change (U.S. General Accounting Office, 1993).  Specific effects of quality improvement efforts have not been adequately anchored in research findings to determine their suitability and usefulness, and the need is widely apparent for identifying how and under what circumstances quality is manifested in school systems
            This medley of studies examines relationships and differences among groups, factors, and conditions associated with quality improvement in schools.  Specifically, this paper summarizes the problems, procedures, findings, and conclusions of four doctoral studies conducted under the supervision of Dr. William K. Poston Jr., Associate Professor of Professional Studies, Iowa State University.  The four studies addressed relationships and inferences in the areas of job satisfaction and work climate, staff development and training, compensation formats and incentives, and financial characteristics.  The doctoral students involved in these four study areas included Dr. Rashid M. Bax, Malaysia Ministry of Education, Dr. Pamela D. Johnson, West Des Moines (Iowa) Community School District, Dr. Thomas E. Behounek, Milo (Minnesota) Public School District, and Dr. Joseph E. Kirchoff, Manchester (Iowa) Community School District.  A number of additional studies are currently underway in the areas of strategic planning, facilities and productivity, and student achievement by several other graduate students at Iowa State University.

Definitions of Terms

            Quality improvement is characterized by specific terminology.  Terms utilized in this study are defined below to provide clarity and understanding of their use in these studies:
 

Executive Compensation:  The sum paid to the chief executive officer of an educational institution and the underlying rationale for its determination, including results of performance or situational circumstances.

Job Satisfaction:  One's perceived emotional state expressed as a point on a scale representing the degree of positivity or pleasure resulting from the appraisal of one's job or one's experience (Locke, 1976).

Malcolm Baldrige Award:  A national award presented by the United States Government to businesses, companies, or organizations for their demonstration of excellence and exemplification of quality principles (Schenkat, 1993).  It is similar to the Deming Award in Japan at least in intent.

Organizational Commitment:  Relative strength of an individual's identification with, and involvement in, a particular organization.  Conceptually, it can be characterized by at least three factors: (1) a strong belief in, and acceptance of, the organization's goals and values; (2) a willingness to exert considerable effort on behalf of the organization, and (3) a strong desire to maintain membership in the organization (Porter, Steers, Mowday, and Boulian, 1974, p. 604).

Perceived Quality Assessment Instrument:  A questionnaire developed collaboratively by William K. Poston Jr. and Rashid M. Bax at Iowa State University designed to elicit perceptions of organizational stakeholders on seven dimensions of quality improvement modeled after the Baldrige Award criteria.

Perceived Quality Effectiveness Index:  A ratio or score produced by dividing the stakeholders' perceptions of the current status response score by the desired status response score, expressed as a decimal.  Ratios less than one (1) indicate need for improvement; ratios greater than one (1) indicate exceeding expectations.

Quality Improvement (synonyms: quality management, continuous quality improvement, or total quality management): a customer-focused strategic and systematic approach to continuous performance improvement (in an organization)  (Vincoli, 1991, p. 28).

Staff Development or Professional Development:  Activities of school organizations that seek to prepare employees for improved performance in their present or future duties and responsibilities.

Staff Development Questionnaire:  An instrument developed for the purpose of measuring the perceived quality of staff development activities in an educational organization (Johnson, 1995).

Systems Theory:  A practice or intention to view and work with organizations as an interdependent set of components including purpose, people, methods, environment, materials and other factors that influence the organization's functioning and activities and that affect and interrelate with all other factors.

 

Purposes and Rationale for the Study

            There were three main purposes for conducting this series of studies.  The first purpose of these studies was to determine if the Perceived Quality Assessment Instrument (commonly referred to as the PQAI) designed and developed at Iowa State University was effective in measuring factors commonly associated with excellence, efficacy, and effectiveness in school organizations.  The second purpose was to identify the extent of any relationships and the nature of any associations among variables measured by the PQAI and other measures used as a part of the studies.
            Finally, the purpose was to find whether or not operations and management actions in schools have an impact or influence upon the perceived level of quality within the system.  Understandings obtained from these data should help school systems seeking improvement of quality to see more clearly what gaps exist among present and desired organizational conditions and what part the studied variables play in a systems approach for school organizations.  The studies also attempted to demonstrate how management decisions might close the gap between the "current" and "ideal" status of factors as measured on the PQAI.
 

Research Questions for the Study

            Questions relevant to the studies were formulated and postulated in four areas -- general perceptions of quality management, staff development, executive compensation, and financial factors.  The questions addressed are listed below

1.  Perceptions of Quality Management Assessment:

• What are the perceived current and ideal levels of quality management and the quality effectiveness index (ratio) between the two levels in the selected districts?
• What components of the perceived quality assessment instrument have the greatest impact on the overall quality rating of the selected districts?
• What value can be ascribed to the Perceived Quality Assessment Instrument in reliability and validity in measuring system quality?

2.  Staff Development Questions:
 

• What are perceptions of quality of staff development programs in the selected districts?
• Is there a relationship between the perceived quality of district staff development and the perceived level of quality management in the selected districts?
• Is there a relationship between ratings of perceived levels of staff development programs and the seven dimensions of district quality?
• How do districts rated highest and lowest in perceived quality of staff development differ or compare on the perceived quality effectiveness index?

3.  Executive Compensation Questions
 

• What factors are used and which are preferred in the compensation of the chief executive officer of school districts by superintendents and governing board presidents and how do these two groups differ in choice of factors determining compensation?
• How do methods used and preferred in determining compensation for the chief executive relate to levels of quality in the system as measured by the Perceived Quality Effectiveness Index?

4.  Financial Factors Questions
 

• To what extent are there differences between districts with perceived high and low quality on the Perceived Quality Assessment Instrument in terms of financial characteristics?
•   Is there a relationship between the financial stability or soundness of school districts and their perceived quality effectiveness index or sub-scales of the Perceived Quality Assessment Instrument?

Hypotheses and Assumptions of the Studies

            All hypotheses were constructed and analyzed in null form for statistical analyses.
            Several assumptions supported the studies, including the following:

• The population of the study was representative of the districts in the state of Iowa, and respondents were representative of the total population of the systems involved in the studies.
• Respondents understood the content and direction of instrumentation.
• Subjects voluntarily participated in the studies by completing instrumentation.
• The measured perceptions accurately reflected actual levels of organizational quality.
• Respondents were knowledgeable about their school system, and responded accurately and honestly to all instrumentation.

Rationale for Quality Improvement in School Organizations

            Continuous quality improvement, also inaccurately called total quality management, has been gathering momentum in the United States and elsewhere.  The systems thinking approach was led by W. Edwards Deming, a Sioux City, Iowa native, before his death in 1993, and it has continued strongly since.  The "new philosophy" has affected businesses, industries, government, and educational institutions (Brown, 1992).  Literature on quality improvement has mushroomed in the past several years with application for school organizations.  Deming often wrote and spoke of "continuous improvement" with the goal of quality, so the term "continuous quality improvement" is frequently used in discussions about school organization improvement, reform, or transformation (Johnson, 1995).
            Education is in need of dramatic change, according to recent critics.  Transformation has been called for in government, industry, and education in the United States, and such change must "be a change of state, metamorphosis, not mere patchwork on the present system of management (Deming, 1990).
            Specific approaches for improvement are less than clear and distinct.  According to Minnesota's Lieutenant Governor, Joanell Drystad, fragmented, individual attempts at system repair have not succeeded, and the only answer is fundamental, systematic transformation.  In an address to over 2000 teachers, administrators, students, policy-makers, and business partners from 26 of the United States, Finland, and Canada, the call was made to "accelerate local transformation efforts in order to meet the national goals by the year 2000 by deploying quality quickly.  By sharing and working together through the total quality systems approach, we can improve our nation's schools, provide better learning options to our students, and ensure a world-class work force"  (Drystad, 1994).
            A study group of forty-four school districts was identified as the population for the research project.  The Perceived Quality Assessment Instrument was administered to the group of school districts.  The criteria used in development of the specific items on the instrument (see Appendix for sample of the instrument) were drawn from research literature and documented successful institutional practice.  The criteria reflect a number of factors that focus upon the development of educational excellence.
            Continuous improvement is an organizational behavior grounded in several important characteristics -- clear goals, mission, and organizational expectations, defined direction for the design and delivery of superior teaching and learning, continuous focus on results and the use of feedback in decision-making, equitable and consistent connections among all organizational components for all clients and stakeholders, and efficient and effective use of resources (Frase, English, and Poston, 1995).  These factors form a foundation for the assessment of organizational functioning, and comprise a comprehensive assessment strategy.  The characteristics of such a strategy should include the following (US Dept. Commerce, 1995):

• Clear ties between what is assessed and the school system's objectives, particularly in what clients are to obtain (learning, services, etc.) or gain from the organization.
• A focus on improvement -- built upon a definition of student performance, faculty and staff capabilities, and program performance.
• Assessment as "embedded and ongoing" that is curriculum-based, criterion-referenced, and aimed at fostering improved understanding and accomplishment of goals and requirements.
• Clear guidelines as to how assessment results will be used.
• An ongoing method to evaluate the evaluation process to improve the connection between goals and client success.

            For effective school improvement to occur, significant changes will have to be made in how school systems function.  Many facets of school operations will have to be modified, but serious commitment calls for an understanding and implementation of quality management.  The past success of quality management in business and industry raise the possibility of applying its principles to education (Teigland, 1994).  It is important that the highest levels of leadership provide direction for quality improvement to occur (Bax, 1994; Walton, 1986).  School administrators must seriously consider quality improvement as one option for bringing about much needed change and improvement in school organizations (Teigland, 1995).
 

The Quality Improvement Studies

            Despite its recency in use as an organizational development tool, quality improvement in schools does not have a long history of research in its efficacy and impact upon commonly held measures of school organizational characteristics.  At Iowa State University, instrumentation was developed for use in measurement of the status of perceived quality within a school organization.  The assessment tool, the Perceived Quality Assessment Instrument (PQAI) was structured to determine the perceived level of quality of a school system based upon selected Baldrige Award criteria.

Baldrige Award Criteria
 
            This group of impact studies involved an adaptation of the Malcolm Baldrige Award criteria.  The Baldrige Award, begun in the United States, was designed to recognize corporations for excellence in achieving quality.  The Malcolm Baldrige National Quality Award (Baldrige Award) was established in 1987 through legislation (P.L. 100-107).  The purposes of the Baldrige Award were threefold:

• To promote awareness of the importance of quality improvement to the national economy;
• To recognize organizations which have made substantial improvements in products, services, and overall competitive performance; and
• To foster sharing of best practices information among U.S. organizations.

            Eligibility for the Baldrige Award was initially open only to for-profit organizations. However, recent developments indicate that eligibility might be extended in the future to not-for-profit organizations.
            The Baldrige Award Program strategy consists of two parts: (1) conceptual and (2) institutional.  The conceptual part of the strategy involves the creation of consensus criteria which project clear values, set high standards, focus on key requirements for organizational excellence, and create means for assessing progress relative to these requirements. The institutional part of the strategy involves use of the Criteria as a basis for consistent communications within and among organizations of all types. Such communications stimulate broad involvement and cooperation, and afford a meaningful and consistent basis for sharing information. An important part of the communications is the sharing of information by Baldrige Award recipients (U.S. Dept. of Commerce, 1995).
            Through the Baldrige Award, rigorous criteria were created to evaluate applicants for the Award. The Baldrige Award Criteria, based upon a set of core values and concepts, focus on key requirements for organizational excellence. These requirements are incorporated in a seven-part Criteria framework. Accompanying this framework is a set of Scoring Guidelines which permit evaluation of performance relative to the detailed Criteria. The evaluation leads to a feedback report--a summary of strengths and areas for improvement. All Baldrige Award applicants receive a feedback report.
            The Baldrige Award Criteria and Scoring Guidelines have led to a number of key developments:

• Creation of a means for self-assessment;
• Replication of an award system by hundreds of organizations, including states, cities, companies, and not-for-profit organizations; and
• Creation of training programs.

            Throughout the life of the Baldrige Award Program, the principal uses of the Criteria have been for such other purposes. To date, more than one million copies of the Criteria have been disseminated, and a like number of copies have been duplicated by others. This compares with a total of 546 applicants for the Baldrige Award.
            Since the Baldrige Award was established in 1987, there have been 22 Award recipients (1988-1994). Award recipients have demonstrated a wide range of improvements and achievements, including product and service quality, productivity growth, customer satisfaction, reduced operating costs, and improved responsiveness.  Also, Award recipients are among the Nation's leaders in investment in developing the skills of the work force.
            Working Toward an Education Category Since the inception of the Baldrige Award in 1987, some educators have been involved in the Program through their service on the Award's Board of Examiners.  In addition, the Award recipients have sought to involve educators and educational organizations, locally and nationally. Also, some state and local award programs already include education categories.  In parallel with these Award developments, many educators have launched quality improvement efforts.  National initiatives such as Goals 2000 reflect a growing national consensus to strengthen education.  As a result of these and related developments, interest has grown in establishing a Baldrige Award category for education. In 1993, a decision was reached to launch Pilot activities in 1994 and 1995 to address the many issues that arise in extending eligibility to education.
 
Perceived Quality Assessment Instrumentation

            Parallel with the development of the Baldrige Award criteria, instrumentation was under development to apply principles of the system to education.  In 1993, an attempt was undertaken to develop instrumentation that focused on the factors of quality improvement in connection with school system organizational performance.  The Perceived Quality Assessment Instrument was developed for use with school districts by William K. Poston Jr. and Rashid M. Bax at Iowa State University in 1993-94.
            The instrument was built along the lines of the Baldrige criteria framework, and included seven dimensions:  leadership, information and analysis, strategic quality planning, human resource development and management, management of process quality, quality and operational results, and client focus and satisfaction.  Six to eight items were developed and included in each category.  Statements were developed addressing the operations and functions of school districts, and respondents were requested to report their perception of the current situation of their system on a five point Likert-type scale.  The respondents were also requested to indicate their perception of the ideal or preferred situation on the same scale on each statement.  Responses included "strongly agree, agree, neutral, disagree, and strongly disagree."

Calculating the Quality Effectiveness Index

            Unique to this instrument was a quality effectiveness index, which was a calculated ratio of the current situation and the ideal situation scores on each item, each of the seven dimensions, and for the total instrument.  The formula for the index was expressed as follows:

QEI = R_c / R_i  

QEI = Quality Effectiveness Index  Rc = Rating of Current Situation  Ri = Rating of Ideal or Preferred Situation

 

            Calculating the quality effectiveness index in this manner permitted a basis for comparison across school systems as to the level of quality considering the ideal or favored status of the system.  The expressed ratio expressed a bi-directional assessment of quality, and indicated a diminution of quality as the ratio decreased from 1.0, and an achievement of expectations as the ratio increased above 1.0.  In other words, if a school system had a quality effectiveness ratio of .50, it was only approximately 50% of the way toward the level of quality it perceived as desirable.  A quality effectiveness ratio of 1.25 indicates surpassing those same expectations by approximately 25% .
            Such an instrument in educational research can be very valuable if carefully planned and developed (Borg and Gail, 1989).  Self-assessment tools like the PQAI have been described in the professional literature as tools for obtaining an aerial view of the territory to be explored (Neuroth, Plastik, and Cleveland, 1992).  By plotting a district's current location on the map, educational leaders can determine what courses of action would be appropriate for improvement.
            Validation and reliability determinations for the instrument are discussed in the findings section of this report.  The Perceived Quality Assessment Instrument criteria are most explicit in the areas of organizational functioning pertaining to teaching and learning.  The focus on teaching and learning depends upon leadership and organizational expectations, information and analysis of assessment data for improvement, strategic planning for quality improvement and mission accomplishment, development and management of human resources congruent with participatory management, employment of process quality principles and procedures, monitoring of quality and operational results, and a clear focus on clientele and their satisfaction.
            The point values and Likert-type scale represent an initial attempt to provide a basis for scoring participating school systems in terms of progress in performance improvement.
 

Methodology of the Studies

Population of the Study

            The study involved school districts in the State of Iowa, a mid-western state with an agricultural-industrial economic climate and a rural-urban mix of communities.  The specific population of the study was identified prior to the commencement of the study, so that questions could be postulated toward the respondents selected (Borg & Gall, 1989).  A graduate seminar group of Ph.D. students and the major professor, common to all the students, was formed for the purpose of framing and jointly planning the study.  The seminar determined several aspects of the study, including the population selected, the variables measured and scrutinized, and the activities pursuant to the completion of the research on perceived quality improvement.
            Forty-four of Iowa's 360 districts were identified and selected to participate in the study by the graduate seminar group.  Participation was voluntary on the part of each school organization, and efforts were made to include systems representative of different sizes and configurations.  The districts selected were representative of the state as a whole in size and geographic distribution.  Agreements to cooperate in the study were received from each district superintendent in order to increase the response rate.  Instruments were delivered to the superintendents' offices for distribution to respondents.  The superintendents were also asked to have the system's official legal representative, commonly referred to as board secretaries, select the respondents randomly and distribute the surveys.
            A sample of personnel completed the study instrumentation.  The Perceived Quality Assessment Instrument (PQAI) was completed by the superintendent, all board members, two administrators, five teachers, three support personnel, and two high school students for a total of 720 possible respondents in the 44 districts. The other instruments were distributed in similar manner, with different personnel responding.
            Staff development information was elicited by a questionnaire designed specifically for that purpose, as was the information on compensation structures.  Both the Staff Development Questionnaire and the Executive Compensation Instrument were designed specifically for this study, and both were validated with standard review and developmental procedures.  Financial information used in the study was obtained from official published sources, provided by the Iowa State Department of Education.

Research Design and Variables of the Study

            A survey design was used to answer the research questions. The first instrument (PQAI) consisted of two parts: Part I - Demographic Information, and Part II Rating of School System Quality Components. The other instruments were drawn from research literature. The dependent variables were the ratios (Quality Effectiveness Index) between current and ideal perceptions of quality management in each district.  The specific calculation involved a ratio of the current perceived quality improvement status divided by the perceived ideal quality improvement rating.  The ratios also included the seven sub-scales: Leadership, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management, Management of Process Quality, Quality and Operational Results, Client Focus and Satisfaction. The independent variable of the study was the perception of quality of the districts' staff development programs. The sub-scale areas and demographic variables of the PQAI are shown in Table 1 below.
 

Table 1.  Sub-scale areas and demographic variables of the PQAI

Quality Components	Demographic Information
Leadership	Position
Information and Analysis	Home Annual Income
Strategic Quality Planning	Gender
Human Resource Development and Management	Age
Management of Process Quality	Level of Education
Quality and Operational Results	Years Experience in Job
Client Focus and Satisfaction

 

Development of the Instruments and Data Source

            The Perceived Quality Assessment Instrument (PQAI).  Use of the questionnaire in educational research can be very valuable if carefully planned and developed (Borg & Gall, 1989).  In this study, three questionnaire surveys were developed for use in this study, and a fourth data-gathering technique was also used.  The first survey instrument was based upon the Baldrige award criteria and labeled, "School System Perceived Quality Assessment Instrument" (Poston & Bax, 1994, see Appendix A).  The instrument was based upon the seven dimensions of the Malcolm Baldrige Award areas:  Leadership, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management, Management of Process Quality, Quality and Operational Results, and Client Focus and Satisfaction.
            Six to eight items are included in each category with statements addressing the operations and policies of school districts. Respondents were asked to judge their current situation and the desired or ideal situation in their school system for each item using a Likert scale (Borg & Gall, 1989) of five possible responses for each (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree).  Demographic items were also included (position, income, gender, age, level of education, and years of experience in current or similar job) to aid in possible statistical comparisons and analyses of the groups.  The instrument was refined and improved following Bax's study (Bax, 1994).
            The Perceived Quality Assessment Instrument (PQAI) was developed and based upon principles of sound informational and research principles.  Borg and Gall (1989) report that specific behaviors can be predicted from attitude measures about those behaviors, therefore a Likert-type scale of five possible responses for each item was again used: Almost Never, Occasionally, Don't Know, Frequently, Almost Always (Knudsen, 1993).  The third response, Don't Know, was included in the third position to encourage either a negative or positive answer from respondents. Questions posed in closed form aid in the efficient quantification and analysis of results (Borg & Gall, 1989).  Some items were reversed from the others, so that not all statements would be written in similar (positive) format (Borg & Gall, 1989), and (2) so that respondents would not feel that every answer should be marked Almost Always to be "correct."  The length of the items and the length of the questionnaire itself were kept as short as possible for ease in understanding and increased chance of the instruments being returned (Borg & Gall, 1989).  In addition, several devices were used to encourage timely participation, including a self-addressed postage-paid mailer, phone calls, and personal contacts.
            The PQAI was validated by thirteen experts in quality improvement who were educational leaders, superintendents, university professors, or university officials with an understanding of quality processes.  The panel is listed in the Appendix.  Each member of the panel was asked to assure that each item accurately reflects the concepts purported to be measured by the instruments, to evaluate the items for clarity and completeness, and to make suggestions for improvement. Based on the recommendations of the panels, the instruments were revised.  The Perceived Quality Assessment Instrument was further pilot tested by a group of Iowa State University doctoral students studying quality management in public schools.  The PQAI instrument took approximately ten to fifteen minutes to complete.
 

Development of the Other Instruments and Data Sources

            Staff Development Instrumentation.  The District Staff Developmental Questionnaire (Johnson, 1995) was developed by one of the members of the study group (Johnson, 1995)  after reviewing staff development literature in the following areas:  (1) planning, (2) administrative support, (3) delivery, (4) follow-up, and (5) evaluation.  (Wood et al., 1982; Fullan, 1985; Joyce & Showers, 1988; Sparks and Loucks-Horsley, 1989; Senge, 1990; Rebore, 1991; Guskey, 1994; McBride et al., 1994; Sparks & Vaughn, 1994) Orlich, 1983; Guskey & Sparks, 1991; Templeman & Peters, 1992; Showers, 1985; Wade, 1985; Bates & Stachowski, 1991; Wood & Thompson, 1993).  The instrument was validated by ten staff development professionals recognized throughout the state and nation as very knowledgeable in this field.  A Likert scale approach was used for the instrument.

            Executive Compensation Instrumentation.  The Current and Preferred Methods of Awarding Superintendent Salary Increases Questionnaire (Behounek, 1996) survey instrument was developed following a review of related literature and an interview of ten superintendents and twelve board members about methods of awarding compensation to superintendents.  Twelve criteria were incorporated in the instrument, with five of the criteria based upon the superintendent's performance and seven of the criteria nonperformance related.  The instrument was validated by a panel of nine practicing administrators and researchers, and field tested with twelve board members.  A Likert-type scale was used for the instrument.

            Financial Information Instrumentation and Data Sources.  The source of financial information for the selected Iowa school districts was obtained from the official financial audits for each system from the 1992-93 school year, the most recent data available (Kirchoff, 1996).  The source of the audits used in the study was the State Auditor's Office, a division of government in Iowa.  The audits were used as the source of student enrollment, fund balances, revenues and expenditures.
 
Human Subjects Committee Approval

            The Iowa State University Committee on the Use of Human Subjects in Research reviewed all research projects as a part of this study and concluded that the rights and welfare of the human subjects were adequately protected, that risks were outweighed by the potential benefits and expected value of the knowledge sought, that confidentiality of data was assured, and that informed consent was obtained by appropriate procedures.
            All instruments employed the use of anonymity for respondents, as there was no request for names of respondents. However, the questionnaires were coded to identify which school system the respondents represented.  This was necessary for follow-up and data analysis.  The completion of the questionnaires was voluntary and constituted consent to participate in the research project. All questionnaires were kept secure throughout the duration of the study and filed for safekeeping after the study was concluded.

Data Collection Procedures

            Study instrumentation was personally delivered to each superintendent or his/her designee in each district in late February and early March of 1994. The accompanying directions for completing the surveys were self-explanatory.  Instructions were printed on the fronts and backs of the questionnaires to mail the completed instruments directly to Iowa State University by dropping them in the U.S. mail.  A postal permit was printed on the back of each. To increase the rate of return, phone calls were made to participating superintendents in April asking them to encourage those who had not yet responded to do so.
            As a result, 471 of the 720 PQAI surveys were received for a return rate of 65.4%.

Statistical Analysis of The Data

            Perceived Quality Assessment Instrument.  After the surveys were returned, the responses were entered into a Microsoft Excel spreadsheet program.  Only one of the PQAI instruments was discarded because it was damaged in the mail.  Four of the districts were removed from the study because an insufficient number of PQAI surveys were returned (one survey and was returned from two districts and three surveys were returned from two others).  As a result, 462 surveys representing forty districts were used for data analysis. The items were ranked on a scale ranging from 1 (strongly disagree) to 5 (strongly agree).  Means for each respondent and each district for each question were computed.  On the Perceived Quality Assessment Instrument, means for each respondent and district for the current and ideal situations were calculated as well as the ratio (quality effectiveness index) of the current divided by the ideal.
            Descriptive statistics were computed on the demographic variables of the PQAI.  Preliminary analyses using the one-way analysis of variance (ANOVA) were conducted to compare the responses of demographic groups.  Factor analysis was conducted on the PQAI to determine reliability.  The ANOVA and Scheffe' post-hoc method were used to determine differences between the four groups.
            Other instruments.  Spearman rho and Pearson product-moment correlation coefficients were computed to test for a relationship between responses on the PQAI and the staff development and compensation instruments and between financial factors and the PQAI.  Stepwise multiple regression was used to find a relationship between staff development and the seven PQAI dimensions.  The differences between high and low districts on the compensation and staff development instruments and financial factors were determined using the t test and analysis of variance techniques.
            The Cronbach alpha reliability coefficient was also calculated for the Staff Development Questionnaire and the Executive Compensation Instrument to determine the internal consistency of the total instruments.
 

Findings of the Studies

Introduction

            The findings of the study are organized into the following sections: (1) General Characteristics of the Sample, (2) Reliability Analysis of the Instruments, (3) Quality Management Perceptions, (4) Staff Development Perceptions, (5) Results of Hypotheses Tested, (6) Evaluation of the Perceived Quality Assessment Instrument, and (7) Summary.

General Characteristics of the Sample

            The primary purpose of this section is to describe the participants in this study who completed the Perceived Quality Assessment Instrument (PQAI) with respect to the following demographic variables: (a) position, (b) home annual income, (c) gender, (d) age, (e) level of education, and (f) years experience in current/similar job. The descriptive information is presented in Table 2 which follows.
 

Position - Of the 458 respondents who filled in this question, 132 (28.8%) were teachers, 76 support staff (16.6%), 68 administrators (14.8%), 30 superintendents (6.6%), 108 board members (23.6%), and 44 students (9.6%). Four people did not complete this category.

Home Annual Income - Of the 358 people answering this question, 24 reported earnings of less than $10,000 (6.7%), 77 from $10,000 to $29,999 (21.4%), 147 from $30,000 to 49,999 (40.9%), and 110 reported earnings of $50,000 or more (30.9%). There were 104 respondents who did not fill in this information.

Gender - The number of male and female participants reporting their gender was fairly even. Male respondents numbered 215 (52.6%) while females totaled 194 (47.4%). The number who did not fill in this item was 53.

Age - The age of the respondents was divided into five categories: (1) under 18, (2) 18-29, (3) 30-55, (4) 56-70, and (5) over 70. The largest group was the 3055 year category (73.8%). The second largest group was the 56-70 year group (11.9%). The smallest group was the over 70 category with only one person. Forty-two people did not report their age group.

Level of Education - The educational level of the respondents was classified into four categories: (1) less than a B.A. degree, (2) B.A. degree, (3) Master's degree, and (4) Doctorate degree. The largest group (151 people) represented those who had less than a B.A. degree, (34.6%). Those having a B.A. degree were the second largest group with 142 people (32.5%). The third largest group, those having a Master's degree, were very close with 129 people ((29.5%). Fifteen respondents (3.4%) reported having a Doctorate degree. Twenty-five respondents did not report their level of education.

Years of Experience - The years of experience of the respondents were divided into four categories: (1) under 5 years, (2) 5-10 years, (3) 11-25 years, and (4) 25 years or more. The findings revealed that 191 (44.3%) of the participants had 11-25 years of experience, 96 (22.3%) had 5-10 years of experience, 88 (20.4%) had under five years, and 56 (13%) had 25 or more years of experience. Thirty-one respondents did not complete this item.

 

Table 2.  Demographic information of respondents on the PQAI

Category		Frequency	Percent
Position	Teacher  Support Staff  Administrator  Superintendent  Board Member  Other  Total	132 76 68 30 108 44 458	28.8 16.6 14.8 6.6 23.6 9.6 100.0
Income	< $10,000  $10,000 - $29,999  $30,000 - $49,999  $50,000 +  Total	24 77 147 110 358	6.7 21.4 40.9 30.9 100.0
Gender	Male  Female  Total	215 194 409	52.6 47.4 100.0
Age	< 18  18 - 29  30 - 55  56 - 70  70 +  Total	23 36 310 50 1 420	5.5 8.6 73.8 11.9 0.2 100.0
Education	< Baccalaureate  Baccalaureate  Master's Degree  Doctorate  Total	151 142 129 15 437	34.6 32.5 29.5 3.4 100.0
Experience	< 5 years  5 - 10 years  11 -25 years  26 + years  Total	88 96 191 56 431	20.4 22.3 44.3 13.0 100.0

 

            One-way analysis of variance and t-test procedures were used to determine differences in the quality effectiveness index among different demographic groups. It was found that there were significant differences in responses by position, gender and education. The Scheff© method was used to analyze these differences and established that there were significant differences in the position category between the responses of teachers and administrators and between teachers and board members, both at the .05 level. A description of respondents by position follows.

Analysis by position

            In order to gain a better understanding of the respondents on the PQAI and their differing characteristics, further analysis was conducted on the position category by generating cross-tabulations with the other variables--income, gender, age, education and experience. The results are shown in Table 3 that follows.

             Position with income - As shown in the following Table , the majority of teachers (50) reported incomes in the $30,000 to $49,999 category with the second highest number (44) being at the over $50,000 level. Support staff reported 29 people in the $30,000 to $49,999 and 27 at the $10,000 to $29,999 level.
            For administrators, the two highest categories were over $50,000 (39) and $30,000 to $49,999 (28). Only one administrator reported making less than $29,999. All of the superintendents except two (27) reported incomes over $50,000, while the remaining two reported earnings in the $30,000 to $49,999 category. The majority of board members (51) were in the over $50,000 category, with 31 at the $30,000 to $49,999 level, and 16 at the $10,000 to $29,999 level. Of those in the other category (students), 23 reported earnings under $10,000 with the remainder (16) dispersed throughout the other three levels.
 
 

Table 3.  Demographic information of PQAI respondents by position

Category	Teacher	Support Staff	Admin.	Supt.	Board	Other	Total	Percent
Income (35 missing)    < $10,000    $10,000 - $29,999    $30,000 - $49,000    $50,000 +  Total	0 30 50 44 124	1 27 29 12 69	0 1 28 39 68	0 0 2 27 29	0 16 31 51 98	23 3 7 6 39	24  77  147  179  427	5.6 18.0 34.4 41.9 100.0
Gender (53 missing)      Male      Female    Total	39 82 121	21 48 69	50 10 60	22 0 22	65 30 95	18 24 42	215 194 409	52.6 47.4 100.0
Age (42 missing)      < 18      18 - 29      30 - 55      56 - 70      70 +    Total	1  18  93  10  0 122	1 4 56 9 0 70	0 1 54 8 0 63	0 1 22 6 0 29	0 0 80 14 1 95	21 12 5 3 0 41	23 36 310 50 1 420	5.5 8.6 73.8 11.9 .2 100.0
Education (25 missing)     < Baccalaureate     Baccalaureate     Masters Degree     Doctorate   Total	1 95 0 0 126	57 10 0 0 71	1 3 0 0 68	0 2 9 9 29	55 32 6 6 403	37 0 0 0 40	151 142 15 15 437	34.6 32.5 3.4 3.4 100.0
Experience (31 missing)     < 5 years     5 - 10 years     11 - 25 years     26 + years   Total	12 30 67 17 126	13 21 33 6 73	18 17 24 9 68	4 8 15 2 29	17 20 48 18 103	24 0 4 4 32	88 96 191 56 431	20.4 22.3 44.3 13.0 100.0

 

             Position with gender - There were over twice as many females as males in the teacher (82 and 39) and support staff (48 and 21) categories, as shown on Table 3.
            However, male administrators (50) outnumbered females (10) 5 to 1, while there were no female superintendents and 22 male superintendents who reported their gender. There were more than twice as many male board members (65) as females (30). In the other category, 18 were males and 24 were females.
            Position with age - The majority of the teachers (93) were in the 30-55 years age group, with 18 falling in the 18-29 category, 10 in the 56-70 division, and one reporting being under 18. Most of the support staff (56) reported being 30-55, with 9 in the 56-70 age group, 4 in the 30-55 range, and one under 18. Administrators, too, reported the majority (54) in the 30-55 group, with 8 in the 56-70 category, and one in the 18-29 age group. Twenty-two superintendents reported being in the 30-55 age range, with six in the 56-70 category, and only one in the 18-29 age group. No board members reported being under 30, while the vast majority (80) were in the 30-55 range, 14 in the 56-70 category, and 1 over 70. In the other group, 21 reported being under 18, 12 said they were 18-29, 5 filled in the 30-55 group, and 3 marked the 5670 age group.
            Position with education - Most of the teachers (95) reported having B.A. degrees, with 30 having Master's degrees, and one reporting less than a B.A. The majority of support staff (57) had less than a B.A. degree, with 10 having a B.A., and 4 having a master's degree. Most of the administrators (62) had a master's degree, with only 3 listing a B.A., and one filling in the less than B.A. category. Nearly two-thirds of the administrators (18) had master's degrees, while nine had doctorates, and two had completed their B.A.'s. Of the board members reporting, over half (55) had less than a B.A., while 32 had a B.A., 10 had a master's degree, and six held doctorates. In the other group, 37 reported having less than a B.A., while three filled in the master's degree response. It would appear that there is an inverse relationship between the education and income levels of the board members. While the majority of board members did not have a college degree, over half of them reported incomes of more than $50,000.
            Position with experience - The results from Table 2 show that the largest number of respondents in every group except other had 11-25 years experience in the current or similar job. The second highest level in every group except administrators and other was the 5-10 year category. The remainder of respondents were distributed fairly evenly between the other two categories.
 

Analysis of other demographics

            The ANOVA procedure revealed no significant differences on the quality effectiveness index in income levels, age, or experience. However, differences were found for education levels. Using Scheff©, significant differences were found between those with less than a B.A. and those with a B.A. degree. No other significant differences were found between the other education levels.
            Although the t test indicated a significant difference in the quality effectiveness index between males and females, when t tests were conducted for gender by position, no significant differences were found in any position level for gender. Thus the difference was only present when all levels were considered as a whole.

Reliability Analysis of the Instruments

            The SPSS program was utilized to determine the reliability of the Perceived Quality Assessment Instrument. Analyses were conducted for the current and ideal sections and for the overall scale (total instrument).  The alpha reliability coefficients are reported in the Table which follows. The alpha coefficients ranged from .68 to .85 for the current section of the instrument, with an overall reliability of .96.  The alpha coefficients were somewhat different for the ideal section, ranging from .61 to .89, with an overall reliability of .94, indicating a high positive correlation among all items.
            Table 4 below presents the information and reliability coefficients for the PQAI Instrument.
 
 

 Table 4.  Reliability analysis of current and ideal sections of the PQAI

	Reliability
PQAI Dimension	Item Numbers	Current	N	Ideal	N
Leadership	1 - 6	.84	454	.82	451
Information and Analysis	7 - 12	.85	443	.82	430
Strategic Quality Planning	13 - 18	.84	451	.87	445
H.R. Development & Mgmt.	19 - 25	.83	451	.61	441
Mgmt. of process Quality	26 - 31	.79	440	.85	437
Quality & Operational Results	32 - 37	.68	454	.75	448
Client Focus & Satisfaction	38 - 45	.85	453	.89	444
Overall	1 - 45	.96		.94

 
             The Cronbach alpha reliability coefficient was also calculated for the Staff Development Questionnaire to determine the internal consistency of the total instrument. Estimates of internal consistency are based on the average correlation among items within a test or instrument. The reliability coefficient for all thirty items was .95, again a high positive correlation.  This level of reliability was determined to be more than sufficient for the purposes of this research.
            On the Compensation Assessment Instrument, the Cronbach alpha reliability coefficient was also calculated to determine an estimate of the internal consistency of the instrument.  Reliability coefficients were calculated for the two sets of questions -- one for current practices and the other for ideal (or preferred) practices in compensating superintendents, or school system chief executive officers.  Table 5 below contains the reliability coefficients for the four sub-areas and the total instrument (Behounek, 1996).
 
 

Table 5.  Cronbach Alpha reliability coefficients for sub-areas of the Compensation Assessment Instrument 

Variable	Alpha	Number of Items
Current Practice
Non-performance related items	.75	7
Performance related items	.83	5
Ideal (Preferred) Practice
Non-performance related items	.87	7
Performance related items	.67	5
Total of all items	.67	24

 

            The level of reliability of the Compensation Assessment Instrument was determined to be adequate, albeit modest, for the purposes of this research (Nunnally and Durham, 1975).

Quality Management Perceptions

            The study sought to determine the perceived current and ideal levels of quality management in each participating district and the Quality Effectiveness Index ratio between the two assessments of perception.  Means were first calculated for the responses for each item for each district. The means of all of the current and ideal responses for each district were then figured. Current means ranged from 2.80 (district HH) to 3.92 (district MM).  Ideal means ranged from 3.77 (LL) to 4.56 (P and R).
            Finally, the ratio between the current and ideal means for each district were determined. The quality effectiveness index ranged from .6532 (district HH) to .9506 (district A). Four districts were removed from the study because of their low return (N for districts F and V = 1 survey each; N for districts I and M = 3 surveys each). It was felt that such a small sample from those districts would not give a reliable representation of the perceptions of the entire district.  The presentation in Table 6 below shows the distribution of means for current and ideal situations and the ratios for the remaining 40 districts.
 

Table 6.  Current and ideal means and quality effectiveness index for districts on the Perceived Quality Assessment Instrument 

District Code	Rank	Current Situation Mean	Ideal Situation Mean	Quality Effectiveness Index
A	1	3.60	3.79	.9506
AA	23	3.10	4.00	.7747
B	10	3.52	4.18	.8408
BB	33	3.44	4.66	.7371
C	7	3.70	4.32	.8572
CC	31	3.32	4.53	.7322
D	36	3.10	4.36	.7119
DD	15	3.39	4.23	.8005
E	30	3.08	4.15	.7419
EE	32	2.97	4.01	.7399
FF	4	3.66	4.15	.8859
G	21	3.44	4.37	.7865
GG	16	3.30	4.12	.8001
H	14	3.24	4.04	.8020
HH	40	2.81	4.30	.6532
II	28	3.22	4.29	.7500
J	13	3.45	4.26	.8083
JJ	11	3.56	4.31	.8250
K	19	3.33	4.19	.7943
KK	38	2.97	4.30	.6899
L	27	3.26	4.32	.7554
LL	5	3.50	3.95	.8843
MM	6	3.92	4.46	.8784
N	26	3.43	4.50	.7619
NN	22	3.40	4.42	.7703
O	3	3.77	4.23	.8909
OO	2	3.71	4.08	.9107
P	39	2.99	4.56	.6545
PP	8	3.64	4.29	.8501
Q	35	3.20	4.36	.7357
QQ	18	3.58	4.47	.7989
R	29	3.42	4.58	.7483
RR	20	3.56	4.50	.7889
S	25	3.23	4.23	.7628
T	24	3.26	4.25	.7669
U	9	3.46	4.09	.8480
W	34	3.27	4.44	.7369
X	17	3.44	4.31	.7990
Y	37	3.38	4.28	.7097
Z	12	3.65	4.44	.8234

 

Staff Development Perceptions and PQAI

            To determine the perceptions of teachers in the participating school districts about their systems staff development programs, means for each question for each district were calculated. Then the mean of all the responses for each district was determined. These means ranged from 1.963 to 3.789.  The same four districts (F, I, M, and V) were removed from the list for comparisons with the group used with the PQAI.  Ranks of districts on the Staff Development Questionnaire were compared with the districts' ranks on the PQAI, as shown in Table 7 below.

Table 7.  Comparison of ranks on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument 

District Code	SDQ Rank	PQAI Rank	Sub-Group
A	1	1	1
K	2	19	1
PP	3	8	1
J	4	13	1
JJ	5	11	1
LL	6	5	1
QQ	7	18	1
L	8	27	1
C	9	7	1
II	10	28	1
O	11	3	2
Q	12	26	2
OO	13	2	2
DD	14	15	2
H	15	14	2
B	16	10	2
RR	17	20	2
P	18	39	2
R	19	29	2
MM	20	6	2
N	21	26	3
BB	22	26	3
Z	23	16	3
GG	24	16	3
NN	25	22	3
AA	26	23	3
W	27	34	3
U	28	9	3
D	29	36	3
T	30	24	3
X	31	17	4
S	32	25	4
Y	33	37	4
EE	34	32	4
E	35	30	4
CC	36	31	4
G	37	21	4
FF	38	4	4
HH	39	40	4
KK	40	38	4

            The forty districts, listed above, were divided into four groups for further analysis.  The sub-groups were comprised of groups of ten districts each, according to their rank on the Staff Development Questionnaire.  A one-way analysis of variance was calculated to determine differences in means of the four sub-groups.  The results of the ANOVA are shown below in Table 8 below.
 
 

Table 8.  One-way analysis of variance of group means on the Staff Development Questionnaire 

Source	df	SS	MS	F	Fcv
Between groups	3	7.53	2.51	129.53	4.51
Within groups	36	.70	.02
Total	39	8.22

                            p= <.0001

             The four groups appeared to differ in terms of their group means on the Staff Development Questionnaire, as illustrated in the Table (Table 9) below.
 
 

Table 9.  Sub-group means and standard deviations on the Staff Development Questionnaire 

Group	N	Mean	St.Dev.
1	10	3.36	.2129
2	10	2.93	.0720
3	10	2.65	.0859
4	10	2.17	.1399

 

             A one way analysis of variance was then employed to determine differences in means between and among the four sub-groups.  Significant differences were found among the groups, and a Scheff© method was used to determine which sub-groups differed from one another.  All groups were found to differ from each other significantly, as shown in Table 10 which follows:
 
 

Table 10.  Comparison of group means on the Staff Development Questionnaire 

Sub-Group Comparison	Mean Difference	Scheffe F test
1 vs. 2	.4366	16.4*
1 vs. 3	.7155	44.05*
1 vs. 4	1.1944	122.76*
2 vs. 3	.2789	6.69*
2 vs. 4	.7578	49.42*
3 vs. 4	.4789	19.74*

                                     *Significant at .01 level
 

PQAI and SDQ Relationships

            The relationship between the PQAI measurement of perceived quality and the Staff Development Questionnaire was explored.  The composite district ratings of staff development programs were correlated with the Quality Effectiveness Index computations.  A Spearman rho test was used to calculate correlations between the two indices.  A correlation of .50131 was found, and it was significant at the .01 level of confidence.  This relationship was determined to be moderate, but significant.
            A second exploration was conducted to determine if there were any relationships between ratings of perceived levels of staff development quality and the seven dimensions of district quality (current) measured by the PQAI in the participating districts.
            To accomplish this purpose, means of the current perceptions in each of the seven quality dimensions of the Perceived Quality Assessment Instrument for the ten highest ranking districts on the Staff Development Questionnaire were computed.  The mean of each district on the Staff Development Questionnaire was compared with the mean of each quality dimension for these districts.  Stepwise multiple regression was used to determine the relationship of the measured quality of staff development to the seven quality dimensions. The seven variables were entered one at a time and a significance test was conducted to determine the contribution of each (Hinkle, Wiersma, & Jurs, 1988). The stepwise solution was terminated when the remaining variables did not make a statistically significant contribution to the regression.
            It was found that three of the seven dimensions--Client Focus and Satisfaction, Quality and Operational Results, and Management of Process Quality--were good predictors of levels of quality staff development. The adjusted R2 (squared) was .956 indicating that 96 percent of the variance on the Staff Development Questionnaire was explained by these three dimensions.  The remaining four dimensions -- Leadership, Information and Analysis, Strategic Quality Planning, and Human Resource Development and Management -- did not predict levels of district staff development in any significant degree.
            Correlation matrices were also constructed in order to show the interrelationships between all the sub-scales on the Perceived Quality Assessment Instrument. Table 11 shows the correlation coefficients for each of the seven sub-scales. As a rule of thumb, correlation coefficients between 00 and 30 show little if any correlation; 30 to 50, a low correlation; 50 to 70, a moderate correlation; 70 to 90, a high correlation; and .90 to 1.00, a very high correlation (Hinkle, Wiersma, & Jurs, 1988).
            Using these guidelines, the matrix shows that the majority of the correlations are moderate positive correlations. None of the relationships are below 50, with the lowest being between Leadership and Quality and Operational Results (.52).
            Five of the relationships were in the high positive category: Human Resource Development and Management and Strategic Quality Planning (.77), Human Resource Development and Management of Process Quality (.73), Management of Process Quality and Quality and Operational Results (.71), Client Focus and Satisfaction and Management of Process Quality (.72), and Client Focus and Satisfaction and Quality and Operational Results (.74). These relationships show that all seven dimensions are related to each other and measure the same concept -- district quality management.
 
 
 

Table 11. Correlation matrix for the seven dimensions of the PQAI

PQAI Sub-scale	Leader-  ship	Infor. &  Analysis	Strategic  Planning	H.R. Dev.  & Mgt.	Process   Quality	Quality  Results	Client  Focus
Leadership	1.00
Information	.63	1.00
Planning	.61	.69	1.00
H.R. Mgt.	.62	.67	.77	1.00
Process	.63	.69	.67	.73	1.00
Results	.52	.62	.62	.67	.71	1.00
Focus	.60	.64	.64	.69	.72	.74	1.00

 

            Still another exploration was conducted into the area of determining whether or not districts ranked highest or lowest in perceived staff development quality differed significantly when measured by the perceived quality effectiveness index.
            Table 12 shows these highest and lowest districts, their ranks on both instruments and their mean scores for each.
 

Table 12.  Mean scores and ranks for the highest and lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument
 

Table 12.  Mean scores and ranks for the highest and lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument

District	Rank on SDQ	Group	SDQ Mean	PQAI Mean	Rank on PQAI
A	1	1	3.789	.9506	1
K	2	1	3.567	.7943	19
PP	3	1	4.442	.8501	8
J	4	1	3.427	.8083	13
JJ	5	1	3.408	.8250	11
LL	6	1	3.278	.8843	5
CD	7	1	3.278	.7989	18
L	8	1	3.273	.7554	27
C	9	1	3.113	.8572	7
II	10	1	3.067	.7500	28
X	31	4	2.408	.7990	17
S	32	4	2.310	.7628	25
Y	33	4	2/280	.7097	37
EE	34	4	2.211	.7399	32
E	35	4	2.203	.7419	30
CC	36	4	2.167	.7322	31
G	37	4	2.150	.7865	21
P	38	4	2.042	.8859	4
HH	39	4	1.994	.6532	40
KK	40	4	1.963	.6899	38

 

            Using the unpaired t test on the means of the two groups on both instruments, significant differences were found for the highest and lowest groups on the Perceived Quality Assessment Instrument.  A correlation coefficient (Pearson product-moment) was then calculated among groups compared on both instruments. A moderate positive correlation (.564) was found for the top ten districts which was significant at the .05 level. The correlation between the bottom ten districts on both instruments (.264) was not significant. However, when district FF was removed which ranked high (number 4) on the Perceived Quality Assessment Instrument, the correlation rose to .710, a high positive correlation.  It was determined that the higher a system demonstrated perceived quality in staff development, the higher its perceived quality effectiveness index.

Analysis of Executive Compensation Perceptions and PQAI

            The compensation of executives and leaders has often been thought to be instrumental in influencing the level of  performance on the job.  In short, some believe that money can motivate (Poston and Frase, 1992).  Deming, long considered the "father of quality management," however, felt that the theory of psychology eroded any confidence that money could motivate individuals on the job (Deming, 1986).  His notion was that money and employee compensation had little to do with the quality of an organization.
            To test that theory in part, this study undertook to examine the perceptions of school board members and superintendents about the effect or influence of salary compensation upon the quality of an organization.  Perceptions from board members and superintendents were elicited with the instrument earlier described.  The following Table demonstrates the tabulated results of the perceptual assessment.
 
 

Table 13.  Aggregate responses of superintendents and board presidents concerning practices and preferences for salary increases

		Board Presidents			Superintendents
Item	Belief statement	N	Mean	St.Dev.	N	Mean	St.Dev.
CURRENT: Unrelated to Performance
1	Length of service to system	44	2.67	.94	44	2.68	.93
2	Increased formal education	44	2.75	.97	44	2.27	.95
3	Cost of living adjustment	44	2.86	1.00	44	2.48	1.09
4	Growth in revenues	44	3.46	1.15	44	3.50	1.07
5	Settlement of teacher contract	43	3.56	1.22	44	3.80	1.17
6	Regional colleagues status	44	3.39	.99	44	3.48	1.07
7	Student enrollment change	44	1.86	1.02	44	2.16	1.16
	Total	44	2.94	1.17	44	2.91	1.25
CURRENT: Related to Performance
8	Performance on job	44	3.96	1.16	44	3.73	.85
9	Achievement of set goals	44	3.61	1.13	44	3.11	.99
10	School Board evaluation	44	3.77	1.05	44	3.61	.99
11	Student achievement	44	2.77	1.14	44	2.02	1.02
12	Test score results	44	2.05	1.12	44	1.73	.90
	Total	44	3.23	1.32	44	2.84	1.25
IDEAL: Unrelated to Performance
1	Length of service to system	44	2.55	1.02	44	2.97	1.05
2	Increased formal education	44	3.41	.79	44	3.66	.78
3	Cost of living adjustment	44	3.41	.79	44	3.41	1.00
4	Growth in revenues	44	3.02	1.00	44	3.48	.95
5	Settlement of teacher contract	44	3.46	1.07	44	3.21	1.10
6	Regional colleagues status	44	3.07	1.13	44	4.18	.62
7	Student enrollment chnage	44	2.46	.79	44	2.59	.87
	Total	44	3.07	1.04	44	3.36	1.03
IDEAL: Related to Performance
8	Performance on job	44	4.36	.72	44	4.59	.50
9	Achievement of set goals	44	4.18	.79	44	4.32	.86
10	School Board evaluation	44	4.11	.75	44	4.43	.50
11	Student achievement	44	3.21	.85	44	2.84	1.08
12	Test score results	44	2.55	.82	44	2.02	.88
	Total	44	3.68	1.05	44	3.64	1.29

 

            For the purposes of this study, the strength of agreement indicated by each level of response was categorized as low if less than 2.25, moderate if between 2.26 and 3.75, and high if more than 3.76.  These designations were selected arbitrarily for purposes of comparison only.  The results in Table 13 represent superintendent and board president perceptions on how increases for superintendents are awarded in current practice and how they should be awarded given conditions of ideal practice.
            Board presidents and superintendents agreed that increases in enrollment have little to do with compensation, but levels of compensation awarded pursuant to teacher contract deliberations have a high impact upon the level of superintendent's compensation.  In the area of performance-related factors, the highest factor of influence to both board presidents and superintendents was the superintendent's performance on the job.  The lowest relationship to compensation was perceived to be with standardized test results.  Ironically, this measure of organizational effectiveness had low agreement as to its importance in determining compensation.
            When considering the ideal situation, board presidents and superintendents agreed that on-the-job performance, achievement of district goals, and board evaluation of the superintendent should have the greatest level of influence on the superintendent's compensation.  Ideally, as in the current status, student test performance was perceived of low importance.  In non-performance rated ideal conditions, superintendents favored most strongly the use of "benchmarking" salaries of peers or colleagues in the region for use in determining the salary of the superintendent.  Changes in enrollment fared no better in perceptions of ideal factors than with current conditions, as it was rated as the lowest in agreement for both groups.
            To more adequately compare board presidents' and superintendents' perceptions about compensation, the mean scores of both groups were subjected to analysis.  Unpaired, two-tailed t-tests were used to determine if significant differences existed between the two groups on the four areas of the study.  Responses from superintendents and board presidents were also compared between performance and non-performance factors and current practice and ideal situation.  In these cases, paired, two-tailed t-tests were utilized.  Significance was set at the .05 level of confidence for the purpose of this analysis.
 
 

Table 14.  t-Test results of group means for determining superintendent compensation

Group	N	St. Dev.	Mean	t	P
Current:  Unrelated to Performance
Board Presidents	44	.50	2.93	.18	.857
Superintendents	44	.71	2.91
Current:  Related to Performance
Board Presidents	44	.90	3.23	2.2	.030
Superintendents	44	.76	2.84
Ideal:  Unrelated to Performance
Board Presidents	44	.49	3.07	-2.70	.008
Superintendents	44	.51	3.36
Ideal:  Related to Performance
Board Presidents	44	.46	3.68	.40	.690
Superintendents	44	.50	3.64

 

            In the area of perceptions of board presidents and superintendents considering the importance of non-performance related job factors, the analysis revealed that there was no significant difference between the two groups.  On the other hand, when looking at how they two group means compare on performance-related job factors, there was a significant difference between board presidents and superintendents in their perceptions.  Board presidents felt more strongly about using performance in the determination of compensation for the chief executive.
            Two more tests of significant differences were employed, dealing with perceptions in the ideal or preferred situation.  When looking at the importance of non-performance ideal factors, board presidents placed less credence on these factors than did superintendents.  The difference was significant, indicating that superintendents placed more importance on non-performance job factors in determining superintendent compensation than did board members in an ideal situation.
            In the ideal situation using performance-related compensation factors, superintendents and board members were not significantly in disagreement.  Both groups rated the use of performance factors in determining the level of job compensation in the "high-moderate" range.
            To compare performance with non-performance factors across groups, a series of analyses employing t-tests were conducted.  Superintendents' and board presidents' perceptions on performance against nonperformance factors in current practice and performance against nonperformance factors in ideal practice were analyzed.  Comparisons of superintendent and board president perceptions of current practices against ideal practices across performance and non-performance areas were also examined.  Results of these analyses are displayed in Table 15 which follows.
 

Table 15.  T-Test results between groups on performance and non-performance factors in determining superintendent compensation

Group	N	St. Dev.	Mean	t	P
Current:  Superintendents
Non-performance factors	44	.71	2.91	.58	.5636
Performance factors	44	.76	2.84
Current:  Superintendents
Non-performance factors	44	.50	3.36	-2.46	.0180
Performance factors	44	.51	3.64
Ideal:  Board Presidents
Non-performance factors	44	.58	2.93	-2.22	.0317
Performance factors	44	.90	3.23
Ideal:  Board Presidents
Non-performance factors	44	.49	3.07	-6.05	.0001
Performance factors	44	.46	3.68

 

            Given these results, it was determined that superintendents didn't differ in rating the relative importance of current practice performance-based compensation when compared to non-performance based compensation factors for superintendents.  However, in the ideal situation, superintendents did rate performance-based compensation factors higher than non-performance based factors with a higher aggregate response.
            Tests of significance were also applied to aggregate responses of board presidents.  Board presidents differed somewhat by placing more importance on performance-based factors than non-performance based factors in both the current situation and in the ideal situation.  The ideal difference was dramatic and strong in favor of using performance-based factors given the opportunity in an ideal or preferred situation.
            Other tests involved comparing aggregate means for superintendents and board presidents between current practice and ideal situations in non-performance and performance-based areas.  The results of these analyses indicated significant differences between current practice and ideal situations on non-performance and performance factors for superintendents, but not for board presidents (Behounek, 1996).
            Another analytical exercise was undertaken to determine if the perceptions about performance-based compensation had any relationship to the perceived quality assessment instrument data.  Superintendents perceptions on performance-based compensation were categorized in three groups " low support, moderate support, and high support " relative to strength.  These groups then were analyzed with ANOVA to determine if there is a difference related to the superintendents' perceptions.  Low support was defined as a mean score of less than 3.0, moderate included means between 3.1 and 4.25, and high included mean scores greater than 4.26.  The categories were selected for purposes of comparison, and a confidence level of .05 was used.  The results of the analysis are presented in the following Table.
 
 

Table 16.  ANOVA summary table comparing Quality Effectiveness Index rating of school systems by perceived support of performance-based compensation by superintendents 

Source	df	SS	X2	F ratio	FP
Between groups	2	.21	.1	1.23	.303
Within groups	41	3.43	.08
Total	43	3.64
Group			N	X	St. Dev.
Low support for performance compensation			6	3.42	.22
Moderate support for performance compensation			35	3.35	.30
High support for performance compensation			3	3.10	.34
Total			44	3.34	.29

 

            Statistically, there is no significant difference in the rated quality of the school systems based upon the level of the superintendent's support for performance-based compensation.  The results, given full awareness of the small sample size, do display some interesting configurations, and a trend is somewhat evident.  The group of superintendents with the highest mean response indicating high support for performance-based compensation, served in districts with the lowest mean quality index (3.10).  The group of superintendents with the lowest mean response indicating low support for performance-based compensation, served in districts with the highest mean quality effectiveness index (3.42).  As school systems are categorized by their superintendent's support of performance-based compensation, the aggregate mean scores of the quality of the districts increase as the support decreases.
            To test the significance of this perceived relationship, a correlation (Pearson product-moment) was calculated to determine if the strength of the superintendents' support of performance-based compensation was related to the ratings of perceived system quality reflected in the aggregate quality effectiveness index ratio.  The results of the analysis are shown in Table 17 below.
 
 

Table 17.  Correlation of school system quality effectiveness index by level of superintendents' support for performance-based compensation

N	Covariance	R	R2
44	-.03	-.18	.03

 
            The results in Table 17 reflect a slight negative correlation between these two variables.  A negative correlation of -.18 is determined to be "little if any" (Hinkle, 1988).  Weak as it is, the relationship between perceived organizational quality and support for performance-based compensation places imperfect credibility on Deming's theory that performance pay is detrimental to an organization (Deming, 1986).
 

Analysis of Financial Factors and PQAI

            The impact of financial soundness upon organizational quality has been demonstrated in a number of ways in the private sector, but has not been demonstrated unequivocally in public education (Kirchoff, 1996).  Despite a paucity of research demonstrating the influence of financial wealth upon educational institutional quality, there has been a belief that financial stability of a system may contribute to collaboration efficacy and to constancy of purpose (Schmoker and Wilson, 1993).
            Several financial characteristics of educational institutions were identified and assembled and considered in relationship to the quality effectiveness index generated by the Perceived Quality Assessment Instrument.  The factors included the financial solvency ratio (FSR), revenues per pupil (R/P), expenditures per pupil E/P), undesignated unreserved fund balance per pupil (UUFB/P), unspent balance per pupil (UB/P), and the transportation cost per pupil (TC/P).  These six factors were examined in two ways, first with a correlational matrix, using Pearson product-moment techniques, and then with the t-test (two-tailed) between the high and low financial factors compared to the PQAI score.  In other words, the most financially sound districts and the least financially sound districts were compared as to the level of PQAI score, or perceived quality effectiveness index.
            To obtain the highest and lowest categories, the sample group of school systems was divided in to three equal groups of fourteen school districts each (this analysis involved the use of 42 of the 44 school systems) based upon the districts perceived quality effectiveness index.  The index is the ratio of the perceived current mean score on the PQAI to the perceived ideal mean score on the PQAI for the school district.  The means and standard deviations for the sample group, the high PQAI group, and the low PQAI group are shown in the Table below.
 
 

Table 18.  Statistical data for the PQAI sample, high, and low groups

Group	N	Mean	St. Dev.
Sample Total PQAI	42	.796	.080
High PQAI Group	14	.885	.044
Low PQAI Group	14	.716	.047

 

            Correlations were calculated for each of the financial characteristics between the high quality effectiveness index group and the low quality effectiveness index group.  The results of those calculations are illustrated in Table 19 below.
 
 

Table 19.  Correlations between financial characteristics and PQAI

	Financial characteristics
PQAI Factor	FSR	R/P	E/P	UUFB/P	UB/P	TC/P
Sample Group Total	-0.011	0.103	0.113	-0.009	0.099	0.226
Leadership	-0.083	0.144	0.189	-0.077	-0.028	0.115
Inf. and analysis	-0.037	0.095	0.095	-0.031	0.111	0.326*
Strategic planning	0.017	-0.002	-0.007	0.006	0.124	0.079
HRD and mgmt.	-0.104	0.240	0.269	-0.060	0.193	0.238
Mgmt. of process quality	-0.091	0.069	0.056	-0.092	0.086	0.177
Quality and op. results	0.024	0.097	0.106	0.029	0.061	0.216
Client focus and satisfaction	0.012	0.120	0.128	0.112	0.112	0.259

             *p <.01
 
 
             A statistically significant relationship was shown between the transportation cost per pupil and the PQAI of the sample in the information and analysis category of the perceived quality assessment instrument, however, the variance (r2) was calculated to be only 0.106.  This means that only 10.6% of the variance in the transportation cost per pupil was attributable to the PQAI and that almost 90% of the variance was attributable to other factors.
            T-tests were also calculated for each of the financial factors comparing the high Quality Effectiveness Index group with the low Quality Effectiveness Index group to determine if the differences between the groups on any financial factors were significant.  The results of this analysis is shown in the table below.
 

Table 20.  T tests between the high and low PQAI groups and financial characteristics 

Group	PQAI	FSR	R/P	E/P	UUFB/P	UB/P	TC/P
Sample Group Total	9.706*	-0.441	0.352	0.379	-0.440	0.411	1.016
Leadership	8.878*	-0.092	2.336*	0.760	0.063	0.763	-0.014
Inf. and analysis	10.665*	-0.551	0.307	0.171	-0.486	0.185	2.250*
Strategic planning	7.976*	-0.262	-0.363	-0.527	-0.309	1.044	0.000
HRD and mgmt.	6.136*	-0.216	1.758	1.581	-0.042	0.378	1.493
Mgmt. of process quality	10.367*	-0.353	0.221	0.269	-0.513	0.568	0.572
Quality and op. results	8.146*	0.218	0.901	1.095	0.319	-0.130	1.751
Client focus and satisfaction	11.159*	0.661	1.077	1.124	0.746	0.897	1.728

      *p <.0001
 

             Significant differences were found between the high and low group PQAI means, for the revenues per pupil in the leadership category, and the transportation cost per pupil in the information and analysis category.  One of the significant t-test results was in the area of leadership and revenues per pupil.  Another was in the area of transportation costs per pupil and the information and analysis sub-scale.  While statistical significance was found between two of the financial characteristics, forty-six differences were not found to be significant.
 

Analysis of the Perceived Quality Assessment Instrument

            The study also sought to determine if the PQAI items on the perception of quality scales of current and ideal status would align consistently with the seven a-priori determined dimensions based on the PQAI (Baldrige) criteria. A factor analysis was used based on a varimax rotation technique. The results of the analysis revealed three possible factors with Eigenvalues greater than 1.0 on both the current and ideal scales.
            The three factors accounted for 42.3% and 43.3% of the total variance on the current and ideal scales respectively. The distribution of the number of items from each PQAI category among the empirical factors suggested is shown in Table 21.  For example, the six a-priori items for the Leadership category (current responses ) were all contained in factor 3; however the eight items for the Customer Focus and Satisfaction were distributed among factors 2, 3, 6, and 7.
 
 

Table 21. Comparison of the current and ideal a-priori PQAI dimensions with empirical factors 

Scale	Items	Factor  1	Factor  2	Factor  3	Factor  4	Factor  5	Factor  6	Factor  7	Factor  8
Eigenvalue		16.28	1.50	1.27	.99	.79	.73	.59	.49
Current
Leadership	1-6			6
Information &  Analysis	7-12		1		5
Strategic Quality  Planning	13-18	3				5
H.R. Devel. &  Mgt.	19-25	5				2
Mgt. of Process  Quality	26-31	2	3
Quality & Op.  Results	32-37		5
Customer Focus	38-45		3	1			1	3
Eigenvalue		16.33	1.99	1.17	.89	.68	.56
Ideal
Leadership	1-6				6
Information &  Analysis	7-12			1		4	1
Strategic Quality  Planning	13-18		6
H.R. Devel. &  Mgt.	19-25		1			6
Mgt. of Process  Quality	26-31	5	1
Quality & Op.  Results	32-37	5					1
Customer Focus	38-45	7

 

            The results, as shown in the Table  above, suggest that the seven original a-priori dimensions may be reduced to three factors on both scales.  However, the majority of the items in the Information and Analysis dimension load on factor 4 on the current scale and factor 5 on the ideal scale and the majority of items in the Human Resource Development and Management load on factor 4 in the ideal scale, indicating the importance of these additional factors.
            On the ideal scale, three dimensions (Management of Process Quality, Quality and Operational Results, and Customer Focus and Satisfaction) seem to load on one factor (factor 1) which suggests that respondents were not able to differentiate among these three dimensions as originally conceptualized or that the three were perceived as measuring the same thing. The factor analysis results show the items grouped differently than on the original instrument showing that the instrument may be measuring different factors than originally conceived and the items may need to be regrouped. However, when combined with the results of the correlation matrix, it appears that, overall, the instrument is measuring one underlying concept on both scales.
 

Summary and Conclusions

            This paper presents the findings of studies of public schools in Iowa in the area of perceived quality assessment conducted at Iowa State University.  Demographic characteristics of the respondents on the Perceived Quality Assessment Instrument from forty-four school districts were described by position, home annual income, gender, age, level of education, and years experience in current or a similar job.
            The research project undertaken incorporated several studies of quality improvement characteristics of public schools.  The project resulted in a compendium of coordinated research aimed at learning more about the relationships and effects of quality improvement efforts with other factors of school district operations.
            Several dissertations resulted from the studies, including writings by Rashid M. Bax, Pamela D. Johnson, Thomas J. Behounek, and Joseph E. Kirchoff, over a three-year period.   All of the dissertations were supervised by William K. Poston Jr. at Iowa State University during the period 1993-1995.  Each of the study components (parts of the total study) gathered specific data relevant to the issues of the individual study component.  The individual study components (doctoral dissertations) focused on the following issues:

• Assessment of quality improvement climate in community colleges (Bax, 1994).
• Teachers' perceptions of training programs and their relationships to total district  perceptions of quality management (Johnson, 1995).
• Performance-based pay of chief executive officers and effects upon quality improvement processes in school organizations (Behounek, 1996).
• Financial characteristics of school organizations and relationships to quality management factors (Kirchoff, 1996).

            Continuous quality improvement is a recent phenomenon in educational institutions, but it has been popular in business and industry since the late 1970's in the United States.  The management philosophy inherent in continuous quality improvement has been largely based upon the ideas of W. Edwards Deming.  The incorporation of Deming's ideas into educational organizational thinking has been gradual and focused on application of the quality concepts into everyday practice.  Given the goals of public education, questions arise as to whether or not the use of continuous quality improvement principles have relevance in the public school organizational environment.  Moreover, it is even more important to know what relationships and effects result from the application of quality improvement strategies in the public school setting.
            This series of studies conducted over a period beginning in 1993 and culminating in 1996, was designed to assess the perceptions of school district stakeholders about the quality of their school district in terms of the seven quality dimensions of the Baldrige Award criteria.  The Baldrige criteria were used in the development of an instrument that was employed in evaluating the levels of quality in public school organizations.  This version of the Baldrige instrumentation was developed and modified for use in the studies (Poston and Bax, 1994).  The instrument, in two versions, was administered to 44 independent local school systems and to 15 community college in the state of Iowa. and focused on the following Baldrige Award areas:

1. Leadership
2. Information and Analysis
3. Strategic Quality Planning
4. Human Resource Development and Management
5. Management of Process Quality
6. Quality and Operational Results
7. Client Focus and Satisfaction

            The criteria for the instrument were modified to fit the public school environment, and the instrument contained 45 items (see Appendix).  Validity and reliability of the instrument was established using statistical inquiry, panel review, and research validation procedures.  The instrument was found to be both valid and reliable for use in educational organizations and institutions.  Respondents were instructed to indicate the current state of affairs on each item of quality, as well as the ideal, or preferred, state of affairs for each item.
            Two other instruments were developed for use as a part of the studies.  Both instruments were developed based upon information from professional literature, and verified by a panel of reviewers.  In the study component by Pamela Johnson, the Staff Development Questionnaire was developed and employed, and the Executive Compensation Questionnaire was developed and used in the study component managed by Thomas Behounek.  Joseph Kirchoff's study employed the use of publicly-documented data available from state government agencies.

Study Procedures

            In the comprehensive studies, it was important to adequately select a population of school districts reflective of the State of Iowa.  The population consisted of forty-four (44) school districts selected, invited, and willing to participate in the studies voluntarily.  The districts represented more than ten percent (10%) of the districts in the State of Iowa, and they were generally reflective of the state in terms of size, economic conditions, and geographic distribution.  The chief executive officers of each school system were requested to assign random distribution of the instrumentation to the stakeholders in their districts to a third party, and respondents were asked to complete all instruments, following the directions provided.
            Those eligible for completing the Perceived Quality Assessment Instrument were the superintendents, all board members, two members of the administrative staff, three members of the support staff, five teachers, and two high school students.  The comprised a total of 18-20 persons from each school district, depending upon the size of the board (five to seven members). All responses to all instruments were anonymous and voluntary.  The total possible number of questionnaires to be returned amounted to 720 each, of which 471 instruments were received.  This amounted to a return rate of approximately 65 percent.  Four districts were eliminated from the final analysis of data due to insufficient returns from those districts.
            The Staff Development Questionnaire was planned to be completed by six teachers in each of the forty-four districts, and they were not allowed to be the same teachers used in the PQAI instrumentation response.  Of a possible 264 instruments, 196 were returned for a response rate of approximately 80 percent.
            The Executive Compensation Questionnaire was administered to all superintendents and board presidents of the 44 school districts.  Remarkably, all 44 superintendents and all 44 board presidents responded by completing and returning the questionnaire.
            The financial information used in the studies was obtained from officially-filed audit reports from each school district from the offices of the Iowa State Department of Education.
            All data were gathered by June, 1994, and all responses were entered into a spreadsheet program for analysis and computation of means for current and ideal status.  The quality effectiveness index was also computed for each district on each item, sub-area, and for the total instrument.  District means were also computed for all instrument data from the Staff Development Questionnaires and the Executive Compensation Questionnaire.  Districts were then rank-ordered from high to low, and different groupings were used for data analysis and comparisons.

Findings

            Of those responding on the PQAI, superintendents (30 each) comprised 6.6% of the sample group, teachers (132) comprised 28.8%, support staff (76) comprised 16.6%,, board members (108) made up 23.6%, and students (44) comprised 9.6%.  Demographic data were also gathered on each of the responding individuals, including occupational positions, income, gender, age, levels of education, and work experience of individual respondents.
            Differences were noted between male and female respondents, which was significant overall, but not in any of the separate sub-areas of the instrument when broken down by gender.  Teachers were noted to differ from board members, and differences between administrators and teachers were also noted on the PQAI.  Respondents with a baccalaureate degree also differed from those without the degree.  No significant differences were noted in the demographic categories of age, income levels, or job experience.

PQAI Findings
            The perceived current level of quality and the ideal level of quality were calculated, and the means for the current status ranging from 2.81 to 3.92.  Ideal means ranged from 3.79 to 4.66.  Quality effectiveness indices were calculated, and ranged from .6532 to .9506.  Two districts had a QEI above .90, and three had a QEI below .70.  The majority (28 or 70%) of districts had a QEI above .75 indicating that those districts perceive that they are achieving at least 75% of their quality management goals.
            Interrelationships among the seven sub-areas of the PQAI were also found using a matrix of relationships, with moderate to high correlations among the factors.  A correlation of .77 was found between human resource development and management and strategic quality planning, .74 between client focus and satisfaction and quality and operational results, .73 for human resource development and management and management of process quality, and .71 between management of process quality and operational results.
            A determination was also sought to establish if the PQAI differentiated in terms of quality between high ranking and low ranking school districts.  Inferential statistics were used to assess the difference and established a significant difference between the ten highest rated and the ten lowest rated groups.  There was not only a significant difference between the high and low groups' quality effectiveness index, but there were also significant differences between the groups in each of the seven dimensions or sub-areas of the PQAI instrument.  Finally, the Perceived Quality Assessment Instrument was evaluated utilizing factor-analysis techniques, and results indicated that some items may need to be regrouped into three categories or sub-areas empirically developed but undefined.

Staff Development Findings
            A significant positive relationship was found between the perceived quality of district staff development and the perceived quality effectiveness index of the districts.  District means on the Staff Development Questionnaire were used to categorize the districts into four equally-sized groups according to their ranks, and there were significant differences between all four groups in their rankings of staff development practices.  In other words, significant differences were found between the districts ranked highest and lowest in perceived staff development quality on the quality effectiveness index.  The top ten districts demonstrated a modest correlation of .564 between the means of the QEI and the SDQ, indicating that districts which have higher quality staff development programs also have higher quality overall district management as measured by the Quality
Effectiveness Index slightly more than half of the time.
            Significant relationships were also found between ratings of perceived levels of district staff development and current ratings on three of the seven dimensions of the Perceived Quality Assessment Instrument.  Client focus and satisfaction, quality and operational results, and management of process quality were found to correlate positively and have predictive value with perceptions of quality in staff development.  The other four areas -- leadership, information and analysis, strategic quality planning, and human resource development and management -- showed limited correlative or predictive value.

Executive Compensation Findings
            Board presidents and superintendents showed agreement that the student enrollment levels should have little effect in determining compensation levels for the superintendent, and also agreed that standardized testing should not be used for such purposes.  That teacher union increases should be considered in any compensation increases for superintendents was also an area of agreement, as was the area calling for use of job description factors in compensation decisions.
            The differences between performance-related and situational-related (nonperformance) factors were evidenced between board presidents and superintendents with the former more supportive of using performance-based criteria than the latter.  However in an ideal setting, superintendents were more inclined to consider performance-based criteria more important than currently.  Performance-based compensation initiatives would find little or no support for such practices from these studies.  Ironically, the use of feedback and assessment information for improvement efforts appears to be contradicted to a degree in the perceptions of key organizational leaders -- superintendents and board presidents.
            A very weak finding was that there may be some credence to W. Edwards Deming's proscription of performance-based (merit) pay, in that a weak inverse relationship was found between performance-based criteria compensation support and the perceived quality of the systems.  Deming's notion that performance-based pay might have a deleterious effect upon organizational quality was confirmed in this study, albeit tenuously.

Financial Factors Findings
            Five selected financial characteristics and the financial solvency ratio were studied in relationship to perceived quality of the schools in the study.  The five factors included revenues per pupil, expenditures per pupil, undesignated, unreserved fund balance per pupil, unspent balance per pupil, and transportation cost per pupil.  Means of the financial characteristics for the study population were calculated, and statistical analyses were completed.
            Significant differences, using t-tests, were found between the high and low QEI groups in two areas -- revenues per pupil in the leadership sub-area, and transportation cost per pupil in the information and analysis category.  The other 46 possibilities were not found to be significant.  The perceived leadership quality increases slightly as the amount of revenue per pupil increases, and vice-versa for transportation cost and perceived quality in information and analysis.  No correlation was found between the sample schools financial characteristics and their any PQAI rating area with one exception -- transportation cost per pupil and information and analysis.

Conclusions and Recommendations

            The comprehensive studies of quality improvement perceptions and other selected factors in Iowa schools revealed several encouraging results and disappointments simultaneously.  The Perceived Quality Assessment Instrument appeared to function relatively well as evidenced by its significant discrimination between high and low rated systems.  It also discriminated between high and low perceived quality school districts on the sub-areas of the instrument.  Reliability and validity of the instrumentation was well within tolerable limits, and the ease of administration of the instrument contributed to a reasonable return of instruments.
            The instrument actually seems to be measuring three empirically-determined factors, but these remain undefined pending further research.  The strength of the instrument's utility lies in its capability to accurately measure a school district's quality improvement status against a definable, solid standard, but currently, the instrument is in need of further refinement.  The quality effectiveness index or ratio provides a convenient, easily understood criterion for comparing districts against the fixed standard, and when used, it does reveal to the organization its level of quality attainment against its own definition of ideal quality.
            The efficacy of staff development programs in relationship to organizational quality was supported in the results of these studies, although the relationship between staff development and overall quality was not as strong as might be expected.  However, effective staff development calls for resources, time, and strong leadership, and teachers' ratings may not be aware of the demands upon those things which require choices and limitations.
            An encouraging finding was the relationship of staff development quality and the perceived organizational quality levels of the PQAI.  It would stand to reason that districts that make strong commitments to staff development would also make commitments to other areas of quality improvement as well.  In addition, training of staff should have some beneficial effect upon effective practice and organizational functioning.  Excellence in training reasonably should be expected to produce excellence in organizational behavior.
            Individual analyses of all items in the seven dimensions of the PQAI show the existence of moderate to strong relationships among the group, indicating that they are dealing with factors that are connected.  The means of the ten districts ranking highest and lowest in staff development quality were compared with their means on the Quality Effectiveness Index, and the top ten were significantly different than the lowest ten, suggesting that although quality staff development is one of the factors important in the implementation of quality management.
            Another important outcome of these studies was found in both the executive compensation and financial factors components -- that some things are not related to quality.  The lack of strong confirmation that performance-based pay is related to or contributory to some level of difference is an important finding in light of continuing efforts to tie job compensation to individual performance.  Such a reward-penalty approach was not supported in these studies, but was not refuted all together.  The notion of performance-based or merit pay systems will no doubt continue to be popular in certain political arenas, but it did not gain vindication in these studies.
            Many educators have sought to establish a clear, straight line relationship between financial wealth of schools and the level of quality within the system.  Again, little support was brought about in these studies for such an inclination.  Although a couple of relationships were found between financial characteristics and perceived quality in these studies, but the absence of such verification encourages researchers and educational practitioners to seek contributory factors in other directions.

Recommendations

            Based on the findings and conclusions of these studies, a number of recommendations emerge.  The Perceived Quality Assessment Instrument appears to have considerable merit as a tool in determining the levels of quality in school systems, and as a tool, it might be effectively used as a team informational device in planning and developing improvement efforts.  The seven dimensions provide a framework for improvement actions at all levels in school organizations.  There should be no hesitation to proceed with continued development, research, and refinement of the perceived quality assessment process.
            Greater use of the instrument in research activities could fashion improvement in the instrument's quality, given feedback and revision of items and organization over time.  The three empirically-derived measurement areas provide an opportunity for other developmental activity.
            In addition, there is considerable room for use of the instrumentation in other studies and research efforts.  The PQAI and similar tools could focus on the improvement of practices and programs in schools by making results-oriented information available.  The common understanding of quality is yet to be defined in educational organizations, and any instrument like the PQAI should be considered for use in improving school performance, planning, training, and institutional assessment.  Without better understandings of these and other factors, improved productivity in school organizations rests upon a better grasp of quality than currently exists.
            Quality improvement work should also involve the use of growth-monitoring strategies.  Using the PQAI as a "baseline" establishing device after further refinement and development, would enable districts to measure current status, establish directions for improvement, and monitor progress toward clearly defined ends and goals.  Trend data, establishment of performance levels, and benchmarking information can all be derived from use of the PQAI, and more effective evaluation of system progress becomes attainable.

Closing Comments

            The purpose of these studies centered on the determination of whether or not the Perceived Quality Assessment was effective in measuring factors commonly associated with excellence, efficacy, and effectiveness in school organizations.  A second purpose was to identify the extent of any relationships and the nature of any associations among variables measured by the PQAI and other measures,  and the final purpose was to find whether or not operations and management actions in schools have an impact or influence upon the perceived level of quality within the system.
            Given the findings and outcomes of these studies, the studies' purposes have been achieved.  However, that is only part of the voyage.  Organizational quality is a continual process, not a destination, and enduring perseverance toward finding new and better ways to improve overall school effectiveness, use of resources and capabilities, and the overall development and well-being of clientele is essential for human progress.  The journey must go on.
 

References

Bates, M. and Stachowski, E. (1991).  Planning comprehensive staff development programs.  San Francisco, CA: Association for Supervision and Curriculum Development.

Bax, R.  (1994).  An assessment of the quality improvement climate as perceived by community college leadership in Iowa.  Doctoral dissertation, Iowa State University, Ames, Iowa.

Bax, R. M. (1994).  Relationships between quality ratings, job satisfaction, and organizational commitment in school districts implementing total quality management.  (Unpublished paper, Iowa State University, Ames,   Iowa, 2 May 1994).

Behounek, T. J.  (1996).  A study of the perceived quality factors and methods of awarding salary increases for   superintendents in selected school districts.  Doctoral dissertation, Iowa State University, Ames, Iowa.

Borg, W.R. and Gail, M.D. (1989) Educational research: an introduction.  New York:  Longman.

Bradley, L.H. (1993).  Total quality management for schools.  Lancaster, Pennsylvania:  Technomic Publishing Co.   Brown,  E.H. (1992).  On Deming and school quality: a conversation with Enid Brown (in Brandt, R., 1992).   Educational Leadership, 50(3), 28-31.

Deming, W.E. (1986).  Out of the crisis.  Cambridge, MA:  Massachusetts Institute of Technology Center for   Advanced Engineering Study.

Deming, W.E. (1990).  Quoted in Walton, M.  Deming Management At Work.  New York, NY:  Putnam.  P. 11.

Drystad, J. (1994)  Quoted in Rubach, L.  Transforming U.S. education with the quality movement.  Quality Progress.   27(9), 93-95.

Frase, L.E., English, F.W., and Poston, W.K.  (1995).  The curriculum management audit:  improving school   quality.  Lancaster, PA:  Technomic Publishers.

Fullan,  M.G. (1991).  The new meaning of educational change.  New York: Teachers College Press.

Graff, L. (1995).  Committing to growth through a learning organization.  (Paper published on the Horizon   "Listserv" online newsletter, 15 November 1995).  Horizon@gibbs.oit.unc.edu.

Guskey, T.R. and Sparks, D. (1991).  What to consider when evaluating staff development.  Educational   Leadership, 47(3), 73-76.

Guskey, T.R. (1994). The most significant advances in the field of staff development over the last twenty-five years.   Journal of Staff Development, 15(4), 5-6.

Hinkle, D.E., Wiersma, W., and Jurs, S.G. (1988).  Applied statistics for the behavioral sciences. (2nd Ed.)   Boston:  Houghton Mifflin.

Johnson, P. D. (1995).  An examination of teachers' perceptions of district staff development programs and their   relationship to perceptions of district quality management.  Doctoral dissertation, Iowa State University, Ames, Iowa.

Joyce, B. & Showers, B. (1988).  Student achievement through staff development.  New York:  Longman.

Kirchoff, J. E. (1996).  A study of quality improvement and its relationship to financial characteristics of selected   Iowa public school districts.  (Doctoral dissertation, Iowa State University, Ames. Iowa, 1996).

Knudson, B.  (1993).  Continuous improvement profile.  Minneapolis, MN:  Positive Directions, Inc.

Lezotte, L. (1994).  The nexus of instructional leadership and effective schools.  The School Administrator, 51 (6),   20-23.

Locke, E.A. (1976).  The nature and cause of job satisfaction. (in Dunnette, M.D., ed.).  Handbook of Industrial   and Organizational Psychology.  Chicago, Illinois:  Rand McNally.

Marchese, T. (1993).  A time for ideas.  Change, May-June, 1993, P. 11.

McBride, R., Reed, J., and Dollar, J. (1994).  Teacher attitudes toward staff development: a symbolic relationship   at best.  Journal of Staff Development, 15(2), 36-41.

Nunnally and Durham, 1975  (see Behounek)

Orlich, D.C. (1983).  Some consideration for effective inservice education.  The Clearing House, 56(5), 197-201.

Porter, L., Steers, R. Mowday, R., and Boulian, P. (1974).  Organizational commitment, job satisfaction,   and turnover among psychiatric technicians.  Journal of Applied Psychology, 59, 603-609.

Poston, W.K., Jr.  and Frase, L.E.  (1991).  Local teacher alternative compensation programs:  rolling boulders   up the mountain of reform.   The Phi Delta KAPPAN, 17(4)  317-320.

Poston, W. K. Jr. (1995).  Making Governance Work: Total Quality Education for School Boards.  Newbury   Park, California: Corwin Press. Poston, W. K. Jr. (1995).  Studies of the aspects of the use of the Malcolm   Baldrige national quality award criteria in the assessment of educational organizations.  (Paper presented to the   National Conference on Education at the American Association of School Administrators, New Orleans,   Louisiana, February, 1995).

Poston, W.K. Jr. and Bax, Rashid, PQAI

Rebore, R. (1991). Personnel administration in education: a management approach.  Englewood Cliffs, NJ: Prentice Hall.

Rowe, J.  (1994).  Outmoded leadership ideas won't work in (the) 21st Century.  (Article in the Des Moines Register,   27 June 1994, Page 8-B).

Schenkat, R. (1993).  Deming's quality: our last but best hope.  Educational Leadership, 51(1), 64-65.

Schenkat, R. (1993).  Quality Connections: Transforming Schools Through Total quality Management.  Alexandria,   Virginia: American Association of School Administrators.

Schmoker, M. and Wilson, R.B.  (1993).  Transforming school through total quality education.  The Phi Delta   KAPPAN, 74(5), 389-395.

Senge, P. (1990).  The fifth discipline: the art and practice of the learning organization.  New York:  Doubleday.

Showers, B.  (1985).  Teachers coaching teachers.  Educational Leadership, 42(7), 43-48.

Sparks, D. and Loucks-Horsley, S. (1989).  Five models of staff development for teachers.  Journal of Staff   Development, 10(4), 40-57.

Sparks, D. and Vaughn, S. (1994).  What every school board members should know about staff development.   Journal of Staff Development, 15(2), 20-22.

Teeter, D.J. and Lozier, G.G. (1993). Six foundations for total quality management.  In D.J. Teeter and   G.G. Tozier, (eds.).  Pursuit of quality in higher education: case studies in total quality management.   New Directions for Institutional Research, No. 78, pp. 5-11.  San Francisco, California: Jossey-Bass Publishers.

Teigland, M.D. (1995).  A study of the beliefs for total quality management comparing superintendents, board   members, and classroom teachers in Iowa schools.  Doctoral dissertation, Iowa State University, 1993.

Templeman, T.P. and Peters, J. (1992).  Teaching research inservice model.  Monmouth, OR: Western   Oregon State College.

U. S. Department of Commerce.  (1995).  The Malcolm Baldrige National Quality Award:  Education Pilot   Criteria, 1995.  National Institute of Standards and Technology, Washington, DC.

U.S. General Accounting Office.  (1993).  System-wide education reform: Federal leadership district-level efforts.   Report to Congressional Requesters.  U.S.G.A.O.: Washington, DC.

Vincoli, J.  (1991).  Total quality management and the safety and health professional.  Professional Safety, 6, 27-32.

Wade, R.K. (1985).  What makes a difference in in-service teacher education?  A meta-analysis of research.   Educational Leadership, 42(4), 48-54.

Wood, F., Killian, J., McQuarrie, F., and Thompson, S.  (1993). How to organize a school based staff   development program.  Alexandria, VA:  Association for Supervision and Curriculum Development.
 

Appendix:

Sample Instruments

Perceived Quality Assessment Instrument

Staff Development Questionnaire

Executive Compensation Questionnaire

Note:  Copies of instruments may be obtained from:

William K. Poston Jr.
Department of Professional Studies
N229 Lagomarcino Hall
Iowa State University
Ames, Iowa  50011

About the Author

William K. Poston Jr., Ed.D.

Associate Professor, Educational Administration
Department of Professional Studies
N225 Lagomarcino Hall
Ames, Iowa  50021
Phone:  (515) 294-9468
Fax:  (515) 294-4942
Email:  wkpjr@iastate.edu
Iowa State University

          Dr. Poston is currently Associate Professor of Educational Administration in the Department of Professional Studies at Iowa State University in Ames, Iowa.  Bill is a former math and physical science teacher and an experienced administrator, with 25 years of experience in school administration including 15 years as a superintendent in the Flowing Wells (Tucson) and Kyrene (Tempe-Phoenix) Schools in Arizona and in the Billings Public Schools, in Montana.  Bill earned his B. A. at the University of Northern Iowa, and his Ed. S. and Ed. D. at Arizona State University.  He has many distinctive professional achievements, including service as the youngest-elected international president of Phi Delta Kappa, and selection as an Outstanding Young Leader in American Education in 1980.
            Bill has authored many articles and has recently published four books:   Making Schools Work:  Practical Management of School Operations (Corwin Press, 1992),    Effective School Board Governance (Phi Delta Kappa International, 1994),   Making Governance Work:  Total Quality Education for School Boards, (Corwin Press, 1994),   The Curriculum Audit: Improving School Quality (Co-author, Technomic Publishing, 1995).
            Dr. Poston teaches courses in contemporary management, strategic planning, school finance, and educational leadership at Iowa State University.  He is also the executive director of the Iowa Superintendents' Academy and the Iowa School Business Management Academy.  He continues to provide extensive service to schools in the areas of evaluation, curriculum management auditing, performance-based budgeting, and continual quality improvement.
        Bill has been married for 35 years to his wife, Marcia, and they have two daughters, both of whom are teachers in the West Des Moines Community Schools.