Factors Affecting the Impact of Professional Development Programs On Teachers ’ Knowledge , Practice , Student Outcomes & Efficacy

This report examines effects of structural and process features of professional development programs on teachers’ knowledge, practice and efficacy. It is based on four recent (2002-2003) studies undertaken through the Australian Government Quality Teacher Programme, designed to enhance teacher quality. The total data set for the survey study includes 3,250 teachers who had participated in eighty individual professional development activities within these studies. Teachers were surveyed at least three months after participating in an activity, which provided them with the opportunity to gauge the impact of programs on their practice. To investigate factors affecting impact, a theoretical model was developed based on recent research into the characteristics of effective professional development and tested using blockwise regression analysis. The model included contextual factors (e.g., school support), structural features of programs (e.g. ,length), process features (e.g., emphasis on content; active learning; examination of student work; feedback; follow-up), a mediating variable (level of professional community generated), and four outcome measures (knowledge; practice; student learning and efficacy). Consistent significant direct effects were found 1 In this article, “programme” refers to the Australian Government Quality Teacher Programme (AGQTP). Projects funded under AGQTP consist of either State and Territory professional learning projects or national strategic projects (see p. 3 for further description). State and Territory professional learning projects develop and deliver professional learning activities for teachers, either on a State or Territory – wide basis or for individual education authorities within that State or Territory. Education Policy Analysis Archives Vol. 13 No. 10 2 across the four studies for the impact of content focus, active learning, and follow-up on knowledge and professional community. Feedback was rarely incorporated into program design. Impact on efficacy was strongly related to the perceived impact of activities on teachers’ practice and student learning outcomes.


Introduction
The role of the public school superintendent is both challenging and varied, and it has been that way for quite some time (Callahan, 1962;Owen, 1998).In recent years, accountability requirements and increased standardization associated with the No Child Left Behind Act (NCLB) may have added to the complexity of the role (Farkas, Johnson, & Duffett, 2003).But little research has explored the influence of such requirements or the impact of their perceived stringency on the work lives of superintendents.Moreover, only a few recent studies have examined the association between contextual factors and superintendents' performance of their roles, and to our knowledge no study has investigated the relationship between these contextual factors and superintendents' perceptions of the stringency of accountability requirements.
Nevertheless, speculation based on prior research suggests that contextual factors and the stringency of accountability measures-real, perceived, or both-might combine in certain ways to influence the attention superintendents pay to the different roles comprising their work.In this study we explore these relationships through a series of quantitative analyses using data from four states, two of which have stringent accountability requirements and two of which have lenient accountability requirements.These analyses gave us the basis for answering several research questions pertinent to an understanding of superintendents' work in the age of accountability.As discussed below, we used allocation of time as a proxy measure of superintendents' attention to certain roles.First, we asked how superintendents allocate their time across managerial, educational, and political roles.Next, we investigated the relationship between location in a state with stringent or lenient accountability and superintendents' perceptions of the stringency of accountability measures.Third, we asked to what extent do contextual variables-district size, district locale, percent free and reduced lunch, percent minority, and per-pupil expenditure-independently and in combination influence superintendents' perceptions of the stringency of accountability measures.Fourth, we asked to what extent contextual variables independently and in combination influence superintendents' propensity to devote time to managerial leadership.Finally, with covariates controlled, we explored the relationship between location in a state with stringent or lenient accountability and superintendents' allocation of time to managerial leadership.

Theoretical Framework
The existing literature about the superintendency characterizes the functions performed by superintendents in various ways.Typologies assign these functions to several distinct roles.Threepart typologies that distinguish among educational, political, and managerial roles enable a fairly exhaustive categorization of the superintendent's functions, and these typologies are used more often than other typologies in studying superintendents' work.For example, on the basis of his analysis of historical data, Cuban (1976) seems to have been the first to identify three roles: Teacher-Scholar, Chief Administrator, and Negotiator-Statesman. Later, he used the terms instructional, managerial, and political leadership to refer to these three roles (Cuban, 1988).More recently, Johnson (1996) used a similar classification when she described the superintendency as encompassing roles in educational, political, and managerial leadership.Her typology was based on an analysis of the work undertaken by 12 newly-hired superintendents (Johnson, 1996).
In Cuban's and Johnson's typologies, the managerial role encompasses planning and administrative functions.The superintendent exercises authority over personnel, finance, and facilities.Exercising these managerial functions, the superintendent strives to ensure organizational stability while also making the district accountable to the public.The superintendent exercises educational (or instructional) leadership by formulating the district's vision, focusing particularly on curriculum and instruction.Recent literature on district-level influences on student achievement also demonstrates that superintendents exercise educational leadership by promoting educational coherence and by working to cultivate instructional capacity (e.g., Cawelti & Protheroe, 2001;Leithwood, Louis, Anderson, & Wahlstrom, 2004;Snipes, Doolittle, & Herlihy, 2002;Waters & Marzano, 2006).Finally, the superintendent assumes a political role to "secure[e] cooperation" from various social groups with the goal of "making outside conditions favorable to successful management" (Cuban, 1976, p. 18).Functioning within this role, the superintendent negotiates with diverse interest groups to reach agreement about district priorities, policies, and resource allocation (Cuban, 1988;Johnson, 1996).One recent study suggested, however, that power dynamics between superintendents and other influential groups (e.g., schools boards and teachers unions) tend to make performance of this role particularly difficult for superintendents of large districts (Fuller, Campbell, Celio, Harvey, Immerwahr, & Winger, 2003).
Literature describing the functions that comprise the superintendent's educational, managerial, and political roles reveals the range of activities performed by these school administrators.Because role enactment is difficult to measure directly, researchers sometimes use the proxy measure of time allocation to examine superintendents' performance of the functions associated with the roles they fill.In fact, studies of time allocation have long been used to examine workers' roles (Burns, 1954;Copeman, Luijk, & Hanika, 1963).In one of the most famous of these studies, Mintzberg (1973) investigated administrators' time allocation to determine the functions that comprised their roles.Studies such as Mintzberg's highlight the consistencies in managers' allocation of time, but there is also variability.Considering the variation in the circumstances confronting different school districts, moreover, it seems likely that a particular superintendent's role consists of a unique combination of functions falling within the three broad role categories.In fact, a few studies do suggest that the degree to which individual superintendents perform functions in each of these three role categories is influenced by factors relating to district context (Fuller et al., 2003;Johnson, 1996).
Despite evidence of variability in superintendents' role performance, certain scholars argue that the optimal role for superintendents is one that focuses primarily on educational leadership (Bjork, 1993;Carter & Cunningham, 1997;Konnert & Augenstein, 1990).Moreover, some research reveals a positive association between superintendents' attention to educational leadership and teachers' confidence, competence, and willingness to take responsibility (Kussy, 1995).Nevertheless, time constraints, role overload, and situations involving conflict often limit superintendents' ability to focus on matters relating to curriculum and instruction (Bredeson, 1996;Fuller et al., 2003;Wirt, 1991).In contrast to those arguing for the priority of the educational leadership role, Johnson (1996) advocates a balance among the three roles.Findings from her study revealed that the failure of a superintendent to pay attention to any of the roles had a negative impact on the district.For example, one superintendent's failure to attend to the educational role diminished that leader's credibility with teachers and principals.Inadequate attention to political leadership resulted in conflict and financial hardship.Lack of focus on managerial functions led to poor communication and distracting bureaucratic errors.
What these perspectives have in common is the belief that excessive focus on managerial responsibilities is likely to detract from effectiveness.As a consequence, conditions that predispose superintendents to attend primarily to the managerial role might work against the sorts of reform that accountability policies hope to promote.It would be ironic if those same policies either directly or indirectly promoted greater attention among superintendents to the managerial domain.This study examines this possibility along with the possibility that other district circumstances might singly or in combination require superintendents to devote more of their time to management than to other domains of leadership.

Related Literature
Although some studies such as Cuban's (1988) historical analysis suggest the possibility that context has an influence on superintendents' apportionment of time, few studies of contemporary superintendents examine such influences directly.Moreover, most of the studies that are available focus on district size as the contextual variable of interest.Other possible contextual influences have not received much attention from researchers.And only a few studies tangentially address the influence of accountability measures on the attitudes and practices of local superintendents (e.g., Cohn, 2005;Whitney & Grogan, 2003).Interestingly, one study (Farkas et al., 2003) seemed to indicate that the locale of districts influenced superintendents' level of support for the accountability measures included in the federal No Child Left Behind Act (2002).Urban superintendents in this study were more supportive of these measures than were their rural and suburban counterparts.Of course, district locale and district size are often confounded because urban districts tend to be larger than districts in other places, and several early studies of the influence of district size on superintendents' performance of their roles fail to distinguish the influence of locale from the influence of size.Nevertheless some of these studies showed that superintendents of larger districts tended to devote more time to educational leadership while superintendents of smaller districts tended to devote more time to managerial leadership.However, two more recent studies offered counterevidence.Duea and Bishop (1980) concluded that superintendents in larger districts focus more on political and educational leadership while those in smaller districts focus more on managerial leadership.These researchers compared superintendents' rankings of the time devoted to different administrative tasks.Specifically, the rankings of superintendents from the largest districts were compared with those from a random sample of small and mid-sized districts.Results revealed that superintendents of large districts spent the most time on political and educational tasks, while those in other districts spent the most time on managerial tasks.Similar findings were reported by Larson, Bussom, and Vickers (1981), who investigated the work lives of six Midwestern superintendents.Their study revealed that there were large differences in the amounts of time that the superintendents spent on different activities, and these differences seemed associated with district size.For example, the percentage of time devoted to desk work and phone calls was inversely proportional to district size.Compared to leaders of other districts, the superintendent of the largest district spent less time on desk work and travel but more time on unscheduled contacts.
Similarly, Lindsey (1989) observed differences in superintendents' time allocation based on district size.Although, like the sample of Larson and associates, his sample was small and he used only descriptive statistics, his data suggested that superintendents of small districts spent comparatively less time on instructional activities but comparatively more time on political activities than superintendents of larger districts.Munther (1997) also reported differences in superintendents' allocation of time based on district size, but his analysis revealed a somewhat different pattern from the one reported in the three studies discussed above.Using Parson's (1960) model of organizational levels of responsibility and control, in which organizational functions are categorized as technical, managerial, or institutional, Munther found that superintendents in small and mid-sized districts spent the most time on managerial tasks.In comparison, superintendents in the largest districts (i.e., over 10,000 students) spent the most time on tasks classified as institutional.Because institutional functions (in Parson's scheme) can be equated to what Cuban and Johnson classified as political functions, Munther's findings seem to suggest that superintendents in larger districts devote more time than those in smaller districts to activities associated with political leadership.Munther also noted that, in general, superintendents in smaller districts tended to pay more attention to technical functions than did superintendents in larger districts.Considering the similarity between what Parsons classified as technical functions and what Cuban and Johnson classified as educational (or instructional) functions, this finding implies that superintendents in smaller districts might be spending more time than their counterparts in larger districts on educational leadership.
According to a recent study (Fuller, Campbell, Celio, Harvey, Immerwahr, & Winger, 2003), particularly those in large urban and suburban districts, the job of superintendents has become so burdensome that many find it impossible to accomplish productive work in any of the three domains.In general, superintendents who were interviewed by these researchers reported that much of their time was devoted to issues relating to employment-issues that were exacerbated by persistent cronyism and patronage politics.Other political dynamics such as labor negotiations, competing demands of constituents, and accountability mandates also consumed a great deal of superintendents' time, leaving them with diminished ability to address other leadership needs.Although this study did not explicitly examine superintendents' allocation of time to different roles, its findings did imply that superintendents spent a great deal of time on activities relating to politics (construed broadly).The limited research on the relationship between contextual conditions and superintendents' allocation of time to different roles suggests tentatively that superintendents in smaller (perhaps rural and suburban) districts tend to pay disproportionate attention to managerial functions and that superintendents in larger (perhaps urban) districts tend to pay disproportionate attention to political functions.Clearly, however, with so few studies contributing to an understanding of such influences, generalizations are premature.The study reported here adds to the extant literature by incorporating contextual influences other than district size, by examining the combined effects of contextual influences, and by examining relevant relationships in consideration of the real and perceived stringency of accountability measures.

Methods
Using data obtained through a large-scale survey of superintendents in four states, the study used post hoc analyses to explore relationships between a set of variables representing superintendents' perceptions of the stringency of accountability measures and their allocation of time-a proxy variable measuring their attentiveness to various leadership domains.

Population and Sample
The population consisted of superintendents employed in public school districts in four states: Texas, Tennessee, New Hampshire, and Nebraska.Two of the states-Texas and Tennessee -represented jurisdictions in which state education agencies applied stringent accountability measures, and two of them-New Hampshire and Nebraska-represented jurisdictions in which state education agencies applied much less stringent measures.
The decision to draw samples from states in which the extent of state-level control differed was supported by research showing that external threats exert an influence on organizational leadership (e.g., Fitzpatrick & Rubin, 1995;Mulder, deJong, Koppelaar, & Verhage, 1986;Mulder, Ritsema van Eck, & deJong, 1970).These studies were conducted in business organizations; but evidence about the stressors associated with accountability mandates suggested the possibility that similar dynamics might be at play in the education arena (e.g., Schoen & Fusarelli, 2008).
We used a systematic procedure to select states that represented extreme cases along the continuum of loose to stringent accountability.Their selection was made on the basis of the following criteria: the presence or absence of rewards and sanctions based on district performance, the extent of these sanctions and rewards, and the extensiveness of accountability testing requirements within the state.The selection process involved the following procedures.First, based on information in the document Rewards and Sanctions for School Districts and Schools (Education Commission of the States, 2001States, , 2002)), states were divided into four categories reflecting the presence or absence of rewards or sanctions at the district level."Stringent accountability" states used sanctions alone or a combination of sanctions and rewards, while "loose accountability" states used rewards only or used neither sanctions nor rewards.Next, within each of the four categories, states were ranked from highest to lowest based on the mean rankings for two conditions: the extensiveness of their testing programs, including number of different grade levels tested and total number of tests administered (range = 0-21) and a state sanction score, which comprised the number of sanctions that were permissible in each state (range = 0-17).The average of these two rankings yielded a number that was taken to represent stringency within category.The selection process also took into account feasibility issues such as whether or not states had sufficient numbers of superintendents representing each of the demographic categories salient to the study, particularly locale and district size.In addition to these considerations, we also sought to find states that were geographically widespread across the contiguous United States.
Based upon the selection process and considerations described above, four states were selected.Tennessee, ranked first with regard to stringency of accountability, was selected to represent the category of states that offered sanctions without rewards, while Texas, ranked fourth overall in level of stringency, was chosen to represent states that offered both sanctions and rewards.New Hampshire, ranked 44th, was chosen to represent states that offered neither sanctions nor rewards, while Nebraska, ranked 50th, was the only state to offer rewards without imposing sanctions.
Several other recent studies (e.g., Amrein & Berliner, 2002;Carnoy & Loeb, 2002;Clarke, Shore, Rhoades, Abrams, Miao, & Li, 2003;Nichols, Glass, & Berliner, 2006;Pedulla, Abrams, Madaus, Russell, Ramos, & Miao, 2003) have likewise attempted to categorize states according to the stringency of accountability measures.Like the methodology employed in the current study, the various selection procedures used in these studies ranked or grouped states according to accountability demands.And like the current study, these studies used selection processes that derived these rankings or groupings based on one or more of the following factors: the extent of testing across grade levels, and the number and severity of sanctions and rewards.
Rankings yielded by the selection process used in the current study were largely aligned to those rankings (or groupings) produced by the various selection processes used in the other studies.As was the case with the current study, four other studies ranked both New Hampshire and Nebraska as states with lower accountability demands (Amrein & Berliner, 2002;Carnoy & Loeb, 2002;Clarke et al., 2003;Pedulla et al., 2003).In all four ranking systems, as well as in the one employed in the current study, Texas ranked as a state with highly stringent accountability.Finally, the selection of Tennessee as a high-stringency state was also generally though not universally supported.While the common ranking system employed by two studies (Clarke et al., 2003;Pedulla et al., 2003) ranked Tennessee as restrictive, systems used in some other studies placed it in the upper middle (Amrein & Berliner, 2003;Nichols et al., 2006) or bottom (Carnoy & Loeb, 2002) with regard to strictness of accountability measures.
Once we selected the four states, we made decisions about sampling.We drew random samples of superintendents working in public school districts in Texas and Nebraska and included all superintendents working in public school districts in Tennessee and New Hampshire.In Texas, our sample of 528 superintendents (out of 1041 superintendents altogether) was large enough to produce results that were representative of the population at the 95% confidence level with a 3% confidence interval; and in Nebraska, our sample of 234 (out of 267 superintendents altogether) was large enough to produce a similar degree of accuracy.

Instrumentation and Variables
We collected data using a two-page questionnaire (see Appendix A).It included items pertaining to time allocation among leadership roles, items eliciting demographic information, and items comprising a scale to measure perceived stringency of accountability.We pilot-tested the instrument with a convenience sample of 49 Ohio superintendents.Data from the pilot test enabled calculation of the internal consistency of the scale measuring perceived stringency, and a Cronbach Alpha reliability estimate of .72 suggested that the scale had sufficient internal consistency to be used for research purposes (e.g., Garson, 2002).Table 1 provides a list of variables and operational definitions used in this study.

Data Collection and Analysis
We mailed questionnaires to the superintendents in the sample and sent a reminder postcard to non-respondents after two weeks, followed by a duplicate questionnaire five weeks after the initial mailing.Of the 941 superintendents included in the sample, 683 returned usable questionnaires, providing an overall response rate of 73%. 1  We entered all data into a database and checked them for usability.Cases were omitted in which summation errors indicated that participants may not have understood the instructions.Univariate outliers past three standard deviations were also removed, along with multivariate outliers identified by computing Cook's Distance and the Mahalonabis Distance.After the removal of outliers, the overall net response rate for all four states was 68% (644/941 superintendents).The net response rates for individual states were as follows: Texas, 67% (352/528); Tennessee, 69% (94/136); New Hampshire, 60% (48/80); and Nebraska, 76% (150/197).Examination of frequencies, histograms, and computed skewness showed that most variables were normally distributed.An exception was enrollment, which showed a substantial positive skew.To correct for the unacceptable level of skewness, natural log transformations were computed and substituted for original enrollment values.Skewness for the transformed variable, log-enrollment, was minimal.Variables used in the regression analyses were also checked for multicollinearity.
1 Some problems were encountered in identifying the total population of superintendents in New Hampshire and in drawing a simple random sample of superintendents in Nebraska.First, contact information obtained from the 2001-2002 Common Core of Data provided the names and addresses for districts no longer in existence in fall 2003 when the questionnaire was mailed.Second, in both New Hampshire and Nebraska, some superintendents serve as leaders in more than one district.The fact that there was not a one-to-one correspondence between superintendents and districts caused two errors with respect to the eventual obtained sample.The size of the population of superintendents was overestimated so more superintendents were surveyed than actually needed to be surveyed, and some superintendents were sent two (or even more) copies of the questionnaire.These circumstances may have had some effect on the randomness of the sample, but the magnitude of that effect is impossible to quantify.Once alerted to the problem, we were able to remove duplicate submissions from those Nebraska and New Hampshire superintendents who mailed back more than one response.And, in some cases, where district demographics differed across the districts served by the same superintendent, they were able to create values representing aggregations across the multiple districts.

2.97
The extent to which the independent variables were collinear was assessed using two standard statistical tests, tolerance and Variance Inflation Factor (VIF).These tests yielded acceptable values; for none of the seven variables did tolerance exceed 1.0 nor fall under 0.4, while no VIF exceeded 2.5.Finally, we created two derived variables, both dichotomous.One classified superintendents based on their location in high-stringency or low-stringency states, and the other categorized them as either apportioning the majority of their time (50% or more) to managerial leadership or apportioning less than the majority of their time to managerial leadership.

Findings
Our discussion of findings provides answers to the five research questions listed above.To contextualize these findings, readers may wish to examine descriptive statistics, which are presented in Table 2.

How Superintendents Allocate Time to Leadership Roles
To assess superintendents' allocation of time, we computed mean comparisons using pairedsample t-tests.We performed these analyses separately for each state and also using the four-state data set.With the combined data, mean comparisons revealed that superintendents devoted the most time to managerial leadership, followed by educational and then political leadership.These significant differences as well as the state-by-state comparisons are shown in Table 3. Analyses using Texas data revealed the same pattern of significant differences.With Tennessee data, the pattern was the same, but only the difference between time allocated to managerial leadership and the time allocated to educational leadership was significant.Again, in Nebraska, the pattern was the same, but significant differences were evident in only two sets of comparisons-between managerial and educational leadership, and between managerial and political leadership.And in New Hampshire, the pattern was the same but t-tests revealed that none of the differences was significant.

Real and Perceived Stringency
To answer the research question, "What is the relationship between location in a state with stringent or lenient accountability and superintendents' perceptions of the stringency of accountability measures?"we used one-way analysis of variance to compare mean scores on the total sanction scale from superintendents in high-stringency states (i.e., Texas and Tennessee) with mean scores from superintendents in low-stringency states (i.e., Nebraska and New Hampshire).
The comparison of means revealed significantly higher scores in the high-stringency states (F (2, 643) = 118.99,p < .0001).The mean score from superintendents in high-stringency states was 13.42 (SD = 2.45) in contrast to a mean score of 10.08 (SD = 2.72) from those in low-stringency states.The difference remained the same when we expanded the model to introduce enrollment as a covariate-a control that we thought was wise to include because of its low, but significant correlation with the total sanction scale (R2 = .15,p = .01).Estimated marginal means calculated using the expanded model were equivalent to actual means obtained when the covariate was omitted.These analyses showed that even with enrollment controlled, superintendents in highstringency states perceived accountability measures to be more stringent than did superintendents in low-stringency states. 2

Predictors of Perceived Stringency
To answer the question, "to what extent do contextual variables-district size, district locale, percent free and reduced lunch, percent minority, and per-pupil expenditure-independently and in combination influence superintendents' perceptions of the stringency of accountability measures?"we constructed multiple regression equations using four state-level data sets.We found it necessary to disaggregate the data by state because these four states differed dramatically with regard to several of the independent variables.A review of the descriptive statistics provided in Table 2 reveals that the most dramatic differences are observed with respect to enrollment, free and reduced lunch rates, and percent of students from minority groups.The greatest difference in mean enrollment was between Tennessee, with a mean of 7,616 and Nebraska, with a mean of 1,090.For free and reduced lunch rates, the greatest difference was between Texas, with a mean rate of 51% and New Hampshire with a mean rate of 18%.These states also had the greatest difference in percentage of students from minority groups, with averages of 37% and 4% respectively.
We tested the same regression model with the data sets from each of the four states, but the model was significant only with the data set from Texas.Perhaps this situation resulted from the fact that the Texas data set was considerably larger than those from the other three states.Nevertheless, even with the Texas data, we found quite modest effects.Overall, the independent variables explained approximately 2% of the variance in perceived stringency, with rurality and enrollment exerting significant influences.Superintendents in rural districts were more likely than their suburban counterparts to perceive accountability as stringent, and superintendents in larger districts were more likely than those in smaller districts to perceive accountability as stringent (see Table 4).

Predictors of Managerial Leadership
Two sets of state-level analyses enabled us to address the research question, "To what extent do contextual variables-enrollment, percent free and reduced lunch, percent minority, per pupil expenditure, urban locale, rural locale, and perceived stringency of accountability-independently and in combination influence superintendents' propensity to devote time to managerial leadership?"3 Texas.The multiple regression equation with managerial leadership as the dependent variable was found to be significant at the .001level, with the independent variables accounting for 18% of the total variance in the dependent variables (Adj R 2 = .18,F (7, 344) = 12.24, p < .001).Enrollment and rural locale were significant predictors of time spent on managerial leadership.Superintendents from smaller and those from rural districts reported allocating more time to managerial tasks than did superintendents of larger and non-rural districts (see Table 5).
In the first set of analyses we tested the influence of relevant contextual variables on superintendents' allocation of time to managerial leadership using multiple regression.In the second set we used logistic regression to test the influence of those same contextual variables on superintendents' membership in one of two groups-the group spending more than 50% of time on managerial leadership or the group spending less than 50% of time on managerial leadership.Findings for Texas and Tennessee are presented first.These are the states we identified as having highly stringent accountability systems.Then findings for the two states with more lenient accountability systems-New Hampshire and Nebraska-are presented.

4
The logistic regression equation painted a similar, but somewhat fuller picture.In this statistically significant equation including the six independent variables targeted in this study [χ 2 (7, N = 352) = 58.08,p < .001,58%], enrollment and rural locale predicted membership in the group of superintendents who reported spending 50% or more of their time on tasks associated with managerial leadership.In this equation urban locale also predicted attention to managerial tasks, though to a lesser degree.Table 6 presents Wald statistics and odds ratios for each variable in the equation.Tennessee.The multiple regression equation with managerial leadership as the dependent variable was found to be significant at the .05level, with the independent variables accounting for approximately 10% of the total variance in the dependent variables (Adj R 2 = .098,F (7, 86) = 2.44, p < .05).Enrollment was the only significant predictor of time spent on managerial leadership.Superintendents from smaller districts reported allocating more time to managerial tasks than did superintendents of larger districts (see Table 5).Results of the logistic regression analysis showed the same pattern-enrollment was the only independent variable that contributed to explaining membership either in the group of superintendents spending 50% or more of their time on managerial leadership or in the group spending less than 50% of their time on managerial leadership. 5ebraska and New Hampshire.With the Nebraska data set, the regression model examining predictors of managerial leadership was significant (Adj R 2 = .06,F (6,143) = 2.69, p < .05),with enrollment exerting a significant negative influence (see Table 5).The logistic regression equation using Nebraska data, however, was not significant.Using the New Hampshire data set, we found that neither the multiple regression equation nor the logistic regression equation was significant.These findings are not surprising considering how little variability in enrollment there is among the New Hampshire districts (see Table 2).
Cross-State Trends.In general these analyses suggest that the lower the district enrollment, the more likely it is that the superintendent will devote a substantial amount of time to managerial leadership.In addition, even though rurality tends to be associated with lower district enrollment, it did exert a separate influence on the amount of time that Texas superintendents devoted to managerial leadership.6

Influence of State Policy Environment
We created an analysis of covariance model to answer the research question, "With covariates controlled, what is the relationship between location in a state with stringent or lenient accountability and superintendents' allocation of time to managerial leadership?"The model included state accountability context (i.e., location in a stringent or loose accountability state) as a fixed factor and two covariates (enrollment and locale) that had previously shown an influence on the percent of time superintendents allocated to managerial leadership.As we expected, enrollment was significantly associated with managerial leadership, but location in a stringent or lenient accountability state had no significant influence.The influence of locale was also non-significant (see Table 7).This finding suggests that circumstances associated with lower and higher district enrollments have a demonstrable influence on superintendents' allocation of time to managerial leadership, regardless of the stringency of the accountability requirements under which those superintendents operate..81 R2 = .18(Adj. R2 = .17)

Summary of Findings
We found that states differ with regard to the stringency of their accountability measures, and superintendents' perceptions of the stringency of state accountability measures relate to their location in high-or low-stringency states.To some degree, then, superintendents appear to be accurate interpreters of their state policy contexts.Nevertheless, structural conditions, namely enrollment and rural location, also appear to influence their perceptions to a modest extent.In terms of perception, then, accountability measures seem more stringent to superintendents in highstringency states, rural locales, and larger districts.Interestingly, however, neither the stringency of accountability measures nor superintendents' perceptions of that stringency appear to influence their choices with regard to the leadership responsibilities they emphasize.Rather, certain structural conditions, in particular low district enrollment and rural locale, seem to predispose superintendents to spend more time on activities associated with the managerial role.

Limitations
The models tested in this study were limited in several ways.The first relates to the use of self-report data, which are susceptible to social desirability bias.In addition to limitations associated with self-reporting, the accuracy of responses was also dependent on the alignment between conceptual categories provided on the questionnaire and conceptual categories arising from superintendents' actual lived experience.If the alignment was close, superintendents clearly understood the concepts to which questionnaire items referred.If the alignment was not close, however, superintendents might have either failed to understand items or might have read into them meanings derived from their own experiences, and these meanings may have differed considerably from the meanings intended by the researchers.
Another limitation begins in the way variables were operationalized.The consolidation of locale codes into three categories, for example, reduced precision and thereby resulted in a loss of potentially relevant variability.Furthermore, the need to represent locale as a dummy variable in the regression equations (e.g., rural/not rural, urban/not urban) resulted in an additional loss of variability.
As mentioned in the footnote above, some sampling problems may also have compromised our findings.In addition, our use of samples from four states, rather than a national sample means that our findings cannot be generalized definitively to superintendents across the nation.Moreover, our discovery of so much variability across states suggests that state context does matter.Our approach to this circumstance was to construct our models on a state-by-state basis, with one exception.To test the influence of being in a high-or low-stringency state, we obviously needed to perform analyses using the entire multi-state data set.
A final limitation relates to the models we used to predict perception of the stringency of accountability and allocation of time to managerial leadership.Although the models incorporated a set of independent variables that were theoretically (and in some cases empirically) linked to the dependent variables, other variables, excluded from the model, are likely to have been at least as influential.Other possible predictors might relate to the personal characteristics of superintendents, their leadership styles, and their professional training.

Discussion
The findings from this study support earlier research showing that district size and locale strongly influence superintendents' leadership (Duea & Bishop, 1980;Gilstrap, 1982;Larson et al., 1981;Lindsey, 1989;Munther, 1997).The portrait of small-and rural-district superintendents suggested by the current study fits in with findings from some previous research indicating that superintendents in smaller (and rural) districts perceive themselves primarily as managers who are required to perform a broad range of administrative tasks in multi-faceted roles (Glass, 1992;Grady & Bryant, 1991;Hoyle, 1978;Kaiser & Webb, 1974;Sabatino, 1993;Tagg, 1982).
Undoubtedly some interpreters would see these findings as evidence that smaller (and also rural) districts are deficient in comparison to larger (suburban) ones.But this claim seems premature.After all, several studies using state data sets have demonstrated that small district size confers an achievement advantage as well as reducing achievement gaps ( Howley, Brooks, Gocmen, & Hammer, 2002;Bickel & Howley, 2000). 7ore defensible, however, is the claim that smaller (and rural) districts have different leadership requirements than larger (suburban and urban) ones.Perhaps because there are fewer administrative personnel in smaller (and rural) districts, superintendents take on more managerial responsibilities.Or possibly, the freedom from political turmoil associated with small size or rurality enables superintendents to spend less time handling conflict and more time on organizational routines.Whatever the causal linkages, however, the fact remains that as districts become larger the character of the superintendent's work changes.
This insight about district leadership is not new, of course.But its continued salience is somewhat surprising in light of changes in educational policy as well as normative claims made in the literature on school leadership.For instance, we expected that the actual or perceived stringency of state accountability measures would influence superintendents' allocation of time, but neither did.We also expected that the emphasis in recent literature on educational leadership (e.g., Bjork, 1993;Carter & Cunningham, 1997;Konnert & Augenstein, 1990), on the one hand, and balanced leadership (e.g., Johnson, 1996), on the other, would have kept superintendents from reporting that they spent large proportions of their time on managerial tasks.Nevertheless, our analyses indicated that in all four states, the managerial category of responsibilities was the one to which superintendents generally devoted the most time.This finding, coupled with our findings about the influences of enrollment and locale on the allocation of time to managerial leadership, suggest that structural conditions play a powerful role in determining the character of superintendents' work.In the face of both accountability measures and prescriptive literature pushing them toward educational leadership, superintendents still devote the largest proportion of their time to management.These conclusions do not directly support policy recommendations but rather point to unanswered questions about the relationship between superintendents' work and district performance.They suggest, moreover, that studies attentive to these questions need to be conducted in different types of districts-distinguished primarily on the basis of size and locale.This suggestion is not new, of course.In fact, it repeats, expands on, and lends empirical support to Cuban's (2001) contention that leadership of urban schools ought not to be confused with leadership of schools in other locales.According to Cuban, The blending of urban with suburban and rural schools encourages presidents, governors, and corporate leaders to design solutions for ailing urban schools that become one-size-fits-all reforms treating all American schools as interchangeable cogs in a large machine.(p. 2) Of course, Cuban's concern for urban districts takes the foreground here, but his insight retains its force when we substitute "rural" or "small-district" for "urban." Moreover, despite the face-validity of Cuban's (2001) claim that leadership of urban districts requires a judicious balance of educational, managerial, and political leadership positioned "to cope with the conflicts arising from issues of race and class as they affect test scores and the broader purposes of public schooling" (p.6), little empirical evidence shows a linkage between such leadership balance and higher performance (see also Johnson, 1996).Additional research along these lines is clearly warranted.
So too is research on the outcomes associated with the predominantly managerial leadership exercised by superintendents in many rural and small-sized districts.Holding the leadership rhetoric in abeyance, researchers need to explore the possibility that some uses of managerial leadership in such districts might support high performance or that some uses of educational or political leadership might impede it.Furthermore, as consumers of the research, educators and policy makers should avoid the tendency to generalize findings across contexts.After all, an accumulating body of evidence, to which our study contributes, points to the likelihood that context has an influence not only on practice, but also on the relationship between practice and performance.

Table 1
Data definitions and sources

Table 4
Predictors of perceived stringency

Table 7
ANCOVA model evaluating location in stringent or lenient state in consideration of known covariates (Type II sum of squares)