Synopsis

Life wasn't meant to be easy. We have four frames for assessment. Four differing sets of assumptions about what assessment is about. Equity is similarly compounded. There are (at least) three differing definitions of equity in current use: The first is based on equal means, treating everyone the same; the second is based on equal ends, treating everybody differently to end up the same; and the third is based on elucidating different ends and different means. The advantages, limitations, and pre-conditions for these three notions to be effective in practice are discussed.

Then I take each frame of reference for assessment in turn, and tease out its compatibility with each notion of equity, and with the hierarchical power relations of which the assessment system is an integral part.
 

The meaning of equity

That seemed like a good start. Notions of equal shares and inclusion. But the meaning gets more complicated as the implications for achieving fairness are developed.
 

Equal treatment

What has been designed here is a nice tight closed logical system; people design educational means and ends to produce people rather like themselves and also produce definitions of intelligence or ability or skill or relevant knowledge based on similar means and ends, thus justifying the fairness of the unequal ends in terms of the unequal intelligences of the people attaining them.

The self fulfilling prophecy continues when we make these unequal ends the criteria for selection to favoured occupations (Goslin, 1963 p156). Here the success of the incumbents, and all are deemed successful by definition once selected, proves the value and validity of the whole process. Certainly none of the people so favoured are likely to suggest that almost anyone could do their job given an appropriate training programme, or, even more unthinkable these days, through an informal apprenticeship.

How do teachers react to this soft definition of equity? For those who see their task primarily as transmitting certain knowledge and skill and attitudes to students the definition is appealing. Because they see their professional task as transmission, they are likely to define clarity of communication in terms of logic and intention rather than in terms of accessibility or effect. Thus their professional integrity will be preserved if they treat all students in exactly the same way. It will even be considered an advantage if all students dress the same in some sort of uniform so that personal idiosyncrasy is visually nullified.

At the other end of this spectrum are teachers, often those who teach very young children, who have some sense of the student as a person with a very particular background and learning style, and who have a sense of responsibility to deal with those differences, albeit with certain specified skills or knowledge as having particular importance. Such teachers will see the gross limitations of this equal treatment definition, and will tend to reject it.

Yet even these teachers are likely to be ambivalent about rejecting this definition entirely, because of their position in the total educational structure. After all, there is a curriculum that all students are expected to master, and the larger and more structured the organisational unit in which they are enmeshed, the more likely they are to feel the pressure and surveillance directed towards particular ends. And the bigger the group of students they are confronted with, the more helpless they are likely to feel about the possibility of treating everyone differently.

Then, confronted with the impossibility of treating the children differently, in confusion they abdicate: if it isn't possible to achieve equity of ends through differential treatment, isn't it best to at least achieve equity of means?
 

Equal ends

The reasoning is clear. People have different prior experience, so they necessarily start a new experience with different prior knowledge and skill. So if they are all treated the same, this differential starting point will produce disparate ends. It follows we must treat all of them differently if we are to give them all the same opportunity to reach the same specified end point. Fairness or equal opportunity thus means giving additional resources and time to those who are originally disadvantaged in order to achieve equality of ends.

Surely that's fair? Possibly. But who decides what these ends are that everyone should strive to reach? Usually they are defined by an unrepresentative group, who have a strong vested interest in maintaining and distributing certain sorts of knowledge, values, skills and myths, and/or of limiting the number of people who will have access to the same. Thus the ends are a narrow selection from a much wider range of possibilities. Why should all the resources go into these particular ends?

Part of the answer relates to the current nature of institutions, and the learning that can occur in them. They are not constructed or resourced in a way conducive to individualised learning, but in terms of much larger learning units.

So teaching institutions tend to ignore the unfair treatment of individual students for two reasons: First, because individual students have no power, this representation of unfairness is rarely articulated; and second, because an adequate differentiated response would administratively smell of disorder, such an approach would be contrary to the institution's structural purpose as a hierarchy, which is to impose order.

Some sub-groups however do have power. Institutions have to respond to claims of discrimination against particular sub-groups of gender, class, ethnicity, or whatever minority has found a voice. This has been useful in the short term as an awareness raising activity about the equity issue.

Such political activity on the part of sub-groups that have found themselves disadvantaged by current structures of teaching has resulted in some shift, at least in terms of rhetoric, towards the equal ends definition of equity. There has been some small acceptance of the idea that it is equity of ends rather than of means that should define equity.

However, the "equal ends" comparison has been applied to groups, not to individuals; the debate has been about whether as many girls as boys can join the power elite, and not about the individualised treatment that might allow all who so desire to be successful. As the debate is about the sharing of domination between groups, it largely ignores the domination within such groups. As such it is also about the sharing of violation, and not about its elimination.
 

Equal ends and the myth of the intelligent child

The other action has been, not surprisingly, to attempt an economic fix. Just as economic health, on the current fashionable models, supposedly bears a long term relationship to standard of living and quality of life for all, so more resources for the "disadvantaged" sub-groups will supposedly produce more equitable ends educationally.

Such an approach ignores the relationship between means and ends. For if it is the means, in this case the particular form of educational environment, that has actually produced the different ends, then more of the same means is hardly likely to improve matters. Indeed, intensifying the same means may produce more discrimination. (Of one thing though we may be sure. More resources for the disadvantaged will certainly benefit those advantaged who have identified the problem, and have some solutions, preferably packaged.)

How could this be? How could an educational environment, created by professional teachers, produce negative results, increase disadvantage? Surely anyone with sufficient motivation and intelligence can succeed?

That's one myth that has always stood in the way of any real progress towards sharing and inclusion. Once you accept the idea of "bright" students and "dumb" students, and the notion that there is a direct causal relation between attitude and success, then inequities are merely a mirror of these individual variables. If girls don't do as well as boys it is either because they're not so bright, they're not motivated, or both. And poor kids are dumber than rich kids and that's why they don't do so well. It's obvious. It's genetic as much as anything. Rich kid's fathers are more intelligent otherwise they wouldn't be rich!

Teachers, armed with prejudicial expectations and judgments as well as assessment data, are often quite clear about who is bright, average, and not so bright in their class, a distinction not always so clear to the outside observer. I've talked to small groups of children in hundreds of schools. I'd often ask the Principal to select a small group of about twelve students, some bright, some slow (one of the in-words for stupid at the time). We'd sit in a circle on the floor in the library and talk about home and school and life and the future for an hour or so. At the end of that time I was never able to tell which of the students were supposed to be the "slow" ones. I suspected sometimes they included those who had made the most significant contributions, and the most profound comments.

The "blame the victim" ideology is pervasive in education, and is maintained through the closed logical system described earlier. Assessment procedures play a crucial role here. After all, the teacher is paid to teach. Yet the failure label is invariably attached to the student.

Different people, ends and means

Fairness is treating people differently so they can end up differently. And the different ends will be determined largely by the students themselves. Fairness than consists in providing different resources so that different people can achieve their own different end points, through their own appropriate means.

Is this individual choice and freedom not illusory? Surely expectations embedded in people's social class or gender will determine their choices, and so inequities of power and wealth will still be perpetuated?

This is not a light criticism, and the strength of such sub-cultural or individual expectations is great. However, this strength is diminished as the awareness and verbalisation of the imposed expectations increases. Sub-cultural expectations do not invalidate the logic of the "difference" definition. They do indicate some of the conditions for an implementation in accord with its purposes.

The professional rhetoric of education is concerned with ideas of "individual differences", of the "whole person," and of "clear thinking, rational man." Less so with the passionate, spontaneous, loving, emotional man, or woman. Even so, we might expect some professional support for the different ends and means definition. There is, however, an inherent contradiction between the structure of educational institutions and this idea of equity. So the learning reality rarely approaches the professional rhetoric.

The structure of the school is hierarchical and competitive. The revered qualities are conformity (called cooperation), emotional suppression (called rationality), and acceptance of absurdity (called maturity or respect). None of these qualities is necessary for effective learning. Indeed, all are inimical to learning beyond the trivial. Yet all are necessary for success in learning at a school, because the institutional structure, the political reality that pervades the learning institution, demands these prerequisite responses.

Such an emphasis on control and order is simply incompatible with the idea of young people (of any people) being the main determinants of what they learn and how they learn it. That would be seen by the institution as anarchy. And whilst some teachers would see it as professionally desirable, they would go on to add that "in reality, of course, . . . "

What they mean is that the imperatives of their professional ethic and of their hierarchical morality are different. And in such a situation the hierarchical imperative will hold precedence. Such political expediency is often mis-named "reality". It is more accurately called political obligation, the moral imperative embedded in the institutional power structure. When professional behaviour is not subservient to this obligation, any teacher risks exclusion from the structure. Professional survival is, in the unreal world of the institution, indeed dependent on political expediency.
 

Equity, frame and hierarchy

Then three definitions of fairness have been described; three morally justifiable ways to describe educational equity, each fraught with its own limitations, and containing its own implicit notions about the meaning of justice.

These notions of frames and equity come together and form a discourse within educational institutions which are almost invariably hierarchical in their power structures, and these educational systems themselves are embedded in wider societal structures of that very special form of hierarchy called bureaucracy. This is not the time and place to go into detail about differences between simple hierarchies and bureaucracies. At the risk of oversimplification, I will note here that simple hierarchies usually have an identifiable person, with describable characteristics, at the apex. Bureaucracies, on the other hand, are led by shadowy and replaceable functionaries. Personal idiosyncrasies in such functionaries are abhorred. One of their tasks is to await their inevitable replacement by robots with phlegm and aplomb (Arendt, 1969; Kavan, 1985).

Now I want to examine the compatibilities between these professional assessment options, meanings of fairness, and the social structure called hierarchy.
 

Hierarchy, equity and the Judge's frame

Specifically, if the Judge is necessary in order that the student may be accurately assessed, then the hierarchical structure is necessary in order to achieve this educational requirement. In addition, if a Chief Judge is necessary to check, or at least ratify, the accuracy of Lesser Judges, then the next level of hierarchy, the Head of Department, is necessitated. And so on. Thus the illusion that hierarchy is necessary for educational purposes is maintained.

Because the Judge's purpose and power are both based on his or her claim to recognise the standard, the equal treatment definition of equity dovetails nicely with this frame. Indeed, the assessor's work is so much simpler if all students have been through the same educational programme, so all have had an equal opportunity to know or respond to the answers to the questions asked. Whilst Judges would deny the necessity for a rank order of students, they would all be willing to admit that their task is so much easier once the rank order has been produced. All they have to do then is locate the standard between two particular students, and the classification of all the other students automatically follows.

The equal ends definition of equity presents the Judge with no theoretical difficulties. In practice however there are great difficulties.

Whilst the Judges believe they can recognise standards, the research indicates clearly that they are capable only of assessing comparative performance, and the "standard" is inevitably linked to the sample of responses provided, as well as to some assumptions about the composition of that sample. For example, given a large sample with a complete range of student work, a Judge will assess some (or many) as being below the required standard. Later, given a sample containing only those assessed previously at above standard, the Judge will now assess some of these at below standard, especially if he or she assumes the sample is covering the full range (Hartog and Rhodes 1936).

So even if the equal ends definition were achieved with a given group, and through differential treatment they had all reached an adequate standard, according to some data, it is almost certain that the Judge will still assess some at below the required standard.

However, as explained earlier, equal ends doesn't really apply to individuals, but to sub-groups. It's the relative percentage of success between sub-groups that assumes importance for the equity watch dogs. In this regard Judges, being rational and aware beings, are often able to adequately attune their prejudices to the political requirements of their time.

If the equal ends definition of difficulty sets a difficult task for the Judge, then rationally the different ends and means definition presents an incomprehensible one. For how could one hundred completely different products, the outcomes of one hundred different curricula, be compared to a single standard? Surely only Judges of very high status, or extreme arrogance, would attempt such a task.

Faint heart made not fair Judge! To the Judge it's no harder than any other assessment task. The Judge is undeterred by the variety of products and purposes. The Judge's standard is inviolate. The Judge simply compares each work to this standard and the decision is clear.

However, to do this they must of necessity apply their own criteria for success, rather than that of the student. In so doing they would countermand the requirements of an educational program directed towards different ends and means equity, in which the purposes, and hence the appropriate criteria, and thus necessarily the acceptable "standards", vary from student to student. Luckily, such rational considerations rarely impose on the Judge's religious rituals.
 

Hierarchy, equity and the General frame

Let's look at this a little more closely. The General frame of reference assumes that any single examiner is prone not only to idiosyncratic error due to differences in criteria and "standards" with other assessors, but also to considerable reliability error in his or her own remarking. That is, they will give different marks or grades if they mark the same papers on different occasions, or if they mark different versions of the same paper at the same time. And not only that, but such errors are increased, not decreased, if prior knowledge of the student is available (generalizability errors, that is). And not only that, but that chief examiners are no better than any others in regard to such heinous errors.

All this would be bad enough, interfering as it does with the "right" of the teacher or lecturer to have ownership of their students, and to alone decide their future. But if the assessment input of any competent person is as good as anyone else's, then the whole hierarchical structure of the organisation is called into question.

Worse is to come. Some studies have found that groups of students assessing their own work are also able to get closer to the "true" score than are individual learned superiors. This is democracy run wild; this is destabilisation of hallowed structures; this is anarchy.

Of course, educational institutions can survive without their Judges, although the professional justifications evoked by their presence does wonders for institutional status. If Judges lose the Wars of the Gradings to professional test agencies, then so be it. There are still plenty of hierarchical tasks to be done in selecting syllabuses, administering tests, limiting admission, marking rolls, ejecting students, and so on.

Even so, removing the myth of the Judge from the ideology of the educational institution is pulling out its teeth, leaving it gumless in academia. The function of the school and university has always been equivocal. Rhetorically defined by its purpose of searching for truth and instilling freedom of thought, its practical purpose has been much more mundane - to conserve the culture by perpetuating its myths and reproducing its social and technical elements.

The risk with academics is that they sometimes take their rhetoric seriously, and actively try to bridge the gap between ideology and practise. Given the somewhat radical stance developed in some schools and universities in the sixties and early seventies, it is not altogether surprising that they should be milked of some of their power during the eighties and early nineties of this century. The economic cringe is obvious. But what more Machiavellian way of producing an academic cringe than by using their own research as justification for removing their Judges' power.

In regard to equity, the equal treatment definition implies some measure of competitive merit, and such a measure would certainly be "fairer", that is more stable and less dependent on the vagaries of particular assessors, if the General frame of assessment were used.

This frame would also be useful in relation to the equal ends definition if professionally normed and standardised tests were used as an end point for a satisfactory standard. However, it would be a mistake to believe that the test measured any pre-existing standard. Rather the standard is defined by a certain score on the test. The validity of any such measure is moot. And indeed, this very mootness has left a gap in which the Judge has been resuscitated. For who else is capable to legitimise an arbitrary cut-off? (See any Public examination manual).

The rank ordering procedures of the General frame are not appropriate to the different ends and means idea of equity, because the educational ends and means are individually negotiable, so there is no single "ability" or "trait" or "domain" on the basis of which the students can be ranked.
 

Hierarchy, equity and the Specific frame

Advocates of this frame are likely to down-play, and underestimate, the differences between the equal treatments and equal ends definitions of equity. It's simply a matter of time, they say. Our objectives are clear, our programs are tested, and everyone can reach the desired standard if they try. Some are a little slower than others, that's all, so they will require a little more time. But, given sufficient time, everyone will succeed (Bloom, 1976).

This is facile. Different treatment involves much more than time. Learning styles and appropriate student-teacher relationships cannot be condensed into this single variable. None the less, this could represent some movement towards student empowerment, in as much as very clear and achievable indicators are given to the student about what they must do in order to complete the course adequately.

There is no theoretical reason why some specific behavioural objectives, and some more general criterion referenced objectives, should not be part of the negotiated contracts associated with the different ends and means definition of equity. However, these would generally be negotiated between student and teacher as part of the learning process, rather that imposed on students and teachers as predefined parts of the course.

In terms of its current usage in education, such negotiation would violate current practice and trends, which uses the criterion referenced outcomes, professionally developed and applied, as the true measure of achievement standard. Ironically, to the extent that the outcomes are inadequately defined, and thus confused, the gateway to incorporate such outcomes into the broad definition of equity becomes enlarged. That is, the outcomes may become differentially specific by negotiated discourse with particular students.

Because it denies hierarchy, however, this rarely happens. It is discouraging to see an assessment frame which seemed to hold promise for the empowerment of students now being used as an instrument of rigidity and conformity, as another meter to objectify disadvantage and enshrine privilege.
 

Hierarchy, equity and the Responsive frame

The Responsive frame is also contradictory both to the equal treatment and the equal ends definitions of equity. Responding to individuals in different ways is obviously not compatible with the equal treatment definition, and spontaneous generation of criteria, negotiated curricula and assessment descriptions, and obviously subjective responses, have little connection to common goals and end points.

This is not to say that some well-defined objectives might not be found acceptable and useful to particular students in describing what they wish to learn, and how they will know when they've learnt it. Nor that some other objectives may be so essential to a course that they are prescribed and proscribed in the beginning.

On the other hand the Responsive frame of assessment is quite compatible with the different ends and means definition of equity. This frame is, in fact, a necessary part of any educational processes that value diversity and freedom of students, and thus include this broad equity concept of fairness and justice.
 

Summary

In the next chapter we look at the criteria of measuring instruments, and how these fit with the four frames for assessment.