Chapter 12: An Inquiry into QualitySynopsisFrom the last two chapters it becomes evident that a fundamental purpose of relating assessment descriptions to standards is to transform notions of quality to notions of quantity. So in this chapter the notion of quality is discussed, and some of the differences with the notion of standard are elucidated.The theory of logical types is briefly explained in terms of its implications for complex constructs with multidimentional aspects and the special properties of the class "safety standards" is discussed. The construction of a bridge with various criteria for quality is discussed to illustrate the different languages that must be used to justify the quality characteristics for each criteria. The subsequent history of the bridge is then used to illustrate how the notion of quality is related to boundary conditions and events, and how this affects notions of permanency and attribution. Some reflections on the nature of quality follow. These are then applied to some of Eisner's ideas about connoisseurship. Persig's ideas about the metaphysics of quality are briefly discussed,
and the relationship between morality and quality on the one hand, and
static and dynamic morality, introduced.
All standards are arbitraryWhen I was younger and groping for a profession that might suit me, I studied Physics and Engineering. I don't remember much of the detail of those studies, but I did learn two things that are pertinent to this chapter: One is that all measurements contain an error; the other is that all standards are arbitrary.I remember very clearly struggling with some calculations to determine
the cross-sectional area of a steel beam for a bridge. Estimations of maximum
loading on the bridge, moments of force and tensile stress resulted in
a value of the cross sectional area of the beam accurate to three figures.
However, before choosing the appropriate steel T section there was one
more step. A safety factor of three must be applied. Or was it four? No
matter, the calculated cross-sectional area must be multiplied by this
arbitrary number in consideration of possible tornadoes, earthquakes, rock
concerts on the bridge, or whatever other natural disasters might inadvertently
occur. This undoubtedly would make the bridge safer for traffic and incidentally
more profitable for the steel manufacturers. And it made the accuracy of
the initial calculation absurd.
Safety and qualityAt this point I want to try and untangle another confusion that has bedevilled the notion of standard, especially as applied in the human sciences. This is the confusion between safety standards and quality standards.In the manufacturing area there is less confusion. Standards that apply to car seat belts, bumper bars, brakes, lights, are clearly basic safety requirements. General design of car, colour, control panel layout, type of upholstery, fuel economy, are aspects of quality. And of course, one aspect of quality is that all safety standards are met. Safety is about prevention. Safety is about what is not, about events that are always immanent, yet, if safety is successful, never materialise. Safety is about the future that is frustrated, about unrealised potential. Because each safety measure blocks a road to disaster, each safety measure is essential in its own right. To meet a safety standard is to claim that one such roadblock is in place. To know that all such safety standards are met is to be reassured and insured against disaster. However, to know that eighty percent of safety standards are met is to know nothing about which particular safety standards are not met. For a gambling man this may be a situation of high desirability, and hence provide an experience of high quality. But in the world of safety standards, this is a recipe for disaster. Quality on the other hand is about manifestation, about potential realised.
Quality is not so much about specific aspects as about their interrelations;
about interpretation rather than measurement; about the whole gestalt rather
than summaries. Further, notions of quality are intimately and necessarily
connected with the observer, and hence are constructed from the observer-object
interaction, rather than claiming to be a measurable component, or sometimes
a presence or absence, of the object or specific attribute being observed.
Theory of logical typesThe theory of logical types is about levels of abstraction in human discourse. One of its axioms is that whatever involves all of a collection must not be one of the collection; that is, that there is a fundamental distinction between a class, and the members of that class. This might seem obvious. Obviously a single man is not all men, and a married woman is not all women.Trivial as this might seem, the conclusion from the theory is far from trivial: that when this clear separation between class and members is not made, messages become confused. As Bateson (1972) describes it, "the theory asserts that if these simple rules of formal discourse are contravened, paradox will be generated and the discourse vitiated" (p280). Human discourse is decidedly more complex that simple logical syllogisms. We do not usually talk like logic machines. We talk very often in and about abstractions, and these abstractions may be at different levels of logical type. We present information (first level), and give an interpretation of that information (second level), in a particular context which affects its meaning (third level). A story that makes fun of a rich Jew has a very different meaning if told by a speaker at an anti-semitic rally than it does when told by a Jewish comedian on a New York stage. Of particular interest here is that errors that lead to confusion occur when the properties of a class are ascribed to members of that class, or vice versa; or more subtly, whenever the discontinuity between class and member is neglected, and they are treated as if they were at the same level of abstraction: Safety and logical typeSafety is not quality. It is one criteria we might use in describing quality. It is a member of the class of such criteria. But it is a very particular member, because it is atomic in its construction. It is comprised of a number of specific safety requirements each of which must be individually met. Not only is the class of events or information called "safety" of a different logical type to the class called "quality," but the essential information about safety is lost when the class "safety" is described, rather than the individual items that describe it. Unless, as we mentioned earlier, the statement about the class is that "all safety measures have been satisfied."Safety and peopleIn many aspects of our life safety measures are important for its continuance. In home, leisure activities and job, safety requirements contribute to our health and that of others. So matters of safety are a part of various educational programs. As such, it would seem important that evidence be obtained that students have incorporated such safety items into their behaviour. Or, at the very least, that they understand and can implement all of the safety requirements. Talk of safety (like talk of sexuality) produces points of high density in the field of power relations.It should be apparent, however, that test or examination information involving rank orders or grades or marks regarding safety represents information about the class of safety items, and as such is inappropriate and confusing. If safety requirements are essential requirements, then marks of 70 per cent or grades of C for safety, or for tests which include questions about safety, present information that is inherently contradictory. By definition, if you have not met all safety requirements you are unsafe. Test-makers and others argue that in the context of a test people make errors and it is not reasonable, because it rarely happens, to expect one hundred percent correct response. This is surely an indication that the test context is inappropriate for obtaining information about a person's acquisition of safety measures. It certainly does not justify accepting that if they can provide evidence that they "know" seventy percent of the safety requirements that their "standard" of safety is adequate. Further, talking about safety measures, or choosing the correct safety
requirement from a number of choices, is an activity of different logical
type than implementing that information in the context of a job. Talking
about something you do is of a different logical type than doing it. So
any measure on a test, even at one hundred percent, cannot be a measure
of safety behaviour. It is a measure of test behaviour. At the very best
it is an indicator, about which empirical evidence could be obtained about
the probability of its correspondence with overt safety behaviour under
specified conditions. In this respect, probabilities less than one would
necessarily indicate test invalidity.
Safety and minimal outcomesThe idea of minimal outcomes is analogous to that of safety. Minimal, or minimum, means the least amount, the lowest possible. If a course of study has a set of minimal outcomes that define its successful completion, then by definition all such outcomes must be demonstrated if the course is to be satisfactorily completed. To set a test incorporating questions related to such outcomes and then use a test score (a statement about the class) to describe the "standard" that has already been described by each of the members of the class, is again to confuse logical types. Such tests are sometimes referred to as mastery tests.There are three additional confusions, two of them the same as for "safety." The first is that only a perfect score is consistent with the definition of minimal. So to attempt to find an appropriate "cut-off" score to use as a standard is to engage in a paradox, is to indulge a contradiction, is to professionalise an absurdity. Berk (1986) was able to identify 38 methods for setting standards and produced a consumer's guide (to choose the most appropriate absurdity). The second confusion involves the fact that context affects meaning. For many educational outcomes the context of a test situation is inappropriate anyway and represents another logical type confusion. For example, any outcomes involving verbal discourse, such as listening skills, group problem solving, giving instructions, cannot be demonstrated in a written or multiple-choice test without logical type confusion occurring. Writing about verbal interaction is not verbal interaction. Choosing the most appropriate response from a multiple-choice selection is not responding oneself in an interpersonal context. Talking about a painting is not painting. The whole test and examination industry is permeated with this sort of confusion. The third confusion is one of ends and means, and is well described
by Burton (1978): "no measure of a single skill can ever be mapped on a
non-trivial vision of real success because any problem can be solved in
more than one way. One can determine whether the respondent has the skills
necessary to solve the problem this way, but one lacks the justification
for imposing successful performance, this way, as a standard"(p273). Burton
believes that "this argument is fatal to any method of setting performance
standards." Burton is perhaps mistaken in believing the issue is amenable
to rational argument, and does not consider that it may be entrenched in
mythical discourse.
Mastery tests and framesMastery tests result in scores produced by the summation into a numerical score of specific objectives attained. In relation to error, they contain all of the errors of specific objectives plus a large labelling error. In adding the results most of the important information is lost, in that we no longer know which specific objectives have been attained and which have not.In this situation, whilst the generation of the test has used the Specific frame of reference, the summation has resulted in a normative test score. We no longer have information about what a student has achieved. We have information only about how many of the objectives have been achieved. This is exactly equivalent to information about how many addition sums are correct, or how many words are correctly spelt, or how many formulas in dynamics we can remember. The description is now clearly normative, and may only be interpreted in terms of whether one student got more or less "right" than another, or in terms of some arbitrary "standard" of how many "correct" answers will be considered "adequate"; how many correct answers constitutes a "pass." In this situation, because information about the particularity of objectives
attained is lost, the whole detailed descriptions tend to be similarly
"lost," or unavailable to those interpreting the test information. Labelling
errors thus become large, as the meaning of the score, and the label attached
to it, are differentially interpreted.
Mastery tests and internal logicIn most courses there are some facts, some understandings, some activities or skills, which are central to what the course is about, so that we could say - if they don't know at least those things, or if they can't do at least these things, then there is no way we could say they have adequately completed the course. In old-fashioned terms, they are the "must knows" or "must dos" of the course. As distinct from the "should know" or "could know" categories.Now there may be some areas of study where curriculum writers or teachers are unable, or unwilling, to specify such a category of "must know" performance. However, when it is so specified, it comprises a description of a finite number of procedures or products that will demonstrate the "knowing" of these crucial things. In other words, within this limited "must know" area, it is possible to specify what must be done, the conditions under which it must be done, and the procedure by which its adequacy will be known. These then could be used to describe the essential requirements of the course of study. They are limited in number and extent, and are specifiable in the specific frame of reference. As they are accomplished, as evidence is obtained that each outcome has been achieved, this can be certified by the teacher or student. If there are ten such outcomes, then successful completion of the course would require that all ten outcomes be so certified. Otherwise they cannot, obviously, be essential. To certify that eight out of the ten essential requirements have been completed is to certify that two of the essential requirements of the course have not been completed, and thus to certify that the student is uncertifiable. More than this, it is to lose the information about which two essential requirements have not been demonstrated. So to obtain a "total score" on a mastery "test" is to contradict the whole concept of essential requirements, and to lose all the relevant information. Unless the total score is a "perfect" score. In many situations the very notion of a "test," of some particular situation constructed to check all of the essential requirements at one time, would itself be contrary to this frame of reference. In the artificial and often pressured "test" situation it might be expected that success in some "essential" activities might not be demonstrated. It is this very argument which has been used to justify the acceptance of less than a "perfect" score in a mastery test. Rather it should be seen for what it is - an argument that invalidates the use of the test. The problem of time-binding is not solved by success in test situations any more that it is by success in the ongoing teaching - learning context. We can never certify that any fact will be recalled at a later date, that any understanding will be retained in the future, that any skill will be demonstrated again successfully next year. We can sensibly certify that a behaviour has occurred once, or twice, or if necessary one hundred times. Regardless, we can never be certain it will be adequately demonstrated on the next occasion. Test givers imply, with their insistence on testing, that demonstrations outside the testing situation are in some way of limited value, credibility and validity. It has always seemed to me that "tests" have all the inadequacies of "on site" or ongoing certification, with quite a few bonus inadequacies added on for good measure. Or more accurately, for worse measure.
A bridge of qualityLet's assume that we want to describe a particular person's performance in a certain area. Building bridges is as good an area as any. And we are interested in the quality of that performance. That is, we are in the area of discourse often called assessment.We might decide that there are four aspects of performance which we want information about; four members of the class we will call quality; four criteria on the basis of which we will assess quality of the bridge produced. Is the bridge safe? Is it economical in cost of materials, construction, and maintenance? What is its environmental impact in its rural context? And how is its aesthetic design judged in a competitive order of merit in relation to other submitted designs? We note in passing that this decision about these particular four aspects of quality is itself a value judgment subject to enormous error in the General frame of reference. It is clear that the language of discourse for each of these four criteria will be different, and attempts to simplify by means of some language that is appropriate to some and not others, or that is appropriate to the notion of "quality" as a class but not to some or all of the members of that class, is to compound confusion by oversimplification (Eisner, 1991, p182). For example, the first question, about safety, may only be addressed by showing that all safety measures are in place; the language that designates individual safety standards is appropriate. The question about being economical involves careful costing; the language of accounting is appropriate, and the language of economics will be necessary to delineate boundaries. The question about environmental impact will draw information from a number of disciplines - geology, biology, ecology, geography, ethics, economics, and so on. Ultimately, the discourse must deal with the balances and trade-offs among conflicting values and pressures; the language of politics and the language of environmental ethics will fight it out. Finally, the order of merit based on the aesthetics of the design will draw on the language of art and architecture, and be involved with issues of the assessors' personal tastes and the profession's current fashions. Finally, however, such complexities will be reduced to a single dimension where better-worse becomes more-less and a rank order is produced. As this competitive order of merit is one aspect of the quality of the
design, it is not that quality. By the same token, no measure of the order
of merit can be the measure of quality, any more than a cut-off point on
the order of merit can represent a cut-off point of quality. All this regardless
of how consistent, stable, generalisable that order of merit may, or may
not, be.
Permanence of qualityThe one where I chose four rather arbitrary aspects of quality to talk about? Yeah, that one. You made it easy for yourself by choosing something very practical and material and solid. I mean, it's stable, you can see it and jump on it. It'll still be there tomorrow so that others can assess its quality for themselves. It does have that illusory aspect of permanence. Why illusory? A bridge is a pretty permanent structure. Even so, the notion of quality is somewhat ephemeral. Let's see how our bridge, built five years ago, has stood up to our quality assessment. First the aesthetic quality, the only one subjected to the rigours of competition, of rank ordering and the notion of the standard. The design was brilliant and quite spectacular. There was some controversy after it was built about its enormity. But mostly there was approval. Then, of course, fashions change. Most "experts" these days consider simplicity a major design virtue. You're saying that if the competition were rerun today this design wouldn't have won? That's what I'm saying. These days big high ornate bridges are out. Simple low bridges are in. What about environmental impact? There's the bridge's visual domination of the landscape, which is much more intrusive than was anticipated. The terrain is very flat. So you can see it twenty kilometres away. But more important for some is the impact it's had on the lesser crested poorigal. The bridge has affected its navigational ability in some mysterious magnetic way. Apparently this area was significant to a change in direction during their yearly migration. Now they fly in circles around the bridge till they drop. Suddenly they've become an endangered species. What about the economic question? Interest rates have gone up by a factor of three, they've put a toll on the bridge, and the government has had to bail out the Roads Board once already. What was once an economic asset has become an money-eating monster. Well, I guess fashion, the environment, and the economy are always a bit suspect in terms of their stability. But at least the bridge is still there, and it's safe. Not exactly. What do you mean, not exactly? Just one of those unfortunate things really. It's not considered a major earthquake area. Almost no activity over the last sixty years. Then last week there was this major tremor. Point eight on the Richter scale. A major fault line developed just a kilometre away from the bridge. Did it damage the bridge? Not exactly. Amazing structure really. Shows how good the design was. Not a crack anywhere. Only one problem. What's that? When the land tilted, the whole bridge tilted with it. The road slopes thirty degrees. So what happens now? Well, the bridge is useless. The only question now is whether to leave it there, or spend half a million to blow it up and remove it, thus saving from extinction the lesser crested poorigal. What is important to understand about this very sad story is that it indicates very clearly that the bridge itself does not have any qualities. Putting it another way, none of the qualities we discussed in relation to the bridge belong to the bridge. They are rather descriptions of how the bridge will interact with other things - with the physical and geological environment, with the economic system utilised to finance it, with the human cultural world in which it is enmeshed. So when any of these environments change from those expected, so does the quality of the bridge. Nor does the bridge have some aesthetic qualities having a magical existence independent of the bridge and its environment. You may conceive the bridge as being beautiful, as some music that you hear is beautiful, or the second law of thermodynamics seems beautiful. And indeed there may be a palpable human response that you have to these three events which justify using a single word, beauty, to describe them. Even so, it is clear that the similarity is contained in your particular response to the events, rather than to the objects that are responded to. All of which does not mean that beauty
is in the eye of the beholder. To take that view is to denigrate the object.
Just as to ascribe the beauty to the object observed is to denigrate the
observer. If the label of beauty is to be pinned anywhere, then it must
be pinned to the event, the interaction, the relation, between observer
and observed. Qualities, like any other form of data, are constructed from
events, not discovered in objects.
Quality, standard and logical type errorLet's look then at what might represent quality in a teacher or student in a school.The function of the school is not only to prohibit and punish and exclude but to produce. To produce good work. Though even here, good work is but a symptom of the more important school product, the good student. The good individual student. Increasingly, it is not so much the work of the student that is valued, but the "whole person" that presages it. Abilities, attitudes, skills, the whole plethora of attributes fantasised to define the good student, the good worker, the good manager, become the focus of attention, the point of application of the standard. This is not new, though it is more overt that it was twenty years ago. I remember doing some consultancy work in a Primary Teachers College in the 1960s. I visited the various faculties, and talked to the lecturers. Indeed, they were concerned that the students had sufficient knowledge to teach the subject. But what was more important was that they had a very positive attitude to the subject, that they really liked teaching mathematics, or music, or history, or science, or physical education, or whatever. On the surface, a useful intent. Yet when I tried to picture what sort of a person this would be, with great enthusiasms for everything that they taught, I could see a successful sales-person, but hardly a successful teacher. It was laudable that these lecturers communicate their enthusiasm to their students. It was their inability to see its overall implications, and its curtailment of any critical thinking on the part of the students (or indeed often on their own part) that was cause for concern. My problem was to discern the difference between a student enthusiastic about the whole curriculum, and a happily conforming blob. The error is a logical type error. In the class "quality" there are many members; there are many aspects of a person that relate to quality performance. One of these may relate to the particular context. Another may relate to standards of proficiency. Another to integrity of values. The language of discourse of these three areas will be different. But all of these discourses must be both utilised and transcended in a discourse on quality, and no measures of the members of the class (assuming such measures are possible), can be a measure of quality. Another example; quality of life
is not the same as standard of living; there is a world of difference,
indeed a life-style of difference, in the two concepts. For the very essence
of quality is its immeasurability, its identification with a world not
wholly material, an association with that mysterious realm of experience
called "soul." Quality is concerned both with essence, with experience
from within, as well as with experience perceived through reflection from
surfaces. Standard of living, on the other hand is a function of measurable
quantities; income, savings, washing machines, televisions, supermarket
shopping bills, and whatever; the countables, the quantifiables, of the
material and materialistic world. Again, "standard" is a member of the
class "quality." And for that very reason the two concepts cannot logically,
and hence rationally, be identified.
Adequacy and labellingHow do we solve the dilemma? If standards cannot do the job expected of them, what do we replace them with? The issue of competence in a job does not go away because of the errors and confusions in its measurement. On the other hand, it is possible within a particular milieu for a group of people to agree with some consistency, and hence certify, that certain work has been carried out adequately. In every family, in every school, in every sporting team, in every job, work is done and considered adequate. It is useful for some purpose and not dangerous. And the conditions of that work, (and hence of that agreement), may be democratic or elitist, may press towards convergence or divergence. In other words, there is a notion of adequacy, or competence, or comparative excellence - in short, of a limited sort of quality, that is both embedded within and produced by any work culture, in terms of which individual performance is assessed. What is also clear is that this notion is fuzzy and multi-dimensional, error prone, describable rather than measurable.What becomes clear here is that this
notion of adequacy, of quality of the work, is not independent of the culture
in which it occurs. The label of adequacy is a label belonging to the whole
interactional milieu in which the work occurs; yet another reason for the
immense errors that become apparent when such work performances, or the
abilities or skills or predispositions or aptitudes that are fantasised
to explain them, are pinned onto particular workers, and to a lesser extent
on particular criteria or products (Fielding,1988; Raven,1992).
QualityQuality refers to a particular experience. The notion of quality is a complex one, involving a number of aspects of the experiential event that can be discriminated. The possible aspects that could be discriminated always exceeds the actual aspects discriminated; an informed choice is made about what particular aspects will be discriminated in this particular case. The choice itself is arbitrary, in that different choices could have been made, some of which would in retrospect be approved. Such choice of course mirrors value. Discourse about any one aspect might or might not refer to some standard of accuracy or adequacy or competency or whatever. Balance or harmony or elegance is an aspect of quality. This involves the relationship between the aspects initially discriminated. All this so far is a description of surfaces, of what the object or performance appears to be from the outside. How does this relational aspect look from the inside? If quality is more the spirit of the product (the person, the event), then quality relates to the interior of the holon. Quality is, in human terms, the expression of the life force immanent in the product, or in the production, or in the person in the process of production; that is, in the production event. Quality then becomes related to a state of consciousness, or its analogue in non-conscious productions. It involves the integrity, the meaning, both of the producer and the product. Quality also involves the integration of the inside and outside; the aligning of truthfulness with truth; of inside and outside awareness; of the aligning of the potential of the stone with the vision and skill of the sculptor; of the sound of the spirit with the song of the singer (Wilbur, 1996). From the inside quality is experienced as the essence of the event, of the spirit of the relational experience. It is thus the meaning of the event as interpreted by its participants. It may be, indeed will be, different to other similar eventful experiences, and because of its idiosyncrasies is not comparable to them in any linear way. So it is not possible to link this notion of quality to ideas of adequacy or competence or of other categorisations which necessarily involve standards. What words then are suitable? Beauty perhaps? Elegance? Flow? Life? Spirited? Words that describe the essence of the experience, of the connection! In relation to people's performances, the notion of quality can be attached either to the creative process of the performance, or to a particular product of the performance. Post-structural analysts want only to attend to the latter, regarding the former as irrelevant. And of course the event that involves a critic interacting with the product is a different event to that event which produced the product. As such the qualities of the two events are necessarily different and essentially non-comparable. The element they have in common is the final product; but this product was the culmination of the first event; it did not exist till the final moment of the first event. On the other hand, it is sometimes
a stable and reproducible element of the second event. The two events are
holarchicaly connected. The first event (culminating in the product) can
exist without the second (the critique). But the second event cannot happen
without the first. It follows, as with all such holarchical connections,
that the attributes that determine quality in the first event are not necessarily
or probably those which determine quality in the second. They are different
creative endeavours; they have different intentions and languages; to misrepresent
this difference is to court confusion.
Eisner, quality, judgment and standardEisner is one of the few writers in the assessment field who has attempted to analyse in depth the notion of quality through his notion of connoisseurship. Eisner (1991) differentiates qualities from qualitative from quality. "By qualities I mean those features of our environment that can be experienced through any of our senses"(p17). So a quality pertaining to a person is any aspect of that person on the basis of which we can differentiate by using our senses. "Aspect" or "attribute" or "property" may be better words to use because they avoid the confusion with the notion of quality we have been discussing. He goes on to claim that "we can only appraise and interpret what we have been able to experience," but then warns that "if our perceptual experience is aborted for the sake of classification, our experience is attenuated"(p17). Eisner adds that "the qualitative aspects of experience are not only secured in attending to qualities out there, but also are manifest in the things we do and make"(p18). In my terminology, aspects are discriminated both in the event that produces a product, and in the event in which it is perceived."The ability to make fine-grained discriminations among complex and subtle qualities" is what Eisner (1991, p63) calls connoisseurship, the art of appreciation. The art of recognising quality, as I am using the term. He recognises a fundamental problem with his notion of connoisseurship: And indeed, that is Eisner's test for the adequacy of the critic's work: "consensual validation in criticism is typically a consensus won from readers who are persuaded by what the critic had to say, not by consensus among several critics"(p113). What is such local consensus except a qualitative calculation of the mean? And note how the second order consensus has distracted attention from the first order contradiction, to which he does not return. Why are collections holding contradictory judgments so difficult for Eisner? In his criticism of specific behavioural objectives, Eisner (1985) says that those who evaluate them "often fail to distinguish between the application of a standard and the making of a judgment" (p115). He then quotes Dewey, who, he says, "makes the distinction quite clear." So what is the distinction according to Dewey? Standards, according to Dewey, define things with respect to quantity. And measuring a quantity is not itself a mode of judgment. And qualities are qualities of individual objects, even though the critic reveals himself in the criticism. So to Dewey, and Eisner, the qualities are indeed inherent in the individual object, even though the description of those qualities is enlightened by the connoisseur. And nowhere, concludes Dewey, "are comparisons so odious as in fine art" (Eisner, 1985, p115). So Eisner is clear that qualities cannot by measured by standards. And of course they can't, because standards are definitions and not measurements. What he must mean is that qualities cannot be measured by comparing with standards, both because measurements and judgments are of a different order, and because comparisons are odious. So he is trapped; qualities are inherent in the object; connoisseurs make the fine discriminations that enable them to describe quality; such judgments are not measurements and abhor standards; even so the judgments might lead to categorisations of the object (of winner of the contest, or worth a distinction, or inadequate at this level), which bypass standards and measurement. Yet connoisseurs differ sometimes fundamentally in their categorisations. I have argued in the previous chapter that such categorisations necessarily invoke standards, and comparisons with them. But even if they don't, two contrary judgments of connoisseurs create a contradiction that denies that connoisseurs can categorise accurately, and this is surely one of the essential aspects of their connoisseurship. An alternative explanation, of course, is that the qualities do not reside in the object, but are rather an aspect of the event that involves the interaction of the object with the critic. In which case to categorise the object is to mislabel the event, and hence by implication to mislabel the person who produced the object. All of which takes us back to Eisner's
original question: What do we do with such differences? Eisner says don't
do what is usually done. And then is silent. Maybe if you ignore them they'll
go away! I note that he is talking about consensual validation in this
section of the book, and validation, as we have seen, is an advocacy argument
for the defence. It follows that the disagreement has to be ignored, because
it represents the essence of the (unspeakable) case for the prosecution
(See Chapter 16 on Validity).
Summaries or collections - the crucial choiceSo Eisner doesn't want to celebrate difference as being at the cutting edge of new knowledge, the collection being the best description, superior not only to a summary, but also to any consensual agreement. For to do this is to deny the possibility of the accurate categorisation of people or their creative products. And that is the cutting edge of the power of the connoisseur. Such power does not ultimately lie in the cogency and plausibility and depth and sensitivity of his critique, however much the connoisseur may wish to believe it is so, and even though this advocacy may well support such power; in practice it lies in judgments that define the standards that produce the categorisations that determine the lives of Jack and Jill and all their little children.This necessity to categorise in a single dimension is illustrated by Rosenberg (1967). In his book On quality in art, he looks at criteria of excellence from the 16th to the 20th century. He quotes de Piles, a 17th century critic, who: Rosenberg (1967) then uses this magical and meaningless sum total to criticise some of de Piles' ratings; "We are disappointed that he rates Michelangelo (37) much lower than Andrea del Sarto (45) . . . We cannot understand why Durer receives a grade of only 36, when a second rate Mannerist like Taddeo Zuccaro gets a total of 46"(p37). And so on. But of course de Piles gave no such grades. He knew it was meaningless to add a mark for colour to a mark for composition to a mark for drawing. In assessment, whether qualitative or quantitative, the crucial choice made is whether to opt for summaries or summations on the one hand, or for collections on the other: to opt for summaries is to go the way of simplicity, of communality, of "truth." A summary celebrates similarities by defocussing differences; to opt for collections is to stay with complexity, with uniqueness, with essential uncertainty. A collection celebrates differences by defocussing similarities. Summaries and summations then are basically conservative; they are uni-dimensional; they are dedicated to notions of order and security. Collections are basically radical; they are multi-dimensional; they are dedicated to notions of creativity and anarchy (in its positive persona). To date, the history of educational
assessment has been a developmental history of the summary. The current
agony of many of its most thoughtful protagonists (Delandshere, 1994) will
only cease when they settle for collections, and deal openly and ethically
with the personal and social consequences of that choice.
Assessment of quality as moral actionPersig (1991) makes a strong link between morality and quality; in fact, to him they are synonymous terms.He looks at the relationship between evolutionary structure and the metaphysics of quality, and shows that there is not just one moral system, there are many: In the metaphysics of quality there's the morality called the "laws of nature," by which inorganic patterns triumph over chaos; there is a morality called the "law of the jungle" where biology triumphs over the inorganic forces of starvation and death; there's a morality where social patterns triumph over biology, "the law"; and there is intellectual morality, which is still struggling in its attempts to control society. Each of these sets of moral codes is no more related to the other than this dissertation is to the flip-flop circuitry which controls the computer on which it is typed. Let's consider this in relation to our bridge; its quality as a physical structure in the inorganic world was unrelated to its quality as part of the social life of people; just as that in turn was unrelated to its quality in that intellectual world that can conceptualise its probable long term effects on the environment, and hence on the lives of humans not yet living. Further, there will often be conflicts between the static social morality that would hold the physical or biological or social structure stable, and the dynamic evolutionary morality that would move it onward: In both cases, the actor must act on a sense of "rightness," of "necessity" that overrides choice. The actor, like the observer, simply cannot tell what the ultimate quality of the action will be, because the actor can never predict all the consequences of action. To claim that the ultimate test is whether the act is free of ego is to beg the question. Any act can be interpreted as ego-dominated, even acts of transcending the ego, which are designed to nourish the "super - ego." Finally, we are left alone with our
own sense of identity, our own sense of integrity. After all the agonising,
all the reflection, we are finally left with a sense of the flow of life,
with the flow of one particular life, of one particular relationship; with
a sense of appropriateness that on the basis of static moralities is sometimes
most inappropriate. And we do what we must do. This is the essence of evolutionary
morality; it is the essence of what constitutes quality in the intellectual
sphere; it is the essence of the meaning of quality in any assessment event
in which a product or a person is the focussed element. It is a demonstration
of what Churchman (1971) and Campbell (1956) call the heroic mood.
Quality productsTraditionally the problem of the relationship between quality and standard has been solved either by ignoring it, or by emersing it in semantic confusion: by fuzzing the boundaries, by assuming the two concepts are isometric, by ignoring the logical type error, by claiming that high standards are of course synonymous with high quality. And as it is self evident (within mythical discourse) that we can measure standards, it follows that we have measured quality.What we have done is something much more damaging; by identifying standard with quality we have confined quality to the straight and narrow, and thus denied its very essence, which is to be found in its spontaneous deviation from the constraints of geometric efficiency. For the standard is a preconceived point (however practically unmeasurable) on a predetermined scale. It may indeed be used to describe a work of conforming excellence, but is quite incapable of recognising the nuances of diversity, the force of spirit that transforms articulate parrots into creative people. One of the characteristics of works of high quality resides in their difference, not of measure, but of style. Quality is perceived not in differences in kind, but its differences in difference; not in differences in length, but in variations of depth: in short, quality diverts us from the linear, takes us to a dimension orthogonal to the flat. "Quality, consciousness, and experience are separate words for what is one whole, as one lived-process" (Beittel, 1984, p110). The essence of quality resides not so much in the aspects or characteristics with which we attempt to describe it, but rather with the relationships between those aspects, and the coherence of the whole gestalt that those relationships produce, and hence with the meanings that such coherence implicitly evokes. And as with all gestalts, it is recognised as such only within the milieu of its production, only against the culture that is its backdrop, only in terms of the event through which it emerges. As no two products in this material world can ever be completely identical, so must the quality that characterises them also differ. As that quality is multidimensional, and contains relational aspects, it is idiosyncratic to each product, as well as to the conditions of its production. In general, discourse on quality is not amenable to that "better and worse," "more or less" description that is a prerequisite for any measure, and hence of any standard, or any categorisation. It is sometimes amenable to discourse, and to aesthetic response, and even to comparison in some of its aspects. And quality is amenable to change, both in its own meaning, and to the meaning it generates in relation to the product it relates to. Hence such discourse may indeed invite change in the product being discussed, and agreement be reached by some or all concerned (in that particular consensual event) that there has been a positive shift in quality. Such discourse, such agreement or
disagreement about quality, is itself a process of quality control, no
less effective because it is collaborative, and no less effective because
people disagree. As such it could provide another method of certification,
as indeed it more or less does among the elite of any profession; a fact
that for many would make a stronger case in this argument than any other.
For example, the final educational judgment of this work is with two examiners,
who may differ greatly in their opinions.
Standard products?So what? If in measuring the standard we have denied what is essential in quality, does it matter? Lack of official recognition of originality, a little repression of creativity, is unfortunate but hardly crucial in the world order. Yet the other side of the coin may well be crucial in the order of the world. For what is involved here is not a single instance of non-recognition, but the very production over thousands of instances of the thinking person, of the learning person, of the person in work, of the person with authority; of, indeed, the moral, rational person.For the standard is more than just one of many nudges and winks that lead the child to God. The standard, as applied continually through the strictures and structures of family and school and occupational work, at first externally and then through internal absorption and prescription, is the major mechanism, the quintessential carrot and stick, that moulds and shapes, that produces and creates that consciousness that defines the way each person sees the world, thinks about it, and acts within it. Not entirely, but largely so. And the individual produced through the notion of the standard, with its sharp cutting edge of adequacy, is a much more conforming, accepting, black and white, uni-dimensional person, and hence one far more socially controllable, than is one produced though the more spontaneous, multi-dimensional and unpredictable notion of quality. Maybe we don't need to de-school.
Maybe all we need to do is to acknowledge the arbitrariness and error that
permeates standards and their measurement, extol the virtues of immeasurable
quality, step lightly and quickly aside, watch the catagorisation structure
crumble, have faith in chaos theory to articulate another structure, and
hopefully nudge it in the direction of greater rationality and equity,
truth and compassion. But that's another story.
SummaryThe notion of the standard intervenes in the discourse about quality, and severely distorts it. The standard is a member of the class quality, is separated from it because of properties of measurement accuracy it is purported to have, yet is still confused with it. When the standard is seen, realistically, as unable to perform its function, we must return to quality as the notion with sufficient mythical, ideological, and intellectual status to replace it. This would predispose us to a rather different political structure, and to the recognition of a world in which simplistic notions of linear competition and dichotomous categorisations are replaced by more complex, ecological, and collaborative axioms.
Return to Table of Contents |