Description of the Book This is a scholarly book, in conception and content (but alas not in execution). We mean by "scholarly" that it digs deep for core concepts in what has been written about qualitative research methods-ideas, constructs, images that help us understand more than one case study, more than one ethnography, more than one narrative. Do such "core concepts" exist? Miller & Fredericks (M&F) think so. Otherwise how could they organize a book around the construct "qualitative confirmation"? What could it possibly mean to "confirm" findings from qualitative research? Reading Smith and Heshusius (1986) or Wolcott (1994), one might doubt the possibility. Then there is postmodernism: doubt, doubt, doubt; everywhere we look there is doubt. A few years ago, the head of the British Museum of Natural History reviewed challenges to Darwinian theories of evolution and found merit in some of them. Attacked by critics for sowing doubt, he replied, "Doubt is splendid stuff." (4) Doubt may indeed be splendid stuff, but we humans seem to crave something else, something more solid in our mental life--something such as confirmation. M&F offer material that is relevant to researchers everywhere but their editors have done them a disservice by presenting the work in a careless and awkward way. The lack of bold faced type and subheadings make an already difficult text even more difficult to read. References are given at the end of each chapter and each appendix, except for Chapter Three which has none. As a result, several are missing and there are many duplications. Outright errors appear that would be caught by even a cursory glance from an editor. (5) A curious feature of the book is that of its 153 pages, 62 are devoted to three appendixes which contain crucial elaboration or explanation. That said, we believe these authors provide a unique and powerful contribution to qualitative research--concepts and procedures that are worthy of study. Confirmation is the central concept of the book, a concept M&F argue, and we agree, has not been adequately addressed within the qualitative field. Chapter One Confirmation theory is introduced as the "major issue in qualitative inquiry", defining it as a description of what it means to say that the evidence relates to the hypothesis in the qualitative process. It is "the theory of when and how much different evidence renders different hypotheses probable" (Swinburne, 1973, p vi). It is acknowledged that this process is not free from assumptions and thus evidence can ultimately only be known through the senses. What constitutes data depends largely on who is looking for it; where mathematics may deal with pure fact, physics may not. The concept of weighing the evidence is introduced and weighing is distinguished from weight. Weight refers to the outcome while the weighing is the process of arriving at the outcome. In addition, weight can only be assessed by giving a rationale for the weighing of the evidence. Although M&F concede the concerns that have arisen over Hempel's classical framework of rules (1965), they value it and build directly upon it. Hempel defined: hypothesis, observation report, observation sentence, entailment, direct confirmation, confirmation and development of a hypothesis for a class of individuals. (quoted by M&F on pages 8 and 9). They also cite Carnap (1962), who argued that although confirmation is central to understanding the evidence relationship, the term is also susceptible to different interpretations. Carnap, however, relied on computational interpretations and M&F argue that that would defeat the purpose of qualitative confirmation. The authors' intent is to use a non-probability framework to focus on a logic of discovery while not excluding the possibility of validation. Since evidence comes in so many forms, rules are needed to help assess the evidence in relation to the research question (hypothesis). It is also necessary to develop a series of related but separate rules which bear only on the characterization of the data itself (p.12). M&F suggest that the ways qualitative researchers make a case for their findings will depend on two major aspects: the labeling of and the justification for using the rules. Two categories of rules are required: constitutive (defining what counts as a social situation or practice) and regulative (prohibiting or prescribing actions in situations defined by constitutive rules-taken from Greenwood, 1989). ). In their discussion, M&F are "trying to understand what it means to be 'rational' in pursuing the activities of qualitative inquiry" (p. 14), and they reach back to "camps" defined by Wittgenstein, Popper and Donald Davidson (1984) for insight into rationality. To the present reviewers surprise, they then add "sociologists" to the list, a camp that "tries to determine the extent 'natives' hold their beliefs even in the face of (supposedly) other more 'rational' beliefs" (p. 14). Sociologists? The last "camp" specializes the thorny and recurring issue of "translation", an issue that permeates any discussion of qualitative research. Whenever we use words to communicate the behaviour and/or feelings of others, the communication involves translation even when we all seem to be speaking the same language. Translation is discussed in every chapter of M&F, from many perspectives. Translation issues revolve around the ideas of indeterminacy and incommensurability. These two constructs are not dealt with at length. The "indeterminacy thesis" is: "because theories of meaning (i.e. those referring to natural languages) are not concerned with 'the fact of the matter', as are scientific theories in the natural sciences, it is possible to derive multiple translation manuals which may be incompatible but adequate for interpreting the behavior in question." The "incommensurability thesis" suggests that there is not even one translation manual that is adequate, as when trying to choose between two competing theories, for example (p.15). (For a comprehensive discussion of these concepts see Quine, 1960.) Translation is often at the level of data analysis rather than confirmation. Rationale for data collection processes and the subsequent description or translation of the data often is directed by the chosen research methodology. M&F close Chapter One with six statements describing how the above considerations relate to the notion of qualitative confirmation. Chapter Two The second chapter is devoted to the subject of hypotheses in qualitative research, not surprisingly since hypotheses are central to their entire approach. Hypotheses are viewed broadly "as statements which direct inquiry, in a Deweyan sense, in relation to a theoretical framework, but without the necessity of such hypotheses being strictly deducible from such a framework" (p. 21). A priori hypotheses may emerge where the researcher has a theoretical framework in mind, for example, beginning with a category of behavior and linking it to a category of individuals. In contrast, the researcher may begin with an interest in a category of individuals and subsequently seek to link them to a category of behavior. M&F argue that both are qualitative because they are formulated in the context of a qualitative study and because their confirmation is dependent on qualitative data; they dodge the obvious tautology by acknowledging that both types can be formulated as well in quantitative studies. Confirmation, they say, is central to all research activities, but in qualitative research it is based on plausible logical relations which apply first to the data and then to the hypothesis. Qualitative research seeks to understand human behavior "from a perspective which, methodologically, requires qualitative data" (p. 25). Rules, they say, are needed if a case for qualitative confirmation is to be made. Since rules must apply both to data and hypothesis, M&F undertake to define what they mean by data-and their definition is entirely conventional: field notes, interview data, historical accounts and the like (pp. 26-7). The term "evidence-instance" is introduced to describe a discrete item of information to be put forward as part of a confirmation argument. A long elaboration on the meaning of "evidence" is presented in Appendix B, including more discussion how data become evidence. ( 6 ) They discuss triangulation, breaking no new ground, and then tackle the really tough question: what counts as evidence and how do you judge its importance? Nothing new here, and we are all reassured to read, "It becomes quite difficult, then, to develop hard-and-fast rules for determining appropriate 'weighting' procedures for these kinds of issues" (p. 30). Some considerations that "may be helpful" include: The weight of the data-evidence is not necessarily synonymous with the "amount" of the data-evidence. The term "weight" can be loosely translated as an "absence of negative cases," that is, disconfirming instances. … Where the term "amount" of evidence is used, the implication is that some of the evidence must consist of (negative) disconfirming instances. … M&F assert that contrary findings do not necessarily disconfirm the original hypothesis, for they may serve to clarify a previously unknown dimension. The matter of confirming vs. disconfirming instances is discussed more fully in Chapter Three. They tell us that using more than one data set can enhance the validity or reliability of data if it can be shown that the data sets are relevant to the problem--however, they neglect to explain how this can occur. M&F relate the purposes of using a triangulation process but the inherent problems associated with it are listed and not explained. We recommend that those unfamiliar with the process of triangulation consult other sources. In what must have been an afterthought, the final paragraph of the final section, "Conclusions", introduces a muddled discussion of the terms "relationship" and "association". Hopes that the muddle will be cleared up are not fulfilled. Chapter Three. In chapter three M&F present their four rules for qualitative confirmation and instruct us in the use of these rules, including how to handle negative cases or cases of disconfirmation. Overall, it is the most carefully constructed and clearly presented chapter. How qualitative findings may become evidence for a hypothesis or research question is the focus of the discussion here and the authors impress us with how their qualitative research rules apply to both sides of the research continuum. One advantage claimed for researchers is that the rules allow for the development of qualitative research studies with confirmation in mind. They also suggest how research findings are to be interpreted when choosing data sets. As for qualitative disconfirmation, some additional advice is given in the interpretation of negative evidence­instances. Researchers usually have to use the qualifier "some" instead of "all" (Quine­Duhem Thesis) because scientific theories may be composed of auxiliary hypotheses for certain predictions (p.48). Auxiliary hypotheses can allow incompatible evidence while still not disconfirming the original hypothesis. The null hypothesis also finds a place in Miller and Frederick's qualitative confirmation theory. Chapter Four. Chapter Four moves into the practical application of rules to five articles published since 1980. M&F pick up on a suggestion from Miles and Huberman (1984) that researchers ask systematically a series of questions before, during and after the research: (p.53): . what is (are) the major research question(s) or hypotheses? . what method(s) will be used and what sort of data yielded? will other(indirect) data sources be utilized? . how will the case be made that the research question/hypothesis has been or has not been confirmed? . if mixed data sources are used, how will they be handled in terms of confirmation/ disconfirmation? All researchers are advised to include a final section in the research report (possibly as an appendix) which explicitly addresses the issue of qualitative confirmation. Such a section helps the researcher focus on the larger epistemological­theoretical questions raised by the study and reminds us that the presentation of our findings can be regarded as a type of translation manual as well as a demonstration of the rigor in one's research. The application of confirmation theory to the five caseshelps a little to understand M&F's method, but their choice of articles left much to be explained. A graduate student listed 20 reports of qualitative research from the years 1970-1990. The same student selected 7-10 for further study, in an attempt to present a "rough cross section" of work. In our view, a more qualitative approach would have been preferable: purposive choice to illustrate application of the rules rather than this version of sampling. In only one study, for example, was there any attempt to discuss negative or disconfirming instances. In most there was no attention to the issue of confirmation, but in one or two there was an implicit suggestion that confirmation had been achieved. There was no discussion of the weighing the evidence in making qualitative claims. Given the rather radical nature of M&F's proposals, it should perhaps come as no surprise that research conducted and published more than a decade ago does not adhere closely to the Rules. In order to help future researchers, M&F provide a "Checklist for Qualitative Confirmation" at the end of the chapter, including the questions mentioned above and ending with "What, after all, has one discovered/concluded from this investigation? Is it truly warranted?" (p. 66-7) Chapter Five. The final chapter (before those three long appendixes, that is) is entitled, "Epistemological Asides and Conclusions"; fearlessly confronting the big question of "truth". Suppose a study has met all the criteria for qualitative confirmation, "Is this all that is necessary for declaring that the findings, then, are 'true'?" (p. 73) M&F advise us to look at various philosophical camps, bringing Winch again (1958, 1964) and again invoking the "camps" (from Chapter One). Qualitative confirmation provides a framework for establishing qualitative data as evidence for the hypothesis, that is, it meets the requirement of rationality, but it is less successful as regards indeterminacy. It is not possible to rule out different or competing translation manuals. As readers will have gathered by now, this concept of "indeterminacy" pervades the book--as it does the field of qualitative research. It is a way the philosophers have discussed whether there can be any "truth" and the way M&F discuss the prevailing view in qualitative circles that there is no truth-there are no methods that allow us to reduce indeterminacy to zero. Readers may feel, with justification, that M&F's discussion of the concept is fragmented and incomplete. Fear not; Appendix A (24 dense pages) is devoted to it. Appendix A. Some notes on the nature of methodological indeterminacy. Here the authors return to the intellectual roots of qualitative confirmation and elaborate the construct, "methodological indeterminacy". M&F revisit Quine and lean heavily on Roth's 1987 book, Meaning and method in the social sciences: A case for methodological pluralism. Roth, echoing Winch (1958), opens his book by referring to "the general collapse of positivism" and concludes (following Quine) that multiple translation manuals are not only possible but inevitable. There is no "fact of the matter" in relation to the human sciences. Where can we go from here? Undaunted, M&F press on, seeking to dodge the philosophical bullet by concentrating on practical concerns, namely methods. …the genuine problem of indeterminacy for the human sciences does not lie at the level of debates concerning "hermeneutic" vs "scientific empirical" views of human action, but rather at the level of specific methodological techniques that ultimately are the constitutive elements for determining the existence (or lack of it) of indeterminant translations. (p. 93, emphases in original) …… To be clear, we are not arguing that the definition of methodology as the application of specific techniques and procedures is sufficient to resolve indeterminacy in all situations, but only that it is a necessary condition both to establish its existence and to demonstrate possible ways of reducing it. (p. 94, emphasis in original) Some clarity does emerge: specific techniques and procedures (methods) are needed to establish the existence of indeterminacy. It is always there, we suppose, but until you apply the techniques and procedures (collect the data?), you don't see it Now that positivism has collapsed, we go farther: application of methods "may generate a type of indeterminacy" (p 101), or "types of indeterminacy that are produced by competing methodologies" (p. 103, emphasis added). This is surely correct, because different methods bring to light different amounts and kinds of indeterminacy-hence the term, "methodological indeterminacy". M&F borrow from quantitative methods and discuss differences "between" and "within" methodological approaches, with regard to both qualitative and quantitative approaches. The discussion of qualitative approaches is much more insightful than that of the quantitative, the latter naïve and verging on the trivial. We question M&F's assertion, citing Fuller (1988), that "there is a heavier 'burden of proof' on the qualitative side of the equation for showing either indeterminacy or lack of it." (p. 105) A better statement, in our opinion, is that for practicing researchers, "indeterminacy is usually perceived itself as a 'working hypothesis'; one which may never in principle be unequivocally 'accepted' or 'rejected' but one, nevertheless, that is capable of empirical inquiry." (p. 105) Appendix A concludes with a short discussion of the status of theory in the human sciences. (M&F leave no topic completely untouched!) They cite a belief among "some quantitative researchers" that improvements in measurement will solve our problems, including give us better theories. In our opinion, they have it backwards: better theories lead to better measures (and the process recycles). Let M&F have the last word, with which we completely agree: On the other hand, for those who do not see the possibility (or usefulness) of equating the human sciences with the natural sciences, the indeterminacy reflected by methodological applications can be "reduced" over time, but its reduction is directed towards making human behavior more "intelligible" rather than more "scientific". (p. 108) Appendix B. Clarifying the "adequate evidence condition" in educational theory and research The authors set out to convince us how important it is to clarify what they call the "evidence condition". Lack of attention to what is meant by evidence "has resulted in a restricted view of this term with a correspondingly unwarranted optimism regarding the formulation, implementation and evaluation of educational policies and practices." (p. 117) Faithful to their book title, they promise a thick description type of analysis to attain their goal. If "dense" is the same as "thick", then they deliver on this promise. Israel Scheffler (1965) is cited early, as we would expect, and not just because he introduced the phrase, "adequate evidence condition". No discussion of evidence in educational research would be complete without at least a mention of this classic. New sources to us are Lakoff (1987) Women, fire, and dangerous things: What categories reveal about the mind, and Lakoff & Johnson (1980) Metaphors we live by, insightful discussions of the nature and process of category creation. Since qualitative researchers live and die by their categories, these are helpful references for anyone (such as the authors of this review) who had overlooked them. The process of categorization is the formation of "Idealized Cognitive Models …culturally unique and semantically-based modes of perception and reasoning whereby users of a language construct cognitively-based models to explain social and physical reality." (p. 124) There are five types of models: Cluster, Metonymic, Social Stereotype, Ideal and Metaphoric. We are not attempting a summary of this appendix. M&F describe the concept of evidence as having "entrenched ambiguity" and "overall complexity", and after reading the appendix several times we heartily agree. Lawyers and judges struggle with it daily, of course, and law schools have whole courses on it. M&F also write, "Paradoxically, however, the more we try to unpack the notion of evidence for the sake of clarity, the more problematic it becomes, especially in trying to provide adequate justifications for educational policy making issues." (p. 119) Here are a few points to give the flavour of the work. Among the ambiguities cited is the "conflation" of the terms evidence and data, as we noted in the description of Chapter Two and in Note (6). According to M&F, data are candidates for evidence but only become evidence "to the extent that they are consistent with both the larger class of rules identified with the particular methodological approach, and with the rules regulating the application of a particular technique or tool within the methodological approach." (p. 118) Whew! Educational implications are offered, including ambiguities surrounding the concept of "school effectiveness" and the lack of consensus on adequate evidence for effectiveness. You might not agree with everything they write, but you will be led to think hard about it. Appendix C. Reciprocal paradigm shifts and educational research: A further view of the quantitative-qualitative dilemma In the concluding appendix, M&F state their belief and concern that while those involved in educational research perceive a paradigm shift from quantitative to qualitative inquiry, many are not convinced that the qualitative-interpretationist view can be sustained on the basis of conventional scientific criteria. Some, Miles and Huberman (1985) for example, manage to sidestep this issue and some recognize qualitative research but believe that the eventual testing or proof of causality or theoretical prediction must be left to scientific empiricism (Popper, 1969, Rosenberg,1988). M&F concentrate their discussion on the interpretationist framework and they examine some commonly held assumptions that underlie the two research processes. They present their position that quantitative and qualitative approaches need not be viewed as contradictory in terms of some educational problems within some contexts, and a methodological mix is not only possible but desirable. Miller and Fredericks do not agree with the asumption that scientific empirical data is quantitative data and that the constructs of validity and reliability are crucial issues only in quantitative studies. One of our central claims will be that not only are the crucial issues of reliability and validity perfectly general across all research based forms of inquiry; but also that in meeting the requirements of reliability and validity, interpretationist accounts may yield warranted conclusions that, while different in form and content from scientific empirical claims, can, nevertheless, be compatible with them. (p. 137) They remind us that these assumptions have their roots in the traditional confirmation theory of Carnap (1962) and Hempel (1958) and are drawn from a "probabalistic-operational" perspective. This approach to the definition of scientific evidence seems to rule out the possibility of interpretationist accounts of data that can only be understood rationally as qualitative data. In response, they put forward and discuss the following assumptions (p. 139): interpretationist accounts within the human sciences are correctly based on the direct or presumptive use of non-quantitative research strategies, these strategies must be viewed as being fundamentally different in their qpplication and in the interpretation of the data they produce from those research strategies based on probability assumptions, they are, nonetheless, subject to the general scientific constraints of reliability and intersubjectivity, and they can be employed as alternative means for "confirming" hypotheses and/or providing for critical tests for theoretical perspectives. These assumptions lead them to confident statements about the tough issues of causality and theoretical interpretation of reports from qualitative studies. "First, the qualitative research agenda stipulates only that causal connections (as either necessary or sufficient conditions) of behavior be meaningful in terms of the native's own accounts and that such connections be accurately described. Secondly, the investigator's theoretical interpretation of these accounts can incorporate 'higher order' or more general constructs, but without the implication that their use must necessarily 'reduce' the original accounts." (p. 145, emphasis in original) Return to the Contents Table. Go on to the next section.