Other views relevant to qualitative confirmation
This section is presented to place M&F within the larger qualitative research community. Our discussion is in no way a complete review of the voluminous literature; it highlights different interpretations of the hypothesis setting process, validty and reliability concerns and understandings of translation and triangulation methods. In this section, we emphasize with italics the words that seem to us to be versions of confirmation. In many ways, LeCompte and Preissle (1993) support M&F's concept of qualitative confirmation. LeCompte and Preissle agree, at least metaphorically, that validity is involved in many aspects of the inquiry process: "How validity is defined and treated varies according to what researchers do, what tasks they are undertaking, and in what phase or stage of the research they are in" (p. 325). Theoretical frameworks, general design, context, participants, researcher experience and procedures of data collection and analysis have a bearing on the issue of validity. As LeCompte and Preissle say, "Consequently, although we urge scholars to discover and formulate what their research philosophy is, we believe that it is only one factor contributing to how validity is defined" (p. 326). They also caution, as do M&F, that replacing qualitative processes with strictly quantitative ones erroneously prompts a single consolidated definition of validity and potentially jeopardizes richness of detail and creativity. For LeCompte and Preissle, qualitative research is idiosyncratic and data analysis entails an emergent process: "Even midway through an analysis,uncertainty and frustration accompany the unfolding direction" (p. 330). They see qualitative research as loosely connecting researchers who come from a broad spectrum of philosophical traditions; there is not just one. They could ask that all the different qualitative researchers state how their philosophies decide validity and then apply those guidelines to the study, but they stop short of this because they see it as an a priori assignment approaching "determinism" (p. 326).
A comparison of their descriptions of research phases demonstrates how they propose to obtain validity (or qualitative confirmation) especially in the areas of: formulating goals, developing a research design, selecting data sources, experiencing and directing the research, collecting data, collaboration, comparing phenomenon and data analysis. Part of developing research questions is to ensure that the research goals, the context of the situation and the interests of the stakeholders are aligned. M&F would say that this process entails what must also be considered when developing a hypothesis. Much of the discussion around validity stems from concerns about the sources that are assumed to provide validity. LeCompte and Preissle argue (as do M&F) that qualitative research methodologies cannot discount the range of ontological and epistemological assumptions and theories that are at their disposal. The key to validity is that researchers must be aware of and acknowledge the use of disparate views. The kinds of evidence colleagues accept as legitimate and adequate may be affected by who is being studied. More data may be required because of who is going to scrutinize the results. Therefore, who or what is to be studied must be considered when deciding on a design. This is similar to M&F's concept of weighing (but not 'weighting').
Will there be other extraneous factors such as background evidence that needs to weigh into the confirmation of the hypothesis? Researcher's background and role in the investigation are central to how the validity is addressed:
History teaches that attention to the individual researcher is relevant to validity in qualitative research. What background and training does the researcher bring to the investigation? How carefully, thoroughly, openly, and honestly are researchers known to do their work? Who was responsible for the researcher's training? What reputation has the scholar earned in previous investigations? What does the researcher report about participation in the research? Introspective and reflective amounts of influence on what is seen and heard contribute to the audience's confidence that the researcher attempted to track these factors." (p.329).This is similar to M&F' constitutive and regulative rule that revealing the researcher's training and practice as well as the ethical or operational practices will help us assess how much credibility can be given to the researcher's work. LeCompte and Preissle agree with M&F that a systematic way of collecting data can be used to give more credence than one that is not. Both pairs also agree that just because something is done correctly, it may strengthen the research but does not necessarily mean that the results are sufficient to meet the criteria. LeCompte and Preissle suggest many ways for researchers to enhance confidence in their results, for example, through collaborative participation with the participant, congruency between theory and observation, intermethod and interobserver checks, personal reflection and introspection. Therefore, although LeCompte and Preissle would determine their analysis procedures at a different stage than M&F, they do agree with M&F that several options are available. Both agree that one should use multiple methods to reduce the possibility that bias will affect the credibility or validity of the results.
LeCompte and Preissle do not strive to provide conventional external validity because small sample sizes usually make this task impossible. However, they state that comparing phenomenon is useful and can be achieved by defining the "typicality" (Wolcott, 1973) of a phenomenon. Threats to comparing phenomenon are whatever obstructs or reduces a study's translatability. Translatability is the degree to which the researcher can adopt theoretical frames, definitions and research techniques accessible to and understood by other researchers in the same or related disciplines. Thus LeCompte and Preissle agree with M&F that alternative translations should be considered. However, LeCompte and Preissle think of translatability in terms of its usefulness in linking with others, whereas M&F think of translatability for reducing bias and confirming evidence or a hypothesis.
The one other area in which LeCompte and Preissle are distinct from M&F is in data analysis. LeCompte and Preissle state simply that one cannot predict what will happen, that trying to develop and use a template to direct the data analysis is impossible. They feel that "qualitative analysis is interpretive, idiosyncratic and so context dependent as to be infinitely variable. A creative analyst can never be sure that the ending will match the point of view adopted in the beginning" (p.330). In closing, LeCompte and Preissle agree with M&F that a single definition of validity is inappropriate for qualitative research. In a qualitative confirmation process, all authors agree that concerns about validity touch every part of the inquiry. LeCompte and Preissle use triangulation to understand a phenomenon; M&F use triangulation to confirm a hypothesis.
Guba (1981) outlined several paradigms for discovering "truth". These include a judicial paradigm that has well established rules for procedure, rules of evidence and criteria for judging the adequacy of the rationale for a proceeding. This judicial paradigm offers guidelines for behaviour. Another paradigm is that of expert judgement. The third is what he refers to as the rationalistic paradigm and is essentially connected to deductive thinking and a logical positivist point of view. The paradigm he obviously prefers is the naturalistic. The naturalistic is characterized by inductive thinking, and phenomenological views of knowing and understanding social and organizational phenomena. He notes that there are shades of grey in viewing these paradigms and that often they are seen as competing but in the task of knowledge production they are all important. Guba stresses that the naturalistic ecological hypothesis is imbedded in a context which is often more powerful in shaping behaviour than differences among individuals. In conclusion, Guba states that understanding the reality of the world requires acceptance of the notion that the parts cannot be separated. He further concludes that because of the assumptions underlying naturalistic enquiry the traditional concerns for objectivity, validity and reliability have little relevance for the design of the research. The validity of the findings is related to the careful recording and continual verification of the data that the researcher undertakes during the investigative practice. This is consistent with Wolcott (1990,1994).
The questions of translatability and comparability trouble qualitative researchers (Goetz & LeCompte,1984,1993; Wolcott, 1973). Appropriate sampling improves the generalizability of quantitative studies but researchers improve the quality of their qualitative studies by (among other things) ensuring that units of analysis, concepts generated, population characteristics and settings are described and defined so that they can be compared between studies. Translatability is related to the degree to which the researcher uses theoretical frames, definitions and research techniques that are accessible to or understood by other researchers in similar or related disciplines (Goetz & LeCompte,1984, p228). M&F do not hold contrary views to these discussions of scientific validation; rather they have built upon them in suggesting the possibility of confirmation of findings. Lincoln and Guba (1985), in outlining their fourteen characteristics of operational naturalistic inquiry, write: "naturalistic ontology suggests that realities are wholes and cannot be understood in isolation from their contexts, nor can they be fragmented for separate study of the parts; because of the belief in complex mutual shaping rather than linear causation, which suggests that the phenomenon must be studied in its full scale influence, and because contextual value structures are at least partly determinates of what will be found" (p.39). Spindler and Spindler (1992), when developing their eleven criteria for good ethnography say in Criterion II, "hypotheses emerge in situ as the study continues. Judgments on what may be significant to the study is deferred until the orienting phase of the field study is completed." M&F dodge these issues for most of their book, but they confront the issues in Appendix C.
On the subject of verity and what constitutes rigor, Marshall and Rossman (1989), suggest the use of "controls" which rely heavily on other researchers: the use of a research partner to play devil's advocate; a constant search for negative instances (from Glaser and Strauss, 1967); checking and rechecking the data, and purposeful testing for rival hypotheses; practicing "value free" note taking, and using "strictly objective" observations; devising and applying tests to the data; using the guidance of previous researchers to control for data quality; and conducting an audit of the data collection and analysis strategies (Lincoln and Guba, 1985, p.148). Marshall and Rossman neglect to tell us how a research partner will play "devil's advocate", explain what constitutes "value free" and "strictly objective" observations--nor do they tell us what types of tests to develop, how to develop them, or at what points we should test the data. Negative instances feature in M&F's rules, but their discussion is thin compared to others such as the above.
Talbot (1995), lists a number of factors involved in the credibility of findings: remaining in the field over a long period of time; using triangulation; negative case analysis; and having participants review researcher's interpretations and conclusions. She would appear to be in M&F's camp, because she uses the term "confirmability" to describe the process where findings, conclusions and recommendations are supported by the data, and she suggests that there should be an internal agreement between the investigator's interpretations and the actual evidence. Confirmability is only one of four factors that establish trustworthiness for Talbot, however, the other factors being credibility, transferability, and dependability. In establishing credibility, she borrows from Goetz and LeCompte (1984) who say that "establishing validity requires determining the extent to which conclusions effectively represent empirical reality, and assessing whether constructs devised by researchers represent or measure the categories of human experience that occur." This definition leans far too heavily on quantitative means in calling for a representation of empirical reality but is explicitly written and comprehensive in description in all other respects. Credibility crops up again as Lincoln and Guba (1985), discuss four constructs against which the trustworthiness of a study can be evaluated: credibility; transferability; dependability and (here's that confirmation again) confirmability. Credibility depends on how accurately the subject is identified and described. Transferability is noted to be impossible from the stance of external validity, but is greatly assisted by providing the greatest possible range of information, and thick descriptive data. The applicability of one set of findings to another setting rests more with the later researcher making the transfer than the original researcher. The authors point out that dependability is difficult to predict in a changing social world. In establishing dependability, the researcher attempts to account for changing conditions in the phenomenon chosen for study as well as changes in the design created by increasingly refined understanding of the setting.
The preface to Strauss & Corbin's (1990) book puts the question of confirmation clearly up front for those who are interested in inductively building theory through qualitative analysis of data. They state that however exciting may be the experience of gathering data, there comes a time when the data must be analyzed and at that time the researcher asks, "How can I have a theoretical interpretation while still grounding it in the empirical reality reflected by my material?" "How can I make sure that my data and interpretations are valid and reliable?" and "How do I pull all of my analysis together to create a concise theoretical formulation of the area under study?" The research question in grounded theory is a statement that identifies the phenomenon to be studied. As the researcher proceeds through the process they can go in different directions and therefore the questions can change. M&F seem able to accommodate this definition of research question and would assure the grounded theory researcher that what needs confirmation is the final conceptualization of the proposed theory and whether or not the symbols of behaviour and language to which the researcher reacted were true. The conceptualization question is one of confirmation; the latter is a question of methodology. Anselm & Strauss strive for rigor with seven evaluation criteria embodied in the questions: (1) How was the original sample selected? (2) What major categories emerged? (3) What were the indicators that led to the development of the categories? (4) What categories directed the theoretical sampling process? (5) What were the hypotheses pertaining to conceptual relations among categories and how were these tested? (6) Were there instances in which the hypothesis did not hold up to what was seen (7) How and why was the core category selected and on what grounds were the final analytic decisions made?
We can see within these criteria many of the issues of confirmation and disconfirmation addressed by M&F. They also relate to the criteria for the evaluation of qualitative research proposed by Lincoln and Guba(1985), who were among the first to suggest criteria for good qualitative research. Strauss and Corbin(1990) and Lincoln and Guba (1985) differ in their area of emphasis, with Strauss and Corbin(1990) being more concerned with internal validity and Lincoln and Guba(1985) wanting the work to shed light on other instances. M&F seek to accommodate these different dimensions and to add a strategy that will enhance work that has been started by others.