This article has been retrieved times since July 16, 1993.

Education Policy Analysis Archives
Volume 1 Number 9	July 16, 1993	ISSN 1068-2341

---

A peer-reviewed scholarly electronic journal. Editor: Gene V Glass, Glass@ASU.EDU. College of Education, Arizona State University,Tempe AZ 85287-2411 Copyright 1993, the EDUCATION POLICY ANALYSIS ARCHIVES.Permission is hereby granted to copy any article provided that EDUCATION POLICY ANALYSIS ARCHIVES is credited and copies are not sold.

---

Evidence, Ethics and Social Policy Dilemmas

Steven I. Miller
L. Arthur Safer

Loyola University Chicago

Abstract:

Within the philosophy of the social sciences, the relationship between evidence, ethics, and social policy is in need of further analysis. The present paper is an attempt to argue that while important social policies can, and perhaps ought to be, grounded in ethical theory, they are seldom articulated in this fashion due to the ambiguity surrounding the "evidence condition." Using a consequentialist-utilitarian framework, and a case study of a policy dilemma, the authors analyze the difficulties associated with resolving policy-based dilemmas which must appeal to evidential support as a justification for an ethical stand. Implication for the relevance of ethics to social policy formulation are discussed in detail.

This paper attempts to examine what we will call the "evidence-ethics-policy triad." Our initial claim will be that while the "ethical" component of the triad has increasingly become an important consideration in the conduct of social policy inquiry, its actual influence on the formulation of social policy is minimal, at best, and, at worse, irrelevant. Likewise, the purported utility of addressing ethical concerns within the context of social science practice directed towards social policy is grounded in a variety of stances broadly spanning, but not isometric with, the traditional divisions of ethics within philosophy, namely, "consequentialism-utilitarianism" and "deontology" (Strike and Soltis, 1985). However, a certain type of asymmetry characterizes the consideration and application of ethical thinking as a possibly relevant guide to social policy considerations.

What we are suggesting, specifically, is that an influential and significant portion of the social science community is, implicitly or explicitly, committed to some version of ethical relativism. Now, such a commitment is "asymmetrical" in the sense that it neglects the possible importance of "deontological" ethical positions in explaining human actions, individually or collectively. In other words, relativism is more closely aligned with the "consequentialist-utilitarian" position in traditional ethical theory with its emphasis on the "outcomes" or "consequences" of human action, and the tacit recognition that such outcomes are socially constructed and changeable overtime and within and across cultural contexts (Barnes and Bloor, 1982, Winch, 1964). Although the contemporary emphasis within the social sciences (both empirical and hermeneutic) on "relativism" might seem removed from the classic utilitarian positions of Mill and Bentham (see Rawls, 1971, Ch. 1, sec. 5), it shares an affinity with these positions by virtue of its emphasis on the possibility of objectively assessing the factors (economic and cultural) which will produce the "greatest good for the greatest number." While the interpretation of such a "maximization" principle might vary cross-culturally, for instance, it is the presumed methodological underpinning of the social sciences to a utility or maximization principle that is the salient "ethical" component of these disciplines.

To put this somewhat differently, the incorporation by the social sciences of "post-modern", "anti-positivistic" or "deconstructionist" values as a result of Kuhn's (1970) initial insights (especially on "incommensurability") has shifted the "ethical" balance away from deontological views to consequentialist views. (However, some critical theorists, such as Habermas (1979), argue that values such as freedom and justice have an "objective justification." In this sense, if Habermas is to be viewed as a social scientist, such ideas could be construed as broadly deontological.) Thus, deontological positions, which favor the interpretation of values under general principles rooted in a "universal" human nature, relevant cross-culturally, have been undermined by the strong emphasis on relativism in the social sciences. Perhaps the only notable exception to this relativistic interpretation of values has been the work of Kohlberg (1976) and, perhaps, Piaget (1965), both arguing, although in different ways, for the universality of "stages" of moral development. While Kohlberg's work has come under criticism (Simpson, 1974, Sullivan, 1977, Wilson, 1976) in terms of gender biases, lack of sufficient evidence for the "higher" stages of moral development, and a variety of methodological concerns about validity and reliability, his work stands as probably the closest empirical-deontologically based argument against strict relativism. None-the-less, there still appears to be a strong commitment among the social science community to the relativist-consequentialist point of view. In our view, this has been brought about by viewing the Kuhnian and post Kuhnian interpretations of scientific incommensurability as overriding the efficacy of deontological interpretations.

II

Perhaps a further rejection of the deontological framework lies in the closer (presumed) affinity between consequentialist- utilitarian assumptions and the requirement of evidence. That is, the historical commitment of the social sciences to justify their knowledge claims empirically (i.e., through reliable and valid methodological techniques) is more compatible with consequentialism with its emphasis on "utility," the "greatest good for the greatest number," and the possibility of measuring "consequences" themselves. Thus, for example, if we can show that welfare-policy X will provide the "most benefits" for a given group, the evidence for such a claim will carry more "weight" than if we argue that it "ought" to be implemented because it falls under a "binding" deontological principle. (See, for instance, Carol H. Weiss, "Ideology, Interests, and Information" in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 1983, pp. 213-245. Weiss discusses the role of social science information and how it may be "weighted" by policymakers for particular ends.) The latter may, indeed, be persuasive; however, if viewed from a social science perspective, the former has the appearance of being more scientific, or at least more allegedly scientific, although possibly "relativistic" in terms of specific applications. Nevertheless, the reliance on evidence as the arbitrator in social policy debates, whose justification is situated in a consequentialist-utilitarian ethical framework, is a double-edged sword. Its positive aspect lies in the ability to generate evidential support for "maximizing" a given position, while its negative aspect lies in the systematic ambiguity surrounding the term "evidence" itself. That is, the very notion of "evidence" within the social sciences is often too quickly perceived to be non-problematic, while in fact, we will argue, it is at the very heart of the problem of trying to justify a relativistic- consequentialist view of social policy formulation. (For a good discussion of the assumptions about empirical evidence that policy analysts rely on see Ruth S. Hanft, "Use of Social Science Data for Policy Analysis and Policymaking," in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 1983, pp. 249-270, especially, pp. 258-261.) The following are a few examples of the ambiguity surrounding the concept of evidence.

1. "He has adequate evidence for this belief in X."
2. "The evidence certainly seems to support her position."
3. "There is certainly conclusive evidence of his guilt."
4. "There is some evidence of his guilt."
5. "The evidence confirms/supports the hypothesis."
6. "Her evidence for X seems better than his."
7. "His evidence was relevant at T1, but not now at T2."
8. "The evidence for his theory is very (highly) probable."

Such a list could be further extended, but the above examples give a representative "flavor" of how the term is often used. We will not compare and contrast the various possible meanings of the term "evidence" in these examples but only note two features: first, it is often taken as a "given" or "primitive" term (i.e., left undefined), and second, the term is "relational" in the sense that its "meaning" is supposedly clarified or modified "in relation to" another term (e.g., "total" evidence). These two features are important because they pinpoint, or at least suggest, where the ambiguity lies (Miller and Fredericks, 1992). This "linguistic" dimension of the term "evidence" has, furthermore, often been overlooked because of the persuasive influence of the branch of epistemology called Confirmation Theory. Stemming from its origins in Local Positivism, Confirmation Theory has become a highly technical and esoteric sub-specialty of epistemology, and specifically in terms of its applications to various problems (e.g., "confirming" theories) within the philosophy of science. Basically, confirmation theory is concerned with explicating the "rules" that exist between a claim (i.e., a hypothesis) and "evidence" for this claim. These "rules" are of two types: (1) logical and (2) inductive. The first refers to traditional and "modern" (e.g., "sentential calculus" and "predicate calculus") extensions of the rules of inference. The second refers to the rules of inductive logic and specifically to the "probability calculus", i.e., the rules one follows in establishing the "probability" of events.

Briefly, then, what confirmation theorists attempt to do is to "workout" the logic (i.e., the rules) that must pertain between a statement and evidence for that statement, so that one can say the statement has been "confirmed." Now, while there is a vast literature on confirmation theory (e.g., Achinstein, 1983), and a host of thorny problems associated with it, we have mentioned it here to illustrate three points: (1) it has become the dominant form of analysis of thinking of how evidence is used to give "warranted support" to statements of belief, (2) it has gained this "power" by reliance on applications of the probability calculus, and (3) it has assumed the term "evidence" to be non-problematic because "evidence" simply becomes any and all statements translatable into probability statements.

The last point (#3) is significant because it has obscured the complexity of the term "evidence" by subsuming it under probability theory. In effect, something is evidence if it can be given a probability estimate. While there is nothing inherently wrong in such a conception of evidence, it does however limit the range of what can count as evidence. Specifically, such a construal rules out, in terms of being acceptable evidence, many "qualitative" claims in such areas as history, law, psychoanalysis, literary criticism, and those branches of the social sciences which opt (for philosophical reasons) for evidence which is not quantifiable. To rule out these other sources of evidence, or relegate them to a lesser status, because they do not conform to a probabilistic definition, is to drastically restrict what can count as evidence. However, to admit "qualitative" evidence is to raise additional questions, most notably, how does one assess such evidence as "support" for a given claim? (A very good defense of using qualitative data for policy analysis is given by Bruce Jennings, "Interpretive Social Science and Policy Analysis," in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 1983, pp. 3-35.) There is, then, a parallel problem of determining the "rules" for this type of evidence. If such rules cannot be formulated, one is "stuck" with the problem of having potentially "good" evidence for a claim but not being able to say how it counts for the claim (see Miller and Fredericks, 1992, and Miller, 1991 to see how such a case for qualitative evidence can be made).

Within these considerations for what constitutes evidence, some other distinctions need to be briefly mentioned. One is what we will call the "temporal status" of the evidence. That is, does one "admit" evidence for a claim that is specifically generated for that claim, or evidence that is "related" to the claim; i.e., obtained from existing evidence about the claim? Some would argue that only the former constitutes "legitimate" evidence since it directly addresses the issue at hand. Others would contend that "related" (or "after the fact") evidence is equally legitimate and admissible if it concerns the claim in question. However, in this instance, one must also address the "closeness-of-fit" issue. That is, if "related" evidence is admitted how "close", methodologically and substantively, must it come to the claim? For instance, will study A be admitted as evidence for the claim if, and only if, it replicates a study that would have ordinarily been conducted on the claim? In other words, how "broad" or "narrow" must related evidence be before it is admitted as "legitimate" evidence for the claim? Do we need "rules here?" We would say "yes", and furthermore argue that one of the failures of traditional epistemology has been its inability to formulate appropriate rules for such cases.

Lastly, there is the vexing problem of what we will call the "mixed evidence" case. In this situation we can have a combination of direct and related evidence bearing on a claim but the evidence comes from a variety of methodological approaches. There may also be variance as to the "closeness-of-fit" criterion, with, however, all the diverse evidence still bearing on the claim. These considerations suggest that the term "evidence" is a highly complex one involving not only different forms or types but also the possibility of different "rules" which translate data into evidence.

The point of these comments, so far, has been to suggest the fundamental importance of the "evidence condition" for the ways evidence is related to a variety of ethical concerns within the specific context of policy-related dilemmas.

III

In a policy-related dilemma, one is confronted with two (or more) seemingly contradictory alternatives. The problem is that only one can be chosen, but choosing one produces negative consequences (for someone) in terms of the other. For ethical theory, the issue becomes one of making a given choice and justifying it by way of an ethical position. This assumes, of course, that policymakers do reflect on the ethical implications of their decision(s). This, probably, is an unwarranted assumption: decisions are often based on such considerations as power, expediency, persuasion and fatigue. Although we believe important policy decisions are seldom made in regard to ethical considerations, we want to outline some of the considerations a policymaker would encounter if he/she were in the position of justifying decisions based on ethical concerns. We are also assuming that the policymaker is reflecting his/her "proper" role as a policymaker (i.e., he/she has the "authority" to do so, leaving aside the source(s) of such authority), and that the level or type of decision to be made, in itself, has no bearing on the type of ethical justification to be utilized, i.e., a more "serious" decision requires (it may be argued) a deontological justification.

An initial way of describing the type of analysis we wish to pursue may be illustrated as follows:

Need for "Evidence"
Type	Yes	No
Consequentialism	a	b
Deontology	c	d

Figure 1

In this simple "crossbreak," we are roughly classifying the two major types of ethical frameworks as: (1) consequentialist (utilitarian) referring, again very broadly, to the type of ethical reasoning which argues that the "goodness", "rightness" or "justness" of an action is determined as a function of the "consequences" it produces, either for the "many" (i.e., the "majority") or for the relevant group under consideration, and (2) deontological, referring to the evaluation of an action by way of "rules" which in some sense are considered to be "moral", "fundamental", "basic", "a priori", or "universal." Likewise, we will not deal with such variations as "rule utilitarianism" or the origins of the rules within the deontological framework, e.g., if they are "truly" universal or culturally specific. These are important issues, but ones that would lead us into other directions.

Also, because of its ambiguity, we have put off trying to define the term "evidence"; however, a tentative (and rather broad) definition follows below:

Evidence consists of statements of belief concerning the "objective truth" of some given state of affairs. Such statements can assume many forms, especially if they are of a "synthetic-a posteriori" variety, and are (in this sense) related to the methodological approaches used to generate them. Statements of evidential belief become evidence for a claim to the extent they can be justified by "rules" which themselves are taken to be "self-evident," "reasonable", or "warranted."

Referring to Figure 1, the question is whether some type of evidence is necessary to address ethical claims? Again, generally, it is assumed that "consequentialist" views require some justification by way of "empirical" evidence. That is, if either "positive" or "negative" consequences are projected for some course of action, there ought to be "evidence" for supporting such claims. On the other hand, for deontological approaches, the appeal to "empirical" evidence is not necessary since the principle itself (i.e., the "rule[s]") are either taken to be self-evident or somehow grounded in "human nature" or human reason. Where "evidence" is appealed to in such instances, it is in the form of an appeal to non-empirical "reasons." (Note: "empirical" is put in quotation marks to indicate that such sources of evidence are fundamentally related to some notion of "sense perceptions" or "sense data," but (also) that they are not limited to only quantitative methodological approaches).

As an aside here, we would like to mention that the issue of the relationship of evidence to certain types of ethical reasoning is situated in a broader (and difficult) epistemological context, namely the "empirical justification" of beliefs (see Moser, 1985 for an excellent overview of this subject). Generally, empirical justification, or more broadly "epistemic" justification, is concerned with formulating and defending a variety of logical "rules" for saying that X has some notion of "justified true belief." There are several different (and often competing) views--e.g., "contextualism," "foundationalism,"--of how one comes to justify a particular claim about the world. Now, while Confirmation Theory, previously mentioned, is an important sub-set of theories of epistemic justification and related to the issue of how evidence relates to certain types of ethical reasoning, the broader issues of epistemic justification are only indirectly relevant (if known at all) to the reasoning utilized by most policymakers in trying to "resolve" dilemmas. (Moore (pp. 281-285), for instance, argues that even if policymakers are aware of the technical aspects of what constitutes adequate evidence in the social sciences, such considerations must be balanced against the practical demands of the policy being formulated. This "tension" may, of course, result in "ethical" problems for the policymaker, i.e., what evidence ought to be chosen and how justified? See Mark H. Moore, "Social Science and Policy Analysis," in D. Callahan and B. Jennings (eds.), Ethics, The Social Sciences, and Policy Analysis. NY: Plenum, 1983, pp. 271- 291.)

Thus, based on our own observations and readings, we do not believe these larger (and technical) concerns play an important part in guiding concrete policymaking decisions. For example, a policymaker is probably not overly concerned (even if she or he is aware of it) of the "infinite regress" problem associated with using empirical evidence for a particular decision (BonJour, 1985). This issue would have the policymaker worrying about the epistemological problem of justifying a given "piece" of empirical evidence by having to justify it by some other piece of evidence (i.e., a belief about it), which, in turn, must be justified, etc. We are not suggesting that policymakers ought not to be interested in these broader epistemological concerns, but only that they probably seldom are. Similarly, we do not believe policymakers (again, even if known) are primarily concerned with the different varieties of their consequentialist or deontological ethical positions in justifying a particular course of action.

For the policymaking process, this suggests that crucial decisions are most likely made by reference to non- epistemological (and non-ethical-theoretical) considerations such as power, persuasion, time, decision-immediacy and so forth.

Nevertheless, continuing with the issue of how some conception(s) of evidence may relate to resolving policy dilemmas by way of ethical positions, we list some of the possibilities in Figure 2.

Policy-Related Dilemmas Alternatives
A		B
	Appeals
1. Consequentialist	vs.	Deontological
2. Deontological	vs.	Consequentialist
3. Consequentialist	vs.	Consequentialist
4. Deontological	vs.	Deontological

Figure 2

Figure 2 simply outlines the possibilities for initially addressing a policy-related dilemma in terms of either a consequentialist or deontological stance. There is, of course, no mention yet of the "evidence" issue; this will be forthcoming.

Thus, we are assuming that the policymaker must choose one of the alternatives, and that there is a twofold "justification process": first, in terms of the ethical position, and, secondly, by appeal(s) to "evidence" to support the position. Even here, however, a complexity arises; namely, the policymaker may choose one alternative because of the availability of evidence and, then, after the fact, justify it by an appeal to an ethical position.

This possibility suggests how a complete understanding of the problem is also one that confronts us with complicated factors relating to the psychological state(s) of the policymaker. Very briefly, some of these might include:

1. A presumption that the policymaker knows how either ethical position may/may not relate to each of the policy alternatives.

2. A presumption that the policymaker can (then) decide on one ethical position for one alternative.

3. A presumption that the policymaker can (then) discount either ethical position for the other alternative, while justifying his/her ethical choice for the chosen alternative by "weighting" it more heavily.

In most cases, it is probably unrealistic to assume such omniscience for policymakers, but such considerations do figure into a complete understanding of such a process. Thus, in Figure 2, the first (1) possibility suggests that the policymaker has (on whatever grounds) decided that for the dilemma under consideration, Alternative A is most susceptible to a consequentialist interpretation, while B is best handled by a deontological position. Now, there is the related difficult problem, again possibly psychological in nature, of knowing the policymaker's a prior preference for either ethical stance, and whether such preference has any bearing on choosing either A or B. Further considerations would include the degree of personal commitment to an ethical position, whether this can be changed in the light of evidence, and the degree of persuasiveness, pro or con, that is provided by others involved in the policymaking process.

We will not pursue these issues but will simply assume that the policymaker is faced with a dilemma, that he or she chooses one alternative, and that he/she does so because of a belief in a certain ethical position that can be "informed" by appeals to "evidence." This relationship is illustrated in Figure 3.

Evidence ------------------ Ethical Position
	Alternative Policy Decision

Figure 3

Now we believe that many policymakers tend to choose some form of consequentialism in decision-making because of some notion that "evidence" is important in making difficult decisions. Therefore, in the following section discussing evidence, we will take the example of a consequentialist vs. consequentialist position, Figure 2 (3). We do this simply as a convenient way of illustrating the complexities of the "evidence condition." Of course, in another sense, a deontological position could be adopted, with "evidence" being the kinds of reasons one puts forth.

IV

As a "case study" of how evidence enters the policymaking process in terms of a consequentialist framework, we will use the recent decision by a major urban university to close its School of Dentistry. While this is an actual event, we have only limited personal knowledge of how (and by whom) the final decision was made. Thus, our "case study" is partially hypothetical, and in the analysis which follows, we will only be concerned with putting further some plausible reasons that a policymaker might utilize for choosing the "closing" option. We shall also try to give reasons for the "not closing" position.

Let us assume the policymaker is confronted with the following dilemma:

Either close the Dental School or do not close the Dental School. If the school is closed it will impact negatively on those associated with it, if the School is not closed it will impact negatively on the University. What should be done?

We will also assume that a decision has not yet been made, that the principal policymaker broadly identifies him or herself as a "Consequentialist," and, as such, wants to examine "evidence" on both sides of the issue. Based on the evidence, the policymaker will then make a decision in which an attempt is made to "maximize" the "best" benefits for the greatest number of people. Furthermore, let us assume that based on his or her own thinking, and the advice of others, the policymaker sets forth some tentative hypothesis for both sides of the issue. We are not concerned here with the origin of such hypotheses, although we do believe this is an important topic as some social- psychological studies have suggested (Hewstone, 1983, Kelley, 1972). A list of plausible hypotheses are given in the following figure.

Dilemma-Related Hypotheses
If we Close the Dental School		If We Do Not Close the Dental School
It will put tenured faculty out of work. It will not permit students to complete their training. It will cause economic hardships not only for faculty but for others working there as well as those providing support services (e.g., food services). It will cause psychological stress among faculty. It will affect the availability of dental care, generally, and for poor people, specifically.		It will increase the University's deficit which will have an impact on other parts of the University--faculty, students, programs, etc.

Figure 4

Now, the assessment of these "hypotheses" in terms of "evidence" must be made, and here a variety of the problems emerge.

"Direct" vs. "Supporting" Evidence

By "direct" evidence, we mean a specific inquiry is carried out to obtain evidence for a claim. For alternative B, this would mean that the administration undertakes a study that would (presumably) support one or both aspects of the hypothesis. The first "aspect" would simply be the presentation of data showing the University's present financial status, and a "justifiable extrapolation" of what it will become if the Dental School is not closed. A "justifiable extrapolation" might consist of some type of econometric model which predicts the impact (i.e., "how much") on the existing deficit (if there is one) and how much it will increase over some time period. The second "aspect" is to show how such an increasing deficit will impact on other sectors of the University, e.g., new faculty additions will have to be "frozen." The evidence is "direct" because it is the result of a "study" carried out specifically on the hypothesis.

On the other hand, "supporting" evidence may consist of an appeal to studies previously done on one or both "aspects" of the above. Now, of course, the supporting evidence idea begs other types of questions: (1) the "closeness of fit" criterion. By this we mean a determination of how "close" the supporting evidence has to be, i.e., should we only "admit" evidence from other studies if such evidence is based on studies "exactly" (i.e., methodologically) like the original?; (2) the "weight of the evidence" criterion. This criterion refers to either the number of studies chosen (as sufficient) for supporting a particular position, or some assessment of the importance of one or more studies within the array chosen as supporting evidence. Here, the total evidence is considered as relevant to the present hypothesis, but some evidence (perhaps on methodological grounds) is deemed to be "better" or "stronger" than the rest, i.e., some type of "ranking" system is appeal to; (3) finally, the evidence believed relevant for a decision, especially supporting evidence, may be "methodologically mixed." By this we mean the policymaker uses evidence produced by a variety of methodological approaches (e.g., quantitative, qualitative, historical) as a means of supporting the particular policy decision. How such choices among evidence are or ought to be made is both a logical and psychological question that cannot be addressed here. Even these simple examples, however, may begin to illustrate the complex relations that came into play when considering the ethics- evidence-policy triad.

Scope of the Hypothesis and Evidence

The number and types of hypotheses in Figure 4 are merely illustrations of many which could be formulated on both sides of the issue. Such hypotheses may be either narrow or broad in scope, and, again, their selection by the policymaker may be a function of psychological, logical, ethical and political factors. Also, as the number of hypotheses for or against a position increases, evidence (of whatever kind) will generally be more difficult to obtain. If such evidence can be attained, and if it is in some sense positively relevant for the chosen position, then it can either be "summed" or "totaled" to strengthen the case for some policy alternative. Alternatively, it may be "ranked" or "prioritized" in such a way that the policymaker selects what she/he considers the "best" evident for the position. Again, such "ranking" could be made on methodological, "political" or other grounds.

In terms of evidence, an interesting case occurs when the scope of a hypothesis is "increased" or "extended" to make a "conjunctive" claim. A conjunctive claim is one in which a desirable or undesirable consequence is "added on" to an original claim. The extended claim may also, then, be given a "causal" interpretation. For example, in Figure 4, under A(2), the original hypothesis is that closing the Dental School will not permit students to finish their training. If, however, another claim is "added on" to this one, we may have something like the following: "Closing the Dental School will not permit students to finish their training and this will result in dramatic increases in stress." Thus, not permitting students to finish their program is (allegedly) sufficient in predicting that some further undesirable (in this case) consequences will follow.

Whether or not such an extended claim is "true" will be a function of what type(s) of evidence can be brought forth to assess it. The point, however, is that extending the original hypothesis involves the policymaker in a more complex evidence- gathering task. Now, not only must she/he determine the viability of the original hypothesis, but also its (presumed) connection with "stress" must be assessed. Methodologically, this involves more decisions concerning sampling, instrumentation, testing and interpretation. Likewise, from the consequentialist position we are adopting, there is a shift in focus: we are now assessing the (presumed) negative consequences not by students' inability to finish their programs, but by the "stress" levels they will (presumably) exhibit. Thus, a "structural" consideration is claimed to lead to a "psychological" one, and it is this latter consideration that is used to evaluate "consequences." There are, additionally, a multitude of other questions here; we will list only a few of them:

1. If the "extended" hypothesis is "supported," will this be sufficient grounds for the policymaker? Will other "related" evidence need to be considered (i.e., "supporting" evidence) to bolster the case? Is such evidence available? If so, can it be critically appraised for relevance, etc.?

2. By way of "methodological strategies," should the extended hypothesis be studied quantitatively (empirically) or qualitatively (ethnographically)? What criteria should be used to make this decision?

3. If this extended hypothesis is supported, should other hypotheses (in A) be tested? How should this decision be made?

4. If the extended hypothesis is "strongly" supported, is this sufficient? For example, let's say "high" stress levels are found on the average for the dental students, but the levels are not evenly distributed across the students. That is, suppose minority dental students experience very high stress levels, but their numerical representation is small compared to the entire sample. Should this "finding" be given additional "weight?" More generally, how many (and on what grounds) other "mediating" or "intervening" factors ought to be taken into consideration?

5. What are the limits of possibly relevant "casual claims" that must be considered? That is, even if high levels of stress are documented is this sufficient evidence, or do we need to postulate that the stress, in turn, will lead to some other undesirable consequence, e.g., mental and/or emotional problems?

6. Whatever positive support is found for the extended hypothesis (or for that matter the original one) how should it be compared to the hypothesis in B?

7. Finally, if different methodological approaches are used between A and B, which "counts" more decisively? And why?

Summary

All of the "presentations" of the evidence-ethics-policy triad we have mentioned so far could be extended in many other ways. What we have been trying to point out, however, are the complex issues that arise not only in the triadic relationship above, when viewed as a totality, but especially in terms of the seemingly simple concept of "evidence." The concept of evidence is complex on many levels: (1) the ambiguities associated with trying to define it, (2) the different forms, types or varieties of evidence which are generated by different methodological approaches; (3) the "rules" (or lack of them) relating evidence to a claim; and, (4) the overarching epistemological (and many times controversial) issues concerning "truth," "evidential adequacy," "empirical justification," and so on.

In addition to these difficult matters, there are other "levels" that "overlap" in equally complex ways. Thus, for the policymaker confronted with a policy-related dilemma and who chooses to resort to some notion of "empirical evidence," there are seldom any clear ("justified") criteria that she/he can appeal to. If, for instance, the policymaker is philosophically trained and/or knowledgeable about the epistemological issues that bear on the role of evidence to "epistemic justification," such considerations - even though philosophically "correct" - are usually irrelevant to the policy decision that must be made. Their "irrelevance" lies in the fact that they are too far removed from the issue at hand and/or do not provide for pragmatic applications by way of clear-cut criteria for decision- making choices.

There are also complex interplays between the "psychology" of the policymaker and his/her understanding of the nature of evidence. Some policymakers, mostly because of their training in social science research methods, view the concept of evidence as that of "findings" produced by empirical studies; the findings, of course, should be "statistically significant!" Usually, these policymakers do not understand the epistemological issues surrounding the "evidence condition", nor the more technical statistical-mathematical assumptions involved in empirical research. Based on our own experiences and readings, policymakers usually have some vague notions about what constitutes "good" or "adequate" evidence. Other "psychological" factors in the policymaking process concern "selective attention" to only certain types of evidence, the choice of believability of "experts" usually utilized by policymakers, accommodations or allegiances to other vested interests, self-serving motives, etc. Thus, while a policymaker may have some personal belief in what, for instance, constitutes a "truth-and-evidence" relationship, such considerations are often over-ridden by the "political" reality of the policymaking process.

Now, the ambiguities associated with the concept of evidence and the "psychological" factors associated with policymaking intertwine (at least hypothetically) with the ethical dimension. Although we doubt most policymakers are aware of ethical theories, let alone how they may figure into approaching and "resolving" policy related dilemmas, if such considerations are taken into account they raise a host of further overlapping and complex issues. Since most policymakers are "pragmatic- realists", they could be classified in a very rough way as belonging in the Consequentialist "camp." That is, their primary concern is determining how a policy choice will result in producing the "best" outcomes for the "most" people. And being committed (at least implicitly) to this position, they are in need of supporting "evidence" to substantiate their particular choice/decision. (Parenthetically, we have not found any examples where a policymaker appeals to a deontological position in justifying a policy choice.)

Given these considerations, the central question becomes one of choosing the "best" evidence for a position within the constraints of one or more hypotheses. The policymaker needs to make at least a minimal case that the evidence chosen is "positively relevant" to the issue at hand. However, this is no simple matter if the evidence comes from different methodological approaches, if some of it is contradictory, or if it is based on "supporting" studies which are only marginally relevant to the issue. There is also the issue of only selecting evidence which supports a favored point-of-view. This last strategy may, however, be justified by the policymaker because the evidence chosen may be relevant, there may be "more" of it than the contradictory evidence, and the contradictory evidence is "judged" to be flawed in some way. For example, the policymaker may choose the findings of an empirical (quantitative) study over the findings of an ethnographic (qualitative) one because the latter is judged not to be (methodologically) "scientific."

Thus, the "moral" of this story, if there is one, is that the evidence-ethics-policy triad is not only quite complex but somewhat paradoxical; "paradoxical" in the sense that:

1. It is commonly believed that "evidence" is a necessary condition to make rational and informed decisions concerning important policy matters; however, once we probe more deeply into what "evidence" is the more difficult it becomes to apply evidence to policy choices.

2. If policy decisions are believed to be grounded (ultimately?) in ethical positions, and if the ethical position chosen is some form of Consequentialism, and if "good" or "desirable" consequences are believed to accrue if some policy is chosen, then there is a need to present evidence that particular consequences will most likely (or do) follow. But here, again, it may be very difficult to justify the decisions about evidence that are eventually made.

3. A similar situation holds if evidence is believed to be the most "rational" way to arbitrate between policy- dilemmas. The "evidence" chosen for supporting one "side" of the dilemma is often difficult to completely justify.

4. Epistemological theories concerning the knowledge- evidence-justification triad are often logically persuasive but seldom practically relevant to the policymaker. (One problem here is that the epistemologist sees "evidence" as a strictly logical relationship or condition that must be met in saying that propositional (i.e., knowledge) claims can be justifiably held; however, the many semantic issues (and some logical ones - e.g., those obtaining between/among different ["mixed"] evidence-instances) are often not adequately addressed).

5. If, in a policy-related dilemma, a "neutral" policymaker is given "good" empirical evidence in favor of one "side" of the dilemma, and "good" deontological "evidence" (i.e., some "good" rule-based reason(s) for the other "side" of the dilemma, is there some "rational" way to choose between the two?

Was, then, the decision to close the Dental School a "good" or "rational" one? Since we cannot reconstruct the events that went into making the decision, we cannot tell. Our guess would be that some form of Consequentialism (at least implicitly) was used as a "rationale" for the decision. It would, of course, be very interesting to know what (if any) "evidence" was used to make the decision; how such evidence was gathered; how evaluated; and how "justified?" Our final guess would be that the decision used only a "minimal evidence" base and that it was "positively relevant" but "selective" evidence.

Of course, these speculations only reinforce our belief that the "triadic" relationship is highly complex and not fully understood. Its lack of understanding underscores the need for in-depth empirical studies, which, if undertaken, may, however, (and ironically) only bring to fore the very problems mentioned throughout. Nevertheless, we presently simply do not have a clear view of how, if, and to what extent, ethical judgements enter into the policymaking process. Our own inclination is to believe, borrowing an analogy from the language of Confirmation Theory and the philosophy of science, that ethical thinking among policymakers serves only as "weak background evidence." That is, policymakers may view their own ethical beliefs as relevant to the decision(s) they make, but only in a very general, implicit or diffuse way.

A part of this attitude may be due to the lack of in-depth training in ethical theory among policymakers, but a portion can also be attributed to such factors as the immediacy of the situation, cross-pressures brought to bear by other "invested" parties, and the need to arrive at some type of "pragmatic" compromise. In a parallel fashion, policymakers who appeal to some type of evidence do so primarily as a means of supporting a favored position. The methodological adequacy of this evidence is simply assumed, although "negative" evidence may (then) be criticized on its methodological "inadequacy." In any event, however, policymakers are not overly concerned with either the technical aspects of differing methodological approaches nor with the broader (and philosophically technical) epistemological issues of justification and evidence. Again, as with ethical theories, many policymakers do not have adequate training in these areas, but the over-riding concern is with the formation of the specific policy: these other concerns are viewed as academically important but practically remote.

Of course, policymakers are seldom pressed, individually or professionally, to "justify" their decisions in terms of articulating in-depth ethical and/or epistemological positions related to the policy. Depending on the policymaker, such deliberations, if they are made at all, are left to a later time, when one leaves "office" and has time for "reflection." Thus, we are left with the interesting and ironic conclusion that what "ought" to be the most central in policymaking, ethical and evidential reasoning, becomes the most peripheral.

References

Achinstein, Peter (Ed.). The Concept of Evidence. Oxford: Oxford University Press, 1983.

Barnes, B. and Bloor, D. "Relativism, Rationalism and the Sociology of Knowledge." In M. Hollis and S. Lukes (Eds.), Rationality and Relativism. Oxford: Blackwell, 1982, 21-47.

Bloor, D. "The Strengths of the Strong Programme," Philosophy of the Social Sciences(11), 1981: 199-213.

BonJour, L. The Structure of Empirical Knowledge. Harvard: Harvard University Press, 1985.

Callahan, D. and Jennings, B. (eds.). Ethics, The Social Sciences and Policy Analysis. NY: Plenum, 1983.

Habermas, J. Communication and the Evolution of Society, Tran. Thomas McCarthy. Boston: Beacon Press, 1979.

Hempel, Carol. Aspects of Scientific Explanation. New York: Free Press, 1965.

Hewstone, M. Attribution Theory: Social and Functional Extensions. Oxford: Basil Blackwell, 1983.

Kelly, H.H. Causal schemata in the attribution process. New York: General Learning Press, 1972.

Kohlberg, L. "Moral Stages and Moralization," in T. Lackona (Ed.), Moral Development and Behavior. New York: Rinehart and Winston, 1976, 31-53.

Kuhn, T.S. The Structure of Scientific Revolutions, 2nd ed. Chicago: University of Chicago Press, 1970.

Marcuse, H. One-dimensional man. Boston: Beacon Press, 1964.

Miller, Steven I. "Some Notes on the Nature of Methodological Indeterminacy," Synthese (88) No. 3, 1991: 359-378.

Miller, Steven I. "Some Comments on the Projectibility of Anthropological Hypotheses: Samoa Briefly Revisited," Erkenntnis (30) 1989: 279-299.

Miller, Steven I. and Fredericks, Janet. "Clarifying the adequate evidence condition in educational issues and research," Educational Theory (42) 1992: 461-472.

Moser, Paul K. Knowledge and Evidence. Cambridge: Cambridge University Press, 1989.

Moser, Paul K. Empirical Justification. Boston: D. Reidel Publishing Co., 1985.

Piaget, J. The Moral Judgement of the Child. New York: Random House, 1965.

Rawls, J. A Theory of Justice. Harvard: Harvard University Press, 1971.

Simpson, E. "Moral Development Research: A Case of Scientific Cultural Bias," Human Development (17) 1974: 81-106.

Strike, K.A. and J.F. Soltis. The Ethics of Teaching. New York: Teachers College Press, 1985.

Sullivan, E. A Study of Kohlberg's Structural Theory of Moral Development: A Critique of Liberal Social Science Ideology. Toronto: Ontario Institute for Studies in Education, 1977.

Wilson, R.W. "Some Comments on Stage Theories of Moral Development," Journal of Moral Education, 1976: 241-248.

Winch, P. "Understanding a Primitive Society." American Philosophical Quarterly 1, 1964: 307-24.

About the Authors

Steven I. Miller
L. Arthur Safer

School of Education
Loyola University Chicago
820 North Michigan Avenue
Chicago, Illinois 60611

---

Copyright 1993 by the Education Policy Analysis Archives

EPAA can be accessed either by visiting one of its several archived forms or by subscribing to the LISTSERV known as EPAA at LISTSERV@asu.edu. (To subscribe, send an email letter to LISTSERV@asu.edu whose sole contents are SUB EPAA your-name.) As articles are published by the Archives, they are sent immediately to the EPAA subscribers and simultaneously archived in three forms. Articles are archived on EPAA as individual files under the name of the author and the Volume and article number. For example, the article by Stephen Kemmis in Volume 1, Number 1 of the Archives can be retrieved by sending an e-mail letter to LISTSERV@asu.edu and making the single line in the letter read GET KEMMIS V1N1 F=MAIL. For a table of contents of the entire ARCHIVES, send the following e-mail message to LISTSERV@asu.edu: INDEX EPAA F=MAIL, that is, send an e-mail letter and make its single line read INDEX EPAA F=MAIL.

The World Wide Web address for the Education Policy Analysis Archives is http://olam.ed.asu.edu/epaa

Education Policy Analysis Archives are "gophered" at olam.ed.asu.edu

To receive a publication guide for submitting articles, see the EPAA World Wide Web site or send an e-mail letter to LISTSERV@asu.edu and include the single line GET EPAA PUBGUIDE F=MAIL. It will be sent to you by return e-mail. General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, Glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-2411. (602-965-2692)

Editorial Board

John Covaleskie Syracuse University	Andrew Coulson
Alan Davis University of Colorado--Denver	Mark E. Fetler mfetler@ctc.ca.gov
Thomas F. Green Syracuse University tfgreen@mailbox.syr.edu	Alison I. Griffith agriffith@edu.yorku.ca
Arlen Gullickson gullickson@gw.wmich.edu	Ernest R. House ernie.house@colorado.edu
Aimee Howley ess016@marshall.wvnet.edu	Craig B. Howley u56e3@wvnvm.bitnet
William Hunter hunter@acs.ucalgary.ca	Richard M. Jaeger rmjaeger@iris.uncg.edu
Benjamin Levin levin@ccu.umanitoba.ca	Thomas Mauhs-Pugh thomas.mauhs-pugh@dartmouth.edu
Dewayne Matthews dm@wiche.edu	Mary P. McKeown iadmpm@asuvm.inre.asu.edu
Les McLean lmclean@oise.on.ca	Susan Bobbitt Nolen sunolen@u.washington.edu
Anne L. Pemberton apembert@pen.k12.va.us	Hugh G. Petrie prohugh@ubvms.cc.buffalo.edu
Richard C. Richardson richard.richardson@asu.edu	Anthony G. Rud Jr. rud@purdue.edu
Dennis Sayers dmsayers@ucdavis.edu	Jay Scribner jayscrib@tenet.edu
Robert Stonehill rstonehi@inet.ed.gov	Robert T. Stout aorxs@asuvm.inre.asu.edu