Introduction

To better understand  the complex and diffuse relationship that exists between social policy and social science research, I present the reflections of twenty-nine Canadian policy officials on access to scholarly research afforded by the Internet. While researchers interested in the political impact of the Internet have tended to focus on "digital democracy" issues of public access to government information, public consultation and participation, and public privacy and surveillance, a far less dramatic change is also taking place to existing policy-making processes. (Note 1) The knowledge economy of the Internet has significantly increased policymakers’ ability to tap into current, critical, and relevant research without leaving their desk. Although many academic journals remain closed to those without access to a good research library, a growing number of individual papers, journals, and research archives are providing full online access at no charge to users. (Note 2) This new availability has transformed the information environment within which policy development takes place, and it is altering the role of social scientists in democratic processes.

Admittedly, the influence of academic research on government policy pales before the impact of the government’s own sources of information, whether from politicians, bureaucrats, or government professionals. As Harvard’s Carol Weiss has observed, it takes "an extraordinary concatenation of circumstances" for "research to influence policy decisions directly" (1991, p. 44). Weiss has concluded, after many years of evaluation work in education, that "governments don't often use research directly, but research helps people reconsider issues, it helps them think differently, it helps them reconceptualize what the problem is and how prevalent it is, it helps them discard some old assumptions, it punctures old myths. It takes time and reconceptualization before research actually leads to a change in policy" (1998). Still, one finds, for example, that a recent U.S. federal education bill calls on school officials who seek federal support to secure "scientific evidence" in arriving at program decisions, as well as related initiatives to improve federal spending on education research. (Note 3) Similarly, the British government’s recent Hillage report calls for education research to play a greater part in policy formation, and to take steps to better prepare itself to play this role (Hillage 1998). Also, the University of London’s Institute for Education has established a Centre for Evidence-Informed Policy and Practice, which seeks, among other things, to make research more accessible for policymakers and the public, as well as to support a more coordinated, less fragmented, approach to education research. (Note 4)

Given the complex and exhaustive knowledge needs of the modern democratic state, academic research has a vital, irreplaceable role to play in not only informing and evaluating government decisions, programs and policies, but also in a broader intellectual sense, whether by challenging assumptions, providing a historical perspective, bringing international comparisons to bear, or offering lessons from experimental models and alternative conceptions. (Note 5) After all, the classic administrative problem, identified by Herbert Simon more than a half-century ago, is the "bounded" rationality of bureaucrats, always limited in their knowledge of the situation, the issues, and the available alternatives (1945). Although this dilemma led Simon to work on artificial intelligence devices that would supplement these limits, it may be that improved and timely access to relevant research can expand the rationality of the policy process. (Note 6)

I would also note, before presenting the policymakers’ reflections on online sources of social science research, that this focus on research-for-policy has been perceived in Great Britain as a threat to academic freedom, as well as, in the case of education research, to the professionalism of teachers (Ozga, 2000; Humes and Bryce, 2001). The increased influence of research on government policy also concerns American political scientists Anne Larson Schneider and Helen Ingram (1997), as this increase has done little to counter widespread public disillusionment with the U.S. government since the 1960s. In their view, the growing influence of "scientific and professional perspectives" can further alienate citizens, with the complexity of their approaches and exclusionary jargon. This use of science may seem to increase the rule of reason at the expense of political maneuvering, but its inaccessibility can further shrink the public sphere and public deliberations, as research reduces social issues to technical ones that can be resolved through expert advice and "best practices" (1997, p. 153). (Note 7)

For Schneider and Ingram, science has become a tax-supported "establishment" of limited accountability, which can leave the public wondering why such work needs public support. They identify the National Institutes of Health as an example of scientists securing "overgenerous funding" benefiting "advantaged populations," in their estimation, as well as scientists and drug companies (1997, p. 164). (Note 8) They are also concerned with the considerable number of scientists employed by governments, as this can reinforce the insular power of the bureaucracy. (Note 9) The obvious point of caution is that policy officials, politicians, and interest groups can selectively represent research studies, if not selectively misinterpret the results (Barker and Peters, 1993). For Schneider and Ingram, what the policy process is missing is a democratic commitment to "re-energize people and create an educated, enlightened active citizenship" (1997, p. 7). Their concern harkens back to the democratic divide between John Dewey, who believed in fostering and relying on an increasingly informed public who have the "the ability to judge the bearing of knowledge supplied by others upon common concerns" (1926, p. 209), and Walter Lippmann (1922), who held that a complex democratic state was best governed through "expert mediation" and "organized intelligence." (Note 10)

My own efforts, very much on the side of Dewey, have been to explore ways of improving access to social science research for the benefit of both public and policy officials. The Public Knowledge Project, with which I work, seeks to improve not only the scholarly quality of research, but what I would refer to as its "public quality," in the sense of enhancing its public access and intelligibility. (Note 11) As a foundation for this work, the Project studies the ways in which policymakers, the press, practitioners, parents, and the public, as well as faculty and students, utilize knowledge from online sources. On the basis of this understanding, we are pursuing better designs and structures for organizing, presenting, and integrating research with related resources, such as existing policy initiatives (Willinsky and Wolfson, 2001).

The study presented here focuses on the current levels of engagement with academic research among policy officials. It is intended to serve as a baseline for current usage, as well as to identify a number of key issues that shape the impact that this knowledge has on the development of policy. My argument is that we have an opportunity and  responsibility within the social sciences to increase the public and scholarly value of our work. New online publishing technologies, as they increase access, can do more to inform and expand deliberations among people and policymakers. This study asks policymakers about how this new medium is assisting or impeding their use of this research. The resulting interviews suggest a number of steps—including both the labeling and context provided for the research—that would vastly improve the organization, and thus the value, of policy-relevant research online for both public and policymakers. The policy officials in this study were keen to engage with online research,  and it appears that the impact on their work would only increase as access and organization are improved. Otherwise, too much of what we know about matters of concern to government policy goes unheard and unseen by those in a position to do something with it, as well as by those who suffer the consequences of this ignorance.

Method

We interviewed 29 civil servants, policy analysts, research officers, and librarians, drawn from the Canadian federal and provincial governments (Table 1). The government departments and divisions that agreed to participate in this research project selected areas for us to conduct interviews in which research might well come into play. The officials (with only two women among the sample) were often referred to us by their superiors, as we explained the nature of our work, as people who were involved in the research and information gathering aspects of their government agency. The Policy Research Initiative, for example, had divisions concerned with Knowledge Integration, as well Social Cohesion and Sustainable Development, suggesting a growing interest in the sort of long-term and broad conceptualizations that the social science research community addresses. There were more prosaic, but nonetheless vital, areas included, such as Employment Insurance and Monetary Analysis.

The questions posed to the policy officials covered three areas of interest, beginning with their "general regard for scholarship and research," including the role it plays in their work and their means of access to it (Appendix 1). The interview went on to consider their use of "electronically available research," including search strategies, favorite sources, and challenges and foreseeable changes. It concluded by asking them to consider two designs for organizing research: one was a prototype site we had previously developed, Policy.ca, and the other a model for providing individual studies with a context. (Note 12) Some subjects declined the offer of anonymity in this research, and their names are used in the analysis that follows, which has been grouped by the nature of the government organization, that is, by government department, agency, research initiative, and library.

Policy Analysts in Government Departments

The mission of Human Resources Development Canada is "to enable Canadians to participate fully in the workplace and the community." (Note 13) In this department, we spoke with an "employment insurance" policy analyst, who described seeking a "diagnositque" from his reading of researchers, stakeholders, trend analyses, and developments in the field, such as a corporation that had a great day-care policy. He was a regular user of government information sites, such as Statistics Canada, but he also found himself often enough facing perplexing Google searches that produced thousands of hits on a topic. Still he admires the speed of information retrieval online, once the wheat has been separated from the vast amounts of chaff: "What we have done is complement the telephone and fax, and what we've got now is much faster." He also made a most interesting case for hard copy by referring to "the cognitive processes necessary for understanding the material" which were assisted by the ability to "spread it out on a desktop to compare it." What also has not changed is that his department still tends to work with the "major players" in the research field, using them to ensure the quality of information as well as funnels and filters for additional research, which may well limit a policy analyst’s reach for fresh perspectives. Still, he looks forward to when "more sophisticated [research] portals are introduced" which will provide access to both overviews and full-text studies. He felt that the universities should make an effort to match the Government On-Line (GOL) initiative with a similar approach to research.

In the Human Resource Partnerships Directorate of this same department, we interviewed two senior policy advisors, Silvano Tocchi and Nicholas Wise, who both work with corporations to improve employment opportunities. They, too, rely on an academic network to filter and serve as a conduit for research. They also use it as a "sounding board" for ideas, suggesting a more interactive policy involvement for researchers, if a rather select, handpicked group. Tocchi and Wise see their own work as immersed in social science research, with their own reports and internal research adhering to academic standards. Given the increasing availability of relevant information, including research, they find themselves playing a mediating role with research in their division, or as Wise explains, "sometimes [there is] a need to ‘tailor’ information in a manner that is more appealing or better understood within the department—packaging it in ways that make it meaningful." They feel it important to browse recent research, without always going after a specific topic, and to that end they subscribe to a number of services that supply a weekly compendium of abstracts or summaries. They were concerned, finally, that efforts to improve information systems would provide links to research carried on outside of the universities, whether to private research companies or to similar departments in other jurisdictions.

With the Social Policy Directorate of Human Resources Development Canada, we held a small seminar with four of its members. This group provides both quantitative and qualitative research support for policy development, especially as a starting point for new initiatives, that also identifies gaps in what is known. Despite the gains through Internet access, they felt that much relevant university research is locked away; they want better systems, greater organization of materials, and more open access to "the real ‘meaty’ and interesting material" as opposed to available summaries and overviews. Difficulties with getting at this research only meant, they pointed out, that think tank and interest group reports can exert greater influence through their accessibility: "Often you may have only an afternoon to come up with the points you need, so you are going to hit the key groups you know are involved in this area, find the pieces there, and not go any further." Yet they felt that overall the Internet’s increased access to research has added to the division’s general level of engagement with research: "People are now doing much more of their own research, whereas before there was a select few people or they went to the library and asked a librarian to find information for them. It is now much easier for an individual to do it themselves."

The Department of Justice was the site of an interview with two Research Officers working on issues of cyber-crime. They also followed the traditional pattern of working with a select set of academics in reading their research, consulting with them personally, and inviting them to roundtables. The Internet, however, had enabled them to expand the group they consulted. Still, they shared concerns with others about the overwhelming amount of information on the Web, as well as about the veracity of what is found there: "Although you save time in retrieving the document, you must spend it trying to verify the authenticity of certain material." To help other government officials deal with this issue, they publish "Just Research," which provides a guide to valuable websites for Department of Justice employees. They also expressed a concern that universities do more than just put their work out there: "We need greater ease of connection to places such as university research sites (possibly through a portal or channel), without the frustration of everyone finding the sites always on their own. Then it will be more like going to a traditional library catalogue, where someone is keeping these links up-to-date." This interest in a centralized site included gaining access to studies going back some twenty years, to permit a historical perspective of the very study of justice issues. They were also very articulate about the need for integration among different types of academic literature, from conference papers to meta-analyses, as well as wondering about copyright issues and access to "research data, instruments, and methods" which might be of interest. They were clearly ready for a far more sophisticated approach to online access to research than is currently available.

In the Department of Foreign Affairs and International Trade, we spoke with an International Drug Officer responsible for international comparative analyses of drug policies. The Internet now dominates his information seeking activities. He quickly pointed out that he has no shortage of information on drugs, which only meant that ascertaining "good and credible information" was his biggest challenge. He wondered if having third-party authenticators or guarantors of quality would not resolve this problem. He certainly appreciated the ability to corroborate studies and crosscheck research claims using the Internet, although it was not always possible to do so. The Senior Policy Advisor whom we interviewed from this department also spoke of his increasing dependence on the Web, with up to 80 percent of his research, he estimated, conducted there. He had a real interest in seeing complete studies rather than just abstracts, although he found pay-per-view services for full-texts an economic roadblock. Still, he also reminded us that policies are far more informed by classified information—from government and intelligence sources—as well as by public resources posted by foundations and other organizations, in part because academics seem to lag behind, in his opinion, in placing information on the web. He felt that research needed a context—especially with access to other works by an author—to be judged useful and reliable: "Material and analysis outside of the study itself is useful; and can, perhaps, guide one’s own interpretation, for example, a legal decision is much more understandable when the commentaries are reviewed."

In Agriculture Canada, we held two sets of interviews, the first interview with two policy analysts from the Strategic Policy Branch, which is concerned with farm economics. The two posed an interesting contrast to each other. One of them relied on a small set of sites, largely government run, such as StatsCan, and worked with a small circle of researchers: "I usually know the academics that are writing in my area." The other analyst browsed widely, constantly in search of new developments. He believed in the "serendipity" factor, using the example of how he had come across an Australian agricultural initiative that has now led to a collaborative effort between the Canada and Australia. Still, for both of them, the Internet had become their main source; the one could not remember the last time he had been to the departmental library in the basement and the other had been led there by online resources. They were interested in how the Internet could provide some historical depth to their review of research, enabling a review of some thirty years of research. In considering the models for improving the organization of research that we have been developing, they wondered about gaining access to Master’s and Ph.D. theses ("which we are seldom aware of") and in being able to establish a context for research—"Reports on reports (reviews), contra views, and summaries are often more revealing than the actual studies." They were also interested in "some form of ‘certification,’" as well as knowing who funded the research, as a study "funded 50% by Monsanto does suggest caution and skepticism when reviewing the results."

The other two policy analysts whom we interviewed from Agriculture Canada were from Policy Planning. Here we found real concerns with the failure of research to address the current political issues in agriculture, often forcing them to rely on the work of other policy analysts, whether here or abroad. They were concerned that the resulting absence of information and data has meant that "future policy solutions are more constrained by where we are, where we have been, and our current domestic situation." They would obviously like to have a greater array of perspectives come into play in policy development. As it is, they tend to stay with a small Canadian academic community—maybe 25-30 professors—who are conducting relevant research for their area, noting that while they come across others, it requires checking "what else they have published" and the credibility of their institution—"If Harvard has hired them...." They were also sensitive to how the use of research in policy settings invoked issues of jargon and translation—"translating academic work into a language our executives understand is my job." Yet there are areas were they feel well served by the Web for developing their professional skills: "I use the Internet more for locating management and strategic thinking material, than policy material, i.e., Kennedy School of Government, Harvard, Stanford, or universities with good management schools, I know who the significant management writers are, and then see what they have made available on the site."

Among the policy officials working in their four government departments, the Internet has taken hold as a primary information source, offering ease and speed, while posing challenges of extracting value and establishing credibility. There is clearly room for social science research to play a greater part in the policy process, for expanding the range of ideas considered and opening up the government’s thinking about possibilities that extend beyond the closed traditional circles of academics.

Policy Analysts in Government Agencies

We worked with two government agencies, the Bank of Canada and the Canadian International Development Agency that are very involved in policy issues, if at one remove from the legislative forum of Parliament. At the Bank of Canada, we spoke with three men involved in monetary modeling, applied studies, and policy research. In each case, they told us, research is what they do. They are economists who both produce reports for the Bank and publish research in academic journals. They pointed out that economics is one academic discipline that has a central open-access repository, RePEc (Research Papers in Economics) that provides free and centralized access to an enormous number of articles, reports and working papers. They see RePEc as a real advantage to their work as researchers, just as they appreciate having electronic access to complete sets of economic journals online, running back to the turn of the century, through J-STOR. (Note 14) Still, they sometimes feel frustrated by disciplinary boundaries that separate other relevant areas, such as biology, mathematics, and statistics, from this bounty of economic literature. And while they cannot imagine working now without the Internet, they would like to see greater consolidation of access and indexing, as well as more full-text resources and citation linking, as the search for research resources still seems unnecessarily difficult. Their opinions speak to the technology’s ability to foster a greater demand for its powers than it can, at any given moment, meet.

Caroline Caza is a Senior Environmental Policy Advisor, with the Canadian International Development Agency, which is devoted to supporting "sustainable development activities in order to reduce poverty and to contribute to a more secure, equitable and prosperous world." (Note 15) Although she does not see herself as a regular consumer of research, Caza’s impression is that academic research does influence the formation of policy, observing that academics are hired on occasion to review the literature and present feasible policy perspectives. She pointed out that within her own agency the press was on to create a "knowledge-based organization," which would increase its capacity to create effective policies. In her own work, Caza prefers reviews of current research which she finds "much more useful forms of information," while individual studies can be "too long" and "too narrow to be able to synthesize the diversity of opinion you that need in order to make a credible policy decision." Yet she also felt that what was frustrating about some information rich websites was "the lack of depth and breadth in, for example, Policy.ca" (referring to the Public Knowledge Project site). And while senior policy officials are always asking for quality information, they may not be in a position to judge that quality themselves. For her part, she has let her subscriptions to academic journals, acquired during her university days, lapse as they failed to serve in her work: "They didn't allow me to be credible by bringing new scientific information to a forum in a way I could use effectively." The exception was Conversation Biology which Caza continued to receive for its Letters to the Editors, which she felt presented leading-edge issues and any controversy surrounding them. So while she was turning from print sources, she was still seeking the ideal information websites which offered authentic and credible information, in what was an all too common theme among policy officials in our sample.

Government Research Initiatives

We spoke with representatives from two of the Government of Canada’s research agencies: the Policy Research Initiative and the Canadian Health Services Research Foundation. The Policy Research Initiative was established "to strengthen the federal government's policy research capacity" and sees itself as "a catalyst for the development of knowledge, people and community," according to its website (original emphasis). (Note 16) Here, the government is fully engaged in bringing research to bear on areas of Canadian policy. It publishes Isuma: Canadian Journal of Policy Research, which features an editorial board made up of academics (although one PRI representative we spoke with felt it was "not as in-depth as a traditional academic journal"). The journal offers an open-access electronic version as well as a print run that is distributed "free to a large cross section of Canada’s policy research community." The Policy Research Initiative also publishes Horizons, "a snapshot of emerging, cross-cutting research in the Canadian and international policy environments." It also has a Policy Research Data Group that works with other departments facilitating "the development of databases required to carry out research in priority horizontal policy areas."

Within the Policy Research Initiative is the Knowledge Integration Project which is directly involved in exploring new approaches to managing and accessing scholarly research, including the use of emerging standards for document management systems, such as Open Archives Initiative, Government Information Locator Service, and Dublin Core Metadata Initiative. The representative of the Knowledge Integration Project with whom we spoke saw his role within the policy community as developing strategies for the long-term acquisition and storage of "knowledge objects," reflecting a government commitment to using new technologies to increase the play of research and other sources of information in the policy process. His concern that it was often difficult to locate and work with the data sets behind major research studies suggested the need for developing far more comprehensive scholarly communication systems that would enable independent and critical analysis of research, as well as greater collaboration and reanalysis of data. As an experienced research reader, he was less than happy with how research was currently made available, both economically and structurally: "Until we define or restructure the ‘Elsevier’ economic business model for publishing, we are going to be stuck with very high transaction costs for finding information.... We need both a common format standard and a common conceptual standard." The inhibiting factor, in his eyes, is "the transaction cost of searching on the Internet" which he finds "phenomenal." And while he tended to use materials that were only one or two years old, he also saw a place for keeping a "500-year time frame" in addressing major policy issues, pointing to an interest in both historical research, as well as research from earlier periods, as did others in our sample. Finally, this knowledge integration officer expressed a strong interest in a quality indicator or systems of endorsement that went beyond basic peer review, of the sort perhaps, used in bibliometrics to evaluate the "impact factor" of a journal (by how often its articles are cited).

A second representative of the Policy Research Initiative, this time from the North American Linkages division, was very clear about the foundation that research provides for policy. She felt that it did fall to the government to strengthen scholarly research in areas of particular policy interest. The government could then draw on these researchers to build an advisory group who could be counted on to keep the government informed and ensure the quality of the resulting policies. In terms of her reliance on the Internet as a source of research, she pointed out, as did others, that assessing the quality of the research is as much an issue as accessing it: "You have to exercise a considerable amount of independent judgment about the veracity of what you are reading." She believes that researchers in her divisions are increasingly relying on the Internet—where the quality of information is still difficult to ascertain—when they should be seeking out the known quantities in print sources. Her worry is that the Internet leaves readers with a misleading sense of having consulted, or at least browsed, all that there is to know in a particular field. Otherwise, it could mean that the Internet, at this point at least, may represent a shift in the focus of attention on research—to what is readily available online—with no real expansion in the consideration of what is known and no guarantee as to the quality of that knowledge.

The Policy Research Initiative representative from the Sustainable Development group, Paul Halucha, offered a glimpse of a still very powerful method of working directly with a small number of researchers to directly address the issue at hand. He described how he brought together academics to identify eight relevant trends in research on sustainability which resulted in an "interdisciplinary knowledge statement." He saw this process offering a "single window" into the government which researchers could use to bring  their work to bear on policy. Yet it was exactly that, a single window, and not an open door. It enabled direct engagement between scholar and policy official, but it was more of a closed process, and needs to be supplemented at least, by more open processes. It was, to be fair, his way of dealing directly with the knowledge demands of his work: "Government officials are faced continually with the daily pressures of producing knowledge." He felt the need for better ways of managing the deluge, as "information...is piling up...and there comes a point at which one gets into declining returns." He also worried about how the Internet does not afford access to older materials, which would diminish "one of the classic public policy values" of "deliberation, rational deliberation." The challenge, then, is to devise systems that do not leave one reverting to narrower, traditional processes of consultation because the new is simply too much, too quickly.

A third representative of the Policy Research Initiative with whom we spoke was Michael Mackinnon, from the Social Cohesion Team. Mackinnon spoke of how the concept of "social cohesion"—which is concerned with the degree of people’s economic, social and political participation in a society—was derived from European sociological research, which had been recently popularized on this continent by Robert Putnam’s work (2001). Mackinnon spoke of the need for better organized websites with taxonomic structures and tagged materials, as well as for research written in comprehensible language, with some way of verifying its status. He also pointed to the value of portal sites that served as well organized gateways to a wide range of resources, commending Policy.ca, as well as Canadian Social Research Links, which is run as a hobby by Gilles Séguin, a Human Resources Development Canada employee. (Note 17) While often satisfied with being able to read research abstracts, Mackinnon also showed an interest in  having the opportunity to go deep and review the actual data sets behind studies, as well as in being able to consult summaries, overviews and syntheses. While all of this would be "of significant interest to policy analysts," he wondered who could afford to produce this level of research support, as an add-on to existing systems, while what may be needed is a new system in which summaries, portals, and taxonomies are all part of the fundamental design of scholarly communication.

The Canadian Health Services Research Foundation is a non-profit organization endowed by the federal government to support both basic and practitioner research on management and policy issues in the area of health services and nursing. (Note 18) It also seeks to bring together decision-makers, policymakers, practitioners and researchers to facilitate the dissemination of this research. We spoke with Michelle Campbell, Assistant Director, Knowledge Transfer Division, who works to ensure that the available knowledge gets to the policy researchers, decision-makers, and administrators in Canada’s health systems. As part of this job, her group does translate research on occasion into common language, on the one hand, and has hired a "knowledge broker," on the other hand, to work with academics. She has learned through this work that while researchers feel they can draw no conclusions when they have only 20 per cent of the answer, policymakers feel that 20 percent of the answer is about as good as it gets. Given than, policymakers are often frustrated by the researchers’ reluctance to make recommendations on the basis of the existing evidence.

Campbell felt that the research abstract should clearly present the implications of the study, for at that point, the policymaker decides whether to examine the study’s conclusion, while only occasionally going back to the findings and discussion. That said, she felt while it was important for policymakers to have access to complete studies, she also wanted a way of moving up from a study to a synthesis or meta-view of the larger research issues, given the range of pertinent research: "The actual breadth of sources is huge...and no one can check them all or find one place where an individual can check them all. Secondarily, the type of research is much more complex to do than traditional clinical intervention research, so that quality is much trickier of an issue." And this time, when the quality assurance issue of information on the Web came up, it was in reference to the "gray literature" which included relevant but unpublished reports and commentaries. And while Campbell would like to see more warranted research online, the efforts of her own agency are sometimes curtailed in this regard. The Canadian Health Services Research Foundation has to be careful in its handling of the research it has sponsored so that it did not diminish that research’s ultimate "publishibility." This could well result in limited public access to publicly sponsored research. The research incentive system of publish-or-perish was working against the very mission of the Foundation. There should be a way to provide the traditional quality control of peer review while providing wider access to this publicly sponsored work on improving health care.

Among the Librarians

To complete the picture of how policy officials work with research, we visited the Ontario Legislative Assembly Library and spoke with two librarians and two research officers. Their job, as one of them put it, is "to help members [of the Legislative Assembly] improve the quality of debate." That is, the information they provided came into play after the policies had been drafted and were being considered by the Assembly. "We’re not often asked to help at the inchoate stages of policy," one of the research officers told us. They saw the information serving as a background to the work of the Assembly, whether in explaining the policy to opposition party members or to the press. Using research to inform the discussion of policy, which was not raised by the policy analysts we interviewed, clearly plays a critical role in a democratic society for setting the scope and terms of the debate. One of the research officers works extensively with academic research  ("I have to keep up with political science research," he said. "I need the journals at my fingertips" ) while the other research officer, who works with legal matters, deals more often with the legislation itself, mainly in comparisons of statues across jurisdictions, for example.  For the former, greater online access to well-organized bodies of research would be welcome, especially against the time constraints of having to prepare reports quickly that are supported by current research. And the currency of research by the time it reaches print, he noted, was often inadequate. The acquisitions manager of the library pointed out that increasing subscription costs have made it difficult for the library to retain academic serials, and that online systems were currently being offered as bundled services that required purchasing access to more than was felt warranted: "I don’t want to buy the whole thing if I need only 10 per cent of it."  It suggested that another sort of system was required to otherwise prevent a decline in the consultation of academic sources of information on policy matters.

Discussion

The interviews with these Canadian policy officials and related personnel make it clear that the Internet is now a favored source of information within government. It is used to tap into the research that is consulted as part of the policy process. It is also clear that the research that is most easily accessible, through portals and open-access sites, is most often consulted, as policymakers referred to how readily they were dissuaded from using pay-per-view and subscription services in their pursuit of knowledge. This means that they are tapping into a skewed and somewhat haphazard view of the current state of knowledge on a given topic. This could be seen as a further argument for establishing an "open access" economy of scholarly communication that would make scholarship freely available online. (Note 19)

Whatever form online scholarly publishing systems finally take, nearly everyone we spoke with agreed on the need for a system that warranted the status or credibility of the research, from clearly marking it as "peer reviewed" (with an accompanying explanation of the process) to providing citation statistics, which indicated whether others have referred to the paper, and in what context. (Note 20) Yet there is more to research online than peer review. The Internet also offers pre-prints and working papers referring to research, which has yet to be reviewed, but these are clearly marked with at least one set featuring a "warning" for "causal readers." (Note 21)

Among the other suggestions I would draw from the participants comments is the need for non-academics to establish a coherent context for a given study that enables readers (1) to review other studies by the author, as well as related studies both current and those going back decades, (2) to go deep into the body of the study, as far perhaps as the research instruments and raw data set where possible, (3) to rise up, above the specific study, to an overview or summary of the larger field of inquiry, (4) to consult related resources, such as relevant court decisions, as one policy maker mentioned, or related policy in other jurisdictions, (5) to access glossaries or reference materials that further open up the language and ideas to a wider readership, and (6) to interact with researchers, to comment on their work and consult directly with them, providing the researcher with a sense of that wider audience which will, in turn, give shape to how they express their findings.

Each of these abilities to work with research is certainly feasible in an online environment. For example, on giving some historical depth to the ability to locate related studies, the J-STOR project has been putting complete sets of journals, dating back to the nineteenth-century online. (Note 22) For our part, we drawn on these interviews to develop a "research support tool" that would provide a way for readers to readily move from an individual research study to related resources not only in research but in policy, practices, the media and other areas (Figure 1). We are also working on improving the indexing of online research to enable more precise and accurate searching for academic resources. The next phase of the larger study is to test the viability of such a support system with the participants of this study, the majority of whom expressed a willingness to take this next step.

Figure 1. Research Support Tool

Figure 1 represents an initial design for a tool that would provide studies in the field of education with a greater context for interpretation, judgment, and utilization by policymakers (and the public), based on the suggestions made in the interviews in this study. A working demo of this tool can be found at http://pkp.ubc.ca/demos/rsttour/index.html.

As encouraging as this potential for improving policy officials’ greater engagement with research may be, there remains the political caution about research’s influence expressed by Anne Larson Schneider and Helen Ingram, whose work I discussed above: "Where scientists and professionals have the greatest influence, however, the consequences are damaging to democratic values" (1997, p. 181). The increasing presence of research, through online technologies may well make citizens feel disempowered and "less capable of self-government" (p. 185) in the face of what Walter Lippmann identified many years ago as the "organized intelligence" necessary to run the modern state (1922). (Note 23) The policy officials with whom we spoke did lend credence to Schneider and Ingram’s fear "that scientific and professional networks...have colonized government so that there are no public arenas but only bureaucratic maneuvering among privileged specialists" (1997, p. 188). Not only did we speak with government economists, for example, shaping policy, but more than one policy official referred to working with a select group of academics who served as research filters and policy consultants. The danger of this limited consultation, outside of public arenas, can be challenged, we believe, by improving the online presence of research as part of knowledge’s public sphere. Creating open access to research would help policymakers, as well as the public, consult a much broader and more diverse range of information. With improved indexing, policymakers and the public would be able to locate innovative and fresh perspectives on specific and pressing issues, as well as ensure that they could consult contrary opinions and gain a sense of the range of approaches and opinions on a topic.

What is clear, from a democratic perspective, is that improving policymakers access to research has to be balanced by similar improvements in public access. This represents the democratic check on research’s political impact factor, but it also promises to raise the level of public deliberation and the very quality of democratic processes. Improving both policy and public access to research entails many of the same concerns with credibility, overview, integration, and context. In the United States, for example, science’s political contribution has been officially channeled through advisory bodies, such as the National Academies (with the Royal Society of Canada proposing a similar approach for the Canadian government). The Academies’ reports are made public and typically try to represent a coherent consensus among experts, with due consideration given to policy implications. As reassuring as such reports can be to the public, Stephen Hilgarter’s investigations have revealed the degree to which they are shaped by "stage management and struggles over the enclosure and disclosure of information" (2000, p. 146). These carefully reviewed reports may still misrepresent the state of dissent among researchers, just as the prestige and authority of these research reports can seem to squelch the deliberative and difficult public processes that make for enlightening, if sometimes exasperating, democratic experiences.

The democratic impact of research is likely to be advanced, given what this sample of policy officials revealed in their interviews, by creating a far more open, coherent, and integrated access to scholarship in ways that might serve policymakers and the public alike. The greater coherence and integration of scholarly communication that might be achieved through these new technologies should not be directed at overwhelming policy processes with the dictates of a scientific consensus. Rather, these new systems should be concerned with bringing the rich diversity of values and findings of research into public view and play. These systems need to make research more fully a part of a public sphere, marked by the constant contest of ideas, in the search for greater understanding of what is and can be known.

As social science researchers, it will not do, I contend, to imagine that our work can stay safely removed from the messy world in which people live and govern. We need to see our work as part of that life. In a few short years, the Internet has already increased the presence of research in the policy process. It has to a large extent replaced the traditional print sources, and well before this new communication medium has arrived at anything like a sufficiently well organized way of handling the publication of research. It falls to researchers, then, to take hold of the democratic possibilities, and to be guided by the clear cautions, in developing new systems for scholarly communication for both public and scholarly use.

Acknowledgment

I wish to thank Michael J. D. Sutton for his talented assistance with the interviews in this article, Anne White for her attentive reading of it, the Max Bell Foundation for its support of this work, and the editor of this journal for his fine editorial assistance.

Notes

1. See Alexander and Pal (1998), Hague and Loader (1999), Heeks (1999), and Wilhem (2000). The potential influence of this new technology on citizen consultation, for example, has only begun to be experimented with, and with mixed results. For our experiments with public and professional consultation on education policy, see the Public Knowledge Policy Forum under prototype websites of the Public Knowledge Project ( http://pkp.ubc.ca) and Klinger (2001).
2. The American Educational Research Association, for example, lists roughly100 e-journals that offer free, full-text access, including the well-respected Educational Researcher and Teachers College Record ( http://aera-cr.ed.asu.edu/links.html). RePEc: Research Papers in Economics provides open access to 102,000 working papers and 56,000 journal articles ( http://repec.org/). The University of Pennsylvania provides an excellent guide to electronic journals in all fields ( http://www.library.upenn.edu/webbin5/resources/ejspublic5.cgi).
3. On the recent new education bill, see Schemo (2002). Also Education policy’s stake in research, at the U.S. federal level, is reflected in the U.S. Education Department’s 2001 fiscal allocation of $185 million for education research and $80 for education statistics. Though it pales before the National Institutes of Health’s $20.3 billion allocation or the Defense Department’s $1.4 billion for "basic research," it represents a considerable and stable investment in creating knowledge that will presumably serve education policy and practice, and President Bush has proposed a 28 percent increase for educational research 2003 (Burd and Southwick, 2002).
4. On the challenges faced in the United States over evidence-based education, see Loveless et al.  (1999), and on its pertinence for education (Author, 2001).
5. See Friedman for a comparison of in-house bureaucracy, private consultants, and academic sources for policy analysis, with the academic "loner" judged to be often out of touch with "shifting sands of policy terrain," while their advice possesses a "somewhat abstract air" and they are "rarely available" when needed (1987, p. 162).
6. It may also be worth noting, on the other hand, the "social functions of ignorance" among bureaucracies, which Moore and Tumin argued, preserves traditional values and fair competition, as well as privilege and stereotypes, giving it a certain value for policy officials over the headlong pursuit of knowledge (1964).
7. Also see Stephen Hilgartner, who has observed that "governments find expert advice to be an indispensable resource for formulating and justifying policy and, more subtly, for removing some issues from the political domain by transforming them into technical questions" (2000, p. 146).
8. The National Institute of Child Health and Human Development have had considerable influence on education policy, although not without controversy among researchers: see Allington and Woodside-Jiron (1999), with response from Mathes and Torgesen (2000); and see Taylor et. al. (2000) with response from Foorman et al. (2000).
9. Here Schneider and Ingram (1997, p. 154) cite Max Weber’s caution in his essay, "Three Types of Legitimate Domination" (1978, pp. 991): "Bureaucracy naturally prefers a poorly informed, and hence powerless, parliament at least insofar as this ignorance is compatible with the bureaucracy’s own interest."
10. Lippmann on the development of democracy in America: "The more enlightened directing minds have called in experts who were trained, or trained themselves, to make parts of  this Great Society intelligible to those who manage it" (1922, p. 234). See Willinsky (2000, pp. 37-42, 100-105) and Aikens (1999).
11. See Willlinsky (2000b) as well as the Public Knowledge Project (http://pkp.ubc.ca).
12. See the Public Knowledge Project website (fn. 10) for access to the two designs.
13. Human Resources Development Canada (http://www.hrdc-drhc.gc.ca).
14. RePEc has some 160,000 items, including 60,000 online (http://ideas.uqam.ca); Also see, J-STOR: The Scholarly Journal Archive ( http://www.jstor.org/)
15. Canadian International Development Agency  (http://www.acdi-cida.gc.ca/).
16. Policy Research Initiative ( http://policyresearch.schoolnet.ca).
17. Policy.ca (http://policy.ca); Canadian Social Research Net ( http://www.canadiansocialresearch.net/).
18. Canadian Health Services Research Foundation (http://www.chsrf.ca/).
19. See Public Library of Science ( http://www.publiclibraryofscience.org/) and Budapest Open Access Initiative (http://www.soros.org/openaccess/).
20. See the NEC ResearchIndex for an online and open access system for representing the actual context in which a study has been cited by another study ( http://citeseer.nj.nec.com/cs).
21. See NetPrints: Clinical Medicine and Health Research set up by the British Medical Journal and Stanford University Self-Archiving Initiative which includes this statement boldly set up at the entrance to the site: "Warning: Articles posted on this site have not yet been accepted for publication by a peer reviewed journal. They are presented here mainly for the benefit of fellow researchers. Casual readers should not act on their findings, and journalists should be wary of reporting them" (http://clinmed.netprints.org/).
22. J-STOR (http://www.jstor.org/).
23. G. Scott Aikens sees the resolution of the Walter Lippmann and John Dewey struggle between expert and public as lying  in the cyberspace potential of public forums and open systems: "Whist Deweyan systems accept the need for rich systems of organized intelligence in complex societies, these can become richer through the active engagement of experts in open and free decision-making systems at the local, regional, and global level" (1999, p. 192).

References

Aikens, G. S. (1999). Deweyan systems in the Information Age. In B. N. Hague & B. D. Loader (Eds.), Digital democracy: Discourse and decision making in the Information Age (pp. 179-194). New York: Routledge.

Alexander, C. J. & Pal, L. A. (Eds.). (1998). Digital democracy: Policy and politics in the wired world. Toronto ON: Oxford University Press.

Allington, R. & Woodside-Jiron, H. (1999). The politics of literacy teaching: How "research" shaped educational policy. Educational Researcher, 28(8), 4-13.

Barker, A. & Peters, B. G. (1993). Introduction: Science politics and government. In A. Barker & B. G. Peters (Eds.), The politics of expert advice: Creating, using and manipulating scientific knowledge for public policy. Pittsburgh PA: University of Pittsburgh Press.

Burd, S. & Southwick, R. (2002, January 11). Congress approves increases of $250 in top Pell Grant and $3-billion for the NIH. Chronicle of Higher Education, A32.

Dewey, J. (1926). The public and its problems. New York: Holt.

Foorman, B. R., Fletcher, J. M., Francis, D. J., & Schatschneider, C. (2000). Response: Misrepresentation of research by other researchers. Educational Researcher 29(6), 27-37.

Friedman, J. (1987). Planning in the public domain: From policy to action. Princeton NJ: Princeton University Press.

Hague, B. N. & Loader, B. D., (Eds.). (1999). Digital democracy: Discourse and decision making in the information age. London: Routledge.

Heek, R. (Ed.). (1999). Reinventing government in the information age: International practices in IT enabled public sector reform. London: Routledge.

Hillage, J., Pearson, R., Anderson, A. & Tamkin, P. (1998). Excellence in research on schools. Department for Education and Employment, London.

Hillgartner, S. (2000). Science on stage: Expert advice as public drama. Stanford CA: Stanford University Press.

Humes, W. & Bryce, T (2001). Scholarship, research, and the evidential basis of policy development in education. British Journal of Educational Studies, 49(3) 329-352.

Klinger, S. (2001) "Are they talking yet?" Online Discourse as Political Action in an Education Policy Forum. Unpublished dissertation, University of British Columbia.

Lippmann, W. (1922). Public opinion. New York: Free Press.

Loveless, T., Peterson, P., Boruch, R., Cook, T., Gueron, J., Hyatt, H., & Mosteller, F. (1999). Can we make education policy on the basis of evidence? What constitutes high quality educational research and how can it be incorporated into policymaking? A Brookings Press Forum. Washington DC: The Brookings Institution.

Mathes, P. G. & Torgesen, J. K. (2000). A response to Allington and Woodside-Jiron. Educational Researcher, 29(6), 4-14.

Moore, W. E. & Tumin, M. M. (1964). Some social functions of ignorance. In B. Rosenberg, I. Gerver, & F. W. Howton (Eds.), Mass society in crisis: Social problems and social pathology. New York: Macmillan.

>Ozga, J. (2000). Policy research in educational settings. Buckingham, UK: Open University Press.

Putman, R. D. (2001). Bowling alone: The collapse and revival of American community. New York: Simon and Schuster.

Schemo, D. J. (2002, January 9). Education bill urges new emphasis on phonics as method for teaching reading, New York Times, A7.

Schneider, A. L. & Ingram, H. (1997). Policy design for democracy. Lawrence KA: University Press of Kansas.

Simon, H. (1945). Administrative Behavior. New York: Free Press.

Taylor, B. M., Anderson, R. C., Au, K. H., & Raphael, T. E. (2000). Discretion in the Translation of Research to Policy: A Case from Beginning Reading. Educational Researcher, 29(6), 16-26.

Weber, M. (1978). Three types of legitimate domination. In C. Roth & C. Wittch (Eds.), Economy and society: An outline of interpretive sociology. Berkeley, CA: University of California Press.

Weiss, C. (1998). Interview by Miranda Christou. Cambridge MA: Harvard University School of Education. Available at http://www.gse.harvard.edu/~oie/weiss.html.

Weiss, C. (1991). Policy research: data, ideas, or arguments. In P. Wagner, C. H. Weiss, B. Wittrock & H. Wollman (Eds.), Social sciences and modern states: National experiences and theoretical crossroads (pp. 307-332). Cambridge, UK: Cambridge University Press.

Wilhelm, A. G. (2000). Democracy in a digital age: Challenges to political life in cyberspace. New York: Routledge.

Willinsky. J. (2000a). If only we knew: Increasing the public value of social science research. New York: Routledge.

Willinsky. J. (2000b). Proposing a Knowledge Exchange Model for scholarly publishing. Current Issues in Education, 3(6). Available: http://cie.ed.asu.edu/volume3/number6/.

Willinsky. J. & Wolfson, L. (2001). The indexing of scholarly journals: A tipping point for publishing reform? Journal of Electronic Publishing, 7(2). Available at http://www.press.umich.edu/jep/.

About the Author

John Willinsky
Department of Language and Literacy Education
University of British Columbia
Vancouver, BC, Canada V6T 1Z4
john.willinsky@ubc.ca

John Willinsky is the Pacific Press Professor of Literacy and Technology at the University of British Columbia where he directs the Public Knowledge Project. He is the author of a number of books, including Learning to Divide the World, which won two outstanding books awards, and If Only They Knew: Increasing the Public Value of the Social Science Research.

Appendix

Interview Schedule Used With Policymakers

General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .