Utilization in Higher Education Rulemaking : A Multi-Case Study of Research Prevalence , Sources , and Barriers

For stakeholders who would like to see more research as a basis for educational policy, it is important to understand the prevalence of research use and the sources of the studies used by policymakers, as well as the factors that hinder research use in educational policymaking. Through an analysis of regulatory documents and interviews with 34 key informants, this multi-case study examined the prevalence and sources of research utilized in higher education rulemaking, which is the process for developing federal regulations that govern higher education. This study also examined barriers to using research in h igher education rulemaking. Findings indicate that while research has been used in this process, factors other than research were discussed more frequently in final regulations. Barriers to research use in higher education rulemaking included time constraints, unavailability of data, politics, lack of government research capacity, and other disjunctions between the research and policy communities. Moreover, the contexts in which particular rules were Education Policy Analysis Archives Vol. 28 No. 95 2 created shaped the prevalence and sources of research used in the regulations’ development. The article concludes with implications for policy and theory.


Introduction 1
In recent decades, educators and policymakers have increasingly called for the use of research to inform educational policy (Cooper, Levin, & Campbell, 2009;Levin, 2013;Malin & Lubienski, 2015;Nutley, Walter, & Davies, 2007). These calls reflect a perspective that using findings from research in policy development can "lead to more informed policy, higher-quality decisions, more effective practices, and, in turn, improved outcomes" (Cooper et al., 2009, p. 160). At the same time, observers recognize that research often does not serve as a basis for policy (Asen, Gurke, Solomon, Conners, & Gumm, 2011;Asen, Gurke, Conners, Solomon, & Gumm, 2013;Henig, 2008). Politics, funding resources, and policymakers' own experiences influence whether and what type of research is conducted or used in policymaking (Alkin & King, 2017;Asen et al., 2013;Levin, 2013;Nutley et al., 2007;Weiss, 1979). For stakeholders who would like to see more research as a basis for educational policy, it is important to understand not only how prevalent the use of research is and the sources of studies that have been used by policymakers, but also the factors that hinder research use in educational policymaking.
This study examined the prevalence of and barriers to research utilization in federal higher education rulemaking, which is the process through which the United States Department of Education creates regulatory policy to administer provisions of the Higher Education Act (Lubbers, 2014;Natow, 2017;Pelesh, 1994). Regulations created through the rulemaking process have consequential implications for higher education institutions and students. For example, federal regulatory policy has provided a basis for discharge of student loan debt for borrowers deemed to have been defrauded by their institutions (Kreighbaum, 2019), and noncompliance with federal regulations may result in the loss of eligibility for institutions to participate in federal student financial aid programs (Lederman, 2019).
Although previous literature has not examined how research has been used in higher education rulemaking, understanding research utilization in this process is important for several reasons. First, federal law requires government agencies to conduct analyses at different points during the rulemaking process, including regulatory impact assessments and cost-benefit analyses, for many regulations (Belfield, Bowden, & Rodriguez, 2018;Costa, Desmarais, & Hird, 2019;Executive Order 12866, 1993). Rules deemed insufficiently based on evidence are more likely to be struck down by a court (Association of Private Sector Colleges & Universities [APSCU] v. Duncan, 2012;Costa et al., 2019), which can in turn result in confusion about the rule's status and expenditure of additional government time and resources on redeveloping the regulations (Kelchen, 2019). The study presented here reveals the extent to which research was considered in the creation of several key higher education regulations and the sources of that research. This study also identifies barriers that have obstructed the use of research in the rulemaking process. Understanding factors that hinder research use can help proponents of evidence-based policymaking to develop strategies for overcoming those barriers so that more research may be considered in the creation of these policies.
This study also provided an opportunity to examine whether research features prominently in a policymaking process that is intentionally structured to bring research-oriented organizations directly into conversation with government officials for the purpose of drafting policy language. As explained more fully below, the Higher Education Act (2018) requires the Department of Education to meet with stakeholder groups to develop proposed regulations through a process known as negotiated rulemaking when developing regulations that affect federal student financial aid programs (e.g., Natow, 2017;Pelesh, 1994). One common explanation for why research is not used more frequently in policymaking is that researchers and policymakers are part of different "communities" that hold dissimilar values, incentive structures, timelines, and practices regarding research (Caplan, 1979;Henig, 2008;Ness, 2010). The negotiated rulemaking process enables higher education institutions, many of which employ researchers and house research centers, to participate in the policymaking process in a more direct way than other policymaking processes typically allow. Indeed, many recent higher education-negotiated rulemaking sessions have included administrators (and in a few cases faculty) employed by individual colleges and universities as participants (e.g., Program Integrity, 2014bStudent Assistance General Provisions, 2015b, 2016bTeacher Preparation Issues, 2014). By involving research-oriented organizations such as universities in a direct and open conversation with other interest groups and federal officials, the higher education rulemaking process provides a unique opportunity to study whether the divide between policymakers and researchers may be bridged by a policymaking process that involves extended public meetings and discussions between these parties.

Overview of Federal Rulemaking for Higher Education
Federal rulemaking is the process through which a federal government agency creates regulations that implement the provisions of duly enacted authorizing statutes (Administrative Procedure Act, 2018; Kerwin & Furlong, 2011; Office of the Federal Register, 2011). The Higher Education Act (HEA) is an authorizing statute, and federal regulations created under its authority are developed through rulemaking in the U.S. Department of Education (Lubbers, 2014;Natow, 2017;Pelesh, 1994). Under the HEA, rulemaking that affects federal student financial aid programs must include a process known as negotiated rulemaking, during which the agency meets with stakeholders to debate and negotiate the content of a proposed rule. Participants in higher education negotiated rulemaking have included representatives of different types of colleges and universities (including public four-year, public two-year, private nonprofit, and private for-profit institutions), student representatives, consumer advocates, the Department of Education's negotiator, and depending on the topic, representatives of other stakeholders such as accreditors, the student lending industry, and more. The negotiating team attempts to arrive at a consensus on the language for the proposed regulation. If consensus is attained (that is, if there is unanimous agreement of all negotiators), then the agency uses that content in the Notice of Proposed Rulemaking (NPRM). But if consensus is not reached, the agency alone writes the NPRM. The HEA requires the department to receive feedback from a broad range of stakeholders prior to negotiated rulemaking regarding the matters to be negotiated. These comments may be provided in writing or in the form of statements made at meetings with agency personnel that take place in different regions of the United States (Kerwin & Furlong, 2011;Lubbers, 2014;Natow, 2017;Office of the Federal Register, 2011;Pelesh, 1994).
Following negotiated rulemaking, the department publishes an NPRM in the Federal Register. This initiates the comment period, during which stakeholders may provide written comments to the Department of Education regarding the rule. After the closing of the comment period, the department examines the comments, decides whether to make adjustments to the rule's language based on those comments, and eventually publishes the final rule -including a description of the comments received and the department's responses to them -in the Federal Register (Kerwin & Furlong, 2011;Lubbers, 2014;Natow, 2017;Office of the Federal Register, 2011;Pelesh, 1994).
According to a decades-old executive order, regulations are deemed "significant" when they are projected to affect the economy by an amount of at least $100 million annually, are anticipated to have a negative effect on the economy, will substantially impact government budgets, or will introduce "novel legal or policy issues" (Executive Order 12866, 1993, p. 51738). Rules that fall under this definition must undergo regulatory impact assessments and reviews by the Office of Management and Budget (OMB) (Belfield et al., 2018;Executive Order 12866, 1993;Office of the Federal Register, 2011;Sunstein, 2013). Stakeholders may meet with OMB staff to provide their perspectives on rules subject to OMB review (Sunstein, 2013). When conducting regulatory assessments, Executive Order 13563 (2011) requires that agencies "use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible" (p. 3821).
After a regulation is finalized, rulemaking politics may continue. A final rule may be subject to nullification or revision by a legislative action, a future rulemaking, or a court order (Kerwin & Furlong, 2011;Office of the Federal Register, 2011). All of these post-finalization events have occurred with respect to higher education rules in the recent past (e.g., Nolter, 2012;Program Integrity, 2019;Stephan, 2017).

The Use of Research and Other Factors in Policymaking Conceptualizing Research Utilization
What is research? There are many definitions of research reflected in public policies and the academic literature. Asen et al. (2011) observed that the federal government's definition of research in the No Child Left Behind (NCLB) law refers to research as "systematic" and "objective," employing "empirical methods" and "rigorous data analysis" with a preference for randomized controlled trials (No Child Left Behind Act, 2002, as quoted in Asen et al., 2011. 2 In a separate article, Asen et al. (2013) provided their own definition of research in somewhat broader terms: "empirical findings derived from systematic analysis of information, guided by purposeful research questions and method" (p. 40). Nutley et al. (2007) broadly conceptualized "research as any investigation towards increasing the sum of knowledge based on planned and systematic inquiry," with no preference for particular types of research methods so long as the methods are appropriate for the what the researcher is studying (pp. 21-22). Penuel and colleagues defined research simply as a "systematic inquiry to answer a specific question" (Penuel et al., 2016, p. 15). Although these definitions differ from each other, they underscore that research involves more than just examining a snapshot of data or mere "number crunching" that is not part of a more systematic analysis; rather, research involves a methodologically justifiable plan and the goal of responding to particular questions or inquiries.
How is research used? There are different ways to use research in policymaking.
process use research to substantiate their points by indicating that research supports them but do not provide specific citations or other details about the research (Asen et al., 2011). Research might also "inform and enrich" policymaking discussions but ultimately "not drive the outcome" (Head, 2015, p. 474). In their examination of how research use has been defined in the literature, Alkin and King (2017) noted that research need not be identified as the basis for a policy in order for research to have been used, provided that the research was discussed or given consideration in the decisionmaking process, even if only as "one of multiple influences" (p. 438).

Barriers to Research Utilization in Policymaking
Despite calls to increase the use of research in policymaking, scholars have documented the infrequency with which research has been used relative to other factors (Asen et al., 2011(Asen et al., , 2013Gollust et al., 2014). For example, in their study of research use by school boards, Asen et al. (2013) found that research was used comparatively less than other factors, such as examples and law. Also, Nutley et al. (2007) wrote that it has been "rare" for policymakers to use "traditional, academic, peer-reviewed literature" in policymaking (p. 62).
Although an examination of barriers to research use specifically in federal higher education rulemaking does not appear in prior literature, several studies have documented barriers to research utilization in other contexts. One barrier is when decision makers do not understand research or perceive it to be valuable (Nutley et al., 2007;Penuel et al., 2016). Another is when an organization or agency lacks sufficient resources to consider research in decision making (Farley-Ripple, 2012;Santesso & Tugwell, 2006;Tseng & Nutley, 2014). Time limitations have also been found to hinder research use in decision making. For example, Farley-Ripple (2012) found that school leaders' myriad responsibilities left them little time "to search, read, and employ education research in their day-to-day work" (p. 799). Additionally, a mismatch often exists between policy timeframes and research timeframes, with the policy world moving at a considerably faster pace than it takes researchers to conduct, write about, and publicize their studies (Henig, 2008;Nutley et al., 2007).

The Influence of Context on Research Utilization
Surrounding contexts can influence decisions about whether and how to use research in decision making. That is, factors such as the history or culture of a potential research user's organization, the resources available to a potential research user, and external political pressure can influence whether, how much, and what kinds of research utilization occur (Alkin & King, 2017;Farley-Ripple et al., 2018;Head, 2015;Henig, 2008;Holzer, Lewig, Bromfield, & Arney, 2007;Levin, 2013;Nutley et al., 2007;Santesso & Tugwell, 2006). Organizational structures, such as whether a division specifically dedicated to research exists within an organization, is another context that can influence research utilization (Farley-Ripple et al., 2018). The subject matter of a proposed policy is another example of a context that may influence the use of research in a policy's creation. As Nutley et al. (2007) wrote, "more radical change requires more extensive debate" (p. 110). This may lead to greater use or scrutiny of research in developing controversial or far-reaching policies.
Another important context involves the networks of research users and policymaking participants (Head, 2015). The extent and kinds of evidence used in particular cases are often influenced by the networks and relationships of those involved (Cooper & Levin, 2010;Farley-Ripple & Grajeda, 2020;Head, 2015;Ng-A-Fook, Kane, Butler, Glithero, & Forte, 2015). These relationships may be between researchers and policymakers directly, but often other entities will serve as intermediaries who are neither policymakers nor researchers but work to connect policymakers with useable research findings (Cooper, 2014;Cooper et al., 2009;Cooper & Shewchuk, 2015;Farley-Ripple et al., 2018;Lubienski et al., 2014;Malin & Lubienski, 2015;Naidorf, 2014;Nutley et al., 2007;Rodway, 2015;Scott, Lubienski, DeBray, & Jabbar, 2014;Tseng & Nutley, 2014). Examples of intermediaries include foundations, policy-oriented think tanks, advocacy organizations, the media, labor unions, and others (Malin & Lubienski, 2015;Ness & Gándara, 2014;Scott et al., 2014). These relationships and networks contribute to "knowledge mobilization," which is the "dissemination of … research through various channels to its ultimate use (or lack thereof) in policy and practice settings" (Cooper, 2014, p. 4). In contexts where policy actors and researchers network with each other or with intermediaries, research is more likely to be considered (Nutley et al., 2007;Rodway, 2015).

Use of Factors Other Than Research in Policymaking
Factors other than research have also been used in policymaking, including data and statistics. When these are presented as snapshots of information and not part of a larger inquiry, they are not considered research. Asen et al. (2013) defined "research" and "data" differently in their analysis of school board proceedings, classifying findings from a study that involved methodical data analysis and purposefully designed procedures as research, whereas "measurable quantitative or qualitative information systematically collected to describe a set of conditions or trends" was considered data (p. 40). Similarly, in their study of educational leaders, Penuel et al. (2016) differentiated data from research, noting that the examination of data was "more open-ended" than conducting research, and that the use of data "seldom addresses specific research questions" (p. 15).
Examples have also been cited in attempts to influence policymaking (Asen et al., 2013;Simons, 2015). An example is "a specific case or incident" that is "typical or exceptional" of the situation at issue in policymaking (Asen et al., 2013, p. 40). Actual examples are "concrete initiatives and projects" that have actually existed (Simons, 2015, p. 715), and hypothetical examples are situations that are reasonably likely to occur but have not actually occurred. A form of hypothetical example described in OMB guidance permits agencies to consider "plausible scenarios" when conducting cost-benefit analyses if there is "scientific uncertainty" with regard to calculating costs and benefits in regulatory policymaking (U.S. Office of Management & Budget, 2003, p. 39).
Other non-research factors used in policymaking include values and beliefs, previous experience or traditions, and laws or public policies (Alkin & King, 2017;Asen et al., 2013;Davies, 2004;Head, 2015;Nutley et al., 2007;Santesso & Tugwell, 2006;Weiss, 1979). Experts' perspectives have also been considered in policymaking (Davies, 2004). Policymakers have sought expert testimony in legislative hearings (Perna, Orosz, & Kent, 2019), via consulting relationships (Morgan et al., 2006), or in written statements, such as public comments during the rulemaking process (Costa et al., 2019). Expert perspectives are not in themselves a separate category of influence in policymaking but typically fit into one of the other categories: experts may present the results of research or other data, discuss something they learned from previous experience, or speak about their own beliefs or values.

The "Two Communities" and Potential for a Bridge
Barriers to research use in policymaking often reflect a gulf between researchers and policymakers characterized by a lack of effective communication, divergent cultures, and inadequate understanding of each other's roles -a phenomenon known as the "two communities" (Caplan, 1979;Farley-Ripple et al., 2018;Ness, 2010;Ness & Gándara, 2014;Nutley et al., 2007). According to this perspective, policymakers and researchers "live in separate worlds with different and often conflicting values, different reward systems, and different languages" (Caplan, 1979, p. 459). Some scholars have critiqued the two-communities perspective as too reductive (Newman, Cherney, & Head, 2016;Nutley et al., 2007;Rich, 1991). However, divides between the research and policy communities have been recognized in the literature (Court & Maxwell, 2005;Ness, 2010), with some observers arguing that differences between the policy and research communities are inevitable and even beneficial (Birnbaum, 2000;Locock & Boaz, 2004).
While it may be overly simplistic to state that policymakers and researchers are members of two separate communities, the limited use of research in policymaking and the fact that research utilization has been hindered by fundamental disjunctions between the research and policy worlds indicate that the two-communities perspective should not be too hastily dismissed. Rather than disregard the theory, some scholars have built on it, acknowledging that the framework is not complex enough to capture the entire picture, but also recognizing the existence of a gap between these communities (Farley-Ripple et al., 2018;Holzer et al., 2007). For example, Holzer et al. (2007) advanced a conceptual model that recognized the different "cultures" of researchers, policymakers, and practitioners, but also took account of a variety of different contextual factors, including political, social, organizational, and individual characteristics.
There is reason to believe that the federal higher education rulemaking process could serve as a bridge between the research and policy communities. This is because a unique aspect of this process is the frequent use of negotiated rulemaking, which is required in the creation of many federal student financial aid regulations (Higher Education Act, 2018;Pelesh, 1994). As explained above, negotiated rulemaking is a process in which stakeholders and a regulatory agency work together to develop the substance of proposed regulations (Kerwin & Furlong, 2011;Pelesh, 1994). Because some stakeholders of higher education rulemaking are colleges and universities, employees of individual higher education institutions have commonly participated in negotiated rulemaking in the U.S. Department of Education. These institutions employ faculty and other researchers, and they often house research centers and receive research funding from government and other sources. Additionally, Washington, DC-based associations have participated in negotiated rulemaking by nominating individuals for negotiating teams or by having their employees serve as negotiators (Natow, 2017). To the extent that these organizations act as "knowledge brokers" for the higher education policy community, their participation may serve as another link between research and policymaking in this process, as "knowledge brokers" have been "widely recognized as potential levers for bridging the two communities" (Farley-Ripple & Grajeda, 2019, p. 67). For these reasons, higher education rulemaking presents a unique opportunity to determine whether aspects of the "two communities" phenomenon persist in a policymaking process that involves direct participation by research-oriented stakeholders and intermediary organizations.

Methods
This case study (Creswell & Poth, 2018;Merriam, 1998;Yin, 2014) investigated how research has been used in the development of the following five federal higher education rules: Gainful Employment, Revised Pay as You Earn (REPAYE), Borrower Defense, Teacher Preparation, and Gainful Employment Rescission. This study received Institutional Review Board approval prior to data collection; data sources included documents and interviews.

The Case-Study Rules
These case studies of higher education rulemaking were purposefully selected via a sampling strategy that combined maximum variation and typical case sampling (Creswell & Poth, 2018;Suri, 2011). This strategy identified regulations developed through the usual process that occurs for higher education rulemaking while leaving room to examine similarities and differences across the diverse subject matters and contexts of these particular rules. Specifically, these five case-study rules were selected because all of them met the following typical case sampling criteria: (1) they were subject to negotiated rulemaking to develop proposed regulatory language; (2) their post-NPRM phase involved receiving comments from interested parties; and (3) all of the case-study rules were reviewed by the OMB before they were finalized. These are standard practices for most regulations developed under Title IV of the HEA. However, these rules also differed in key ways, to provide maximum variation among the sample of rules. The differences include subject matter and regulatory history, as explained in the descriptions of the case-study rules immediately below.
Gainful Employment Rule. Issued in 2014, this rule was the Obama administration's second attempt to define the term "gainful employment" that appears in the HEA, which provides that career-focused programs may be eligible to receive federal student financial aid if the programs "prepare students for gainful employment in a recognized occupation" (Higher Education Act, . Among other things, this rule required data disclosures and set an acceptable debt-income rate for an institution's "typical graduate" that "does not exceed 20 percent of his or her discretionary income or 8 percent of his or her total earnings" (U.S. Department of Education, 2014, para. 3). This regulation followed the nullification of the repayment rate provision of the first Gainful Employment rule (issued in 2011) when a court found the rate to be "arbitrary and capricious" because it did not possess a "reasoned basis," such as expert opinion or studies indicating the rate was appropriate (APSCU v. Duncan, 2012, pp. 30-31; see also Nolter, 2012).

REPAYE Rule.
This 2015 rule revised a federal income-based student loan repayment program to expand eligibility to a broader set of borrowers and to provide different debt forgiveness qualification criteria for borrowers depending on whether they had only undergraduate debt or debt from graduate school as well. The rule also limited the amount of interest charged on federal loans depending on the borrower's income (Stratford, 2015;Student Assistance General Provisions, 2015a). The REPAYE negotiated rulemaking committee reached consensus when negotiating the proposed rule (Stratford, 2015); it is the only case-study rule on which consensus was reached.
Borrower Defense Rule. This rule was issued in 2016, toward the end of the Obama administration. The rule governed the circumstances under which student loan borrowers could assert, as a defense to debt repayment, fraud on the part of their higher education institutions. The regulation restricted institutions' ability to require students to agree to arbitration rather than lawsuits for resolving disputes (Kreighbaum, 2016;Student Assistance General Provisions, 2016a). The Trump administration took office not long after this rule was issued. At first, Trump's Department of Education delayed implementing the rule before a court ruled that the delay was not permissible (Kreighbaum, 2018b). Then in 2019, the department substantially modified the rule and repealed some key aspects of it, including the ban on required arbitration agreements (U.S. Department of Education, 2019).
Teacher Preparation Rule. The Teacher Preparation rule was also issued in 2016 toward the end of the Obama administration. This rule set new accountability requirements for teacher preparation programs under Title II of the HEA. For example, the rule would have required teacher education programs to provide the federal government with data on their graduates' job placement (including the number of alumni working in schools deemed "high-need") and indicators of academic performance of the students taught by program alumni (Stephan, 2017;Teacher Preparation Issues, 2016). Early in the Trump administration, the Teacher Preparation rule was repealed via a Congressional Review Act resolution before it took effect (Stephan, 2017).

Gainful Employment Rescission.
In 2019, the Department of Education released a final regulation that rescinded the 2014 Gainful Employment rule (Program Integrity, 2019). This is the only case-study rule that was created by a Republican presidential administration.

Documentary Data
I obtained copies of each case-study final rule (and any applicable corrections) from the online version of the Federal Register. For each rule, I conducted a line-by-line analysis of the sections that discussed the reasoning behind the regulations. These sections, which I collectively refer to as the reasoning portions of the rules, included the Executive Summary, the Analysis of Comments and Changes, the Regulatory Impact Assessment, and other sections of the final rules that discussed the Department of Education's rationale for why the regulations contained the provisions that they did (see also Office of the Federal Register, 2011). Because the regulatory text to be codified in the Code of Federal Regulations did not contain background or explanatory information, those sections of the final case-study rules were not included in this line-by-line analysis.
I coded each page for the discussion of research and other factors that appeared on the page. These factors fell into six categories: research, data/math, experience, examples, beliefs/opinions, and law. The number of pages in the Federal Register on which each factor appeared was chosen as the unit of measure because page counts in the Federal Register have been used in other studies of regulation to measure such attributes as regulatory complexity and the amount of rulemaking activity (e.g., Acs & Cameron, 2013;Brito & de Rugy, 2008;Cochran, 2001;Kerwin & Furlong, 2011). Pages are also an appropriate measure because the reasoning sections of final rules often contain tables and numbers that would not be conducive to measuring by paragraph or word counts.
The definition of research used in this study was broadly conceptualized to include any systematic analysis conducted according to a methodological plan to address a particular area of inquiry (derived from definitions of research found in Asen et al., 2013;Holzer et al., 2007;Nutley et al., 2007;and Penuel et al., 2016). Although this analysis employed a broad definition of research that included diverse methodologies and sources, to be considered research (as opposed to data, examples, or other factors described below), an item must have contained some kind of systematic inquiry and involved analysis that went beyond mere anecdotes or "number crunching." In keeping with Penuel et al.'s (2016) definition, this study conceived of research as "an activity in which people employ systematic, empirical methods to answer a specific question" (p. 16).
Similar to Asen et al. (2013), I applied the data/math code to the discussion of statistics or general trends apart from a broader, systemic analysis. For example, the following statement from the Gainful Employment rule was coded as data/math: "One commenter asserted that the 150 percent normal time graduation rate for public and private non-profit open-enrollment colleges is 28.3 percent and 39.7 percent respectively …" (Program Integrity, 2014a, p. 64903). Also included in the data/math code were discussions of mathematical formulas or principles.
Experience was defined as testimonials or first-hand accounts based on "direct observation of or participation in events or activities" (Asen et al., 2013, p. 40), as well as descriptions of traditions, past practice, and other previous experiences (Asen et al., 2013;Davies, 2004). The experience code also included practices that are standard or considered "best practices" in a particular field because such standards and practices are generally developed through previous experience in the field (e.g., Kocbek & Juric, 2010;Rosenthal, 2012). The definition of example was based largely on Asen et al.'s (2013) characterization of the term, which involved "a specific case or incident used to illustrate typical or exceptional characteristics of a topic or issue" (p. 40). Such instances were coded as examples regardless of whether they were real or hypothetical and included discussions of "plausible scenarios" permitted by OMB guidance for regulatory impact assessments (U.S. Office of Management & Budget, 2003, pp. 18, 39;2011, pp. 14-15). Beliefs/opinions referred to beliefs, values, viewpoints, ideologies, and other subjective perceptions (Davies, 2004;McCright & Dunlap, 2008). Finally, law was defined as rules, legal mandates, public policies, and governmental guidelines for behavior and activities, including statutes, court decisions, regulations, executive orders, agency guidance, and other legal instruments issued at any level of government (Asen et al., 2013;Clarke, 2016). I also reviewed negotiated rulemaking records and obtained available reports of research cited in each final rule to identify the types and sources of research discussed in the rules.

Interview Data
I conducted in-depth, semi-structured interviews with 34 individuals who had some involvement in the Department of Education's higher education rulemaking process in recent years. Interviewees were purposefully sampled key informants (Marshall, 1996), who had knowledge about one or more of the case-study regulations as well as other rules developed through higher education rulemaking. Interviewees represented a broad range of policy actors, including people with experience working as Department of Education or other federal officials, employees of non-profit organizations, and representatives of state government offices, students, consumers, the lending industry, accreditors, and higher education institutions. Table 1 depicts the categories of professional positions held by this study's interviewees. Because some interviewees held more than one position through which they worked on rulemaking issues (e.g., a former federal official who later worked at a non-profit organization doing higher education policy work), the total number of positions reflected in Table 1 is greater than the total number of respondents who participated in this study. Other federal staff 5 Higher education institution or association representing same Public four-year college/university 3 Public two-year/community college 3 Private non-profit college/university 6 Private for-profit college/university 2 Financial aid administrator or association representing same 5 Student loan industry/guarantor or association representing same 3 Accreditor or association representing same 2 Student, consumer, student-veteran, or legal aid advocate 6 Think tank staff 4 State-level government staff or association representing same 2 The interview protocol included broad, open-ended questions about respondents' experiences and observations regarding research use in higher education rulemaking, including how the department and other participants utilized research during rulemaking and whether particular circumstances or other factors made it more or less likely for research to be used. Other interview questions asked about research use in different stages of the rulemaking process, the methodologies of research used, and factors other than research that were used. Because previous studies have not examined research utilization specifically in the higher education rulemaking process, interview questions were informed by the literature on research use prevalence, hindrances, and facilitators in other policymaking contexts (e.g., Asen et al., 2011Asen et al., , 2013Gollust, 2014;Nutley et al., 2007). 3 Data analysis of interviews followed practices described by Miles, Huberman, and Saldaña (2019) and included multiple coding stages, analytic memos, and matrices to categorize, reduce, and visualize findings from the dataset. I used Dedoose qualitative analysis software to code transcripts with a standard coding scheme created with concepts from the literature as well as categories that emerged during interviews. I then exported coded data relevant to research use prevalence and barriers into a spreadsheet and reviewed those excerpts again to identify additional emergent themes and patterns.

Prevalence, Contexts, and Sources of Research Utilization
Prevalence of research utilization. Table 2 indicates the number and percentage of pages in the reasoning portions of the final case-study rules that discussed research and other factors. The Analysis of Comments and Changes section of final rules is unique in that it singularly focuses on responding to comments received during the Notice and Comment period (Natow, 2017; Office of the Federal Register, 2011). For this reason, the number and percentage of pages on which research and other factors were discussed in the case-study rules appear separately from the rest of the reasoning sections in the table. Table 2 is presented with a caveat: There is no requirement that research must be discussed in any particular proportion of pages in final regulations or that research should necessarily be discussed more frequently than other factors in final rules. Moreover, because much of the text of final rules involves summarizing and responding to comments received from the public, it would be expected that those subsections would discuss matters raised in the comments, which may not include research. Thus, the fact that research was discussed in final rules less frequently than other factors does not in itself indicate that research was not considered sufficiently in the development of the rule. However, Table 2 provides important and relevant information for a study of research use in higher education rulemaking. First, information about the frequency with which research and other factors were discussed in final rules allows for a comparison of how often research was featured vis-à-vis other factors. The information in the table also allows for an analysis of how frequently research was discussed in some rules versus others, which is important for understanding how a given rule's context may influence research use in the rulemaking process. As Table 2 demonstrates, there was some variation among the case-study rules regarding the proportion of pages and sections in the final regulations that discussed research, data, and other factors. One consistency across all five case-study rules is that research was discussed on fewer pages of the final regulations than all of the other factors reflected in the table. The only final rule in which research was discussed on more than half the pages was the 2019 Gainful Employment rescission, and the majority of research discussed in that rule occurred in the Analysis of Comments and Changes. The 2014 Gainful Employment rule and the Teacher Preparation rule, which both discussed research on just under one-quarter of the pages, cited research more often than REPAYE or Borrower Defense. In three of the five case-study rules, examples were discussed on a larger proportion of pages than any other factor, and examples were mentioned on at least two thirds of pages in all of the final regulations. This is consistent with Asen et al.'s (2013) finding that examples were used more frequently than other types of evidence in their study of local school boards. Also notable is the fact that, in every case-study rule, data or mathematical analyses were discussed on upwards of 90% of the pages of reasoning portions that were not the Analysis of Comments and Changes, and were discussed less frequently in the Analysis of Comments and Changes than in the combined other reasoning sections. This suggests that regulation drafters were eager to cite data or mathematical calculations to justify the rules even when not doing so to respond to public comments.
A number of this study's interviewees made statements indicating they had not observed research playing a large role in higher education rulemaking. One negotiated rulemaking participant stated that "honest to goodness research" has been used in a "precious little" amount during higher education rulemaking. Another negotiator conceded that "you don't see a lot of research being introduced" into the process, and a representative of community colleges noted that "not that much" research has generally been discussed in the development of the rules. Many other respondents made similar comments.
Although research did not appear as frequently as other factors in the final case-study regulations, interviewees and rulemaking documents indicated that research was, in fact, used in the development of these regulations, even if infrequently. Moreover, when research was utilized in higher education rulemaking, it was often used for political purposes, such as to substantiate the arguments or promote the policy preferences of the research users. This "political" use of research is to be expected in policymaking environments such as the higher education rulemaking process (Ness, 2010;Weiss, 1979). 4 Contexts of research utilization. Interviewees indicated that a reason why some rules, such as the two Gainful Employment regulations and the Teacher Preparation rule, discussed research on more pages than other rules was related to contextual circumstances specific to those rules. The 2014 Gainful Employment rule was created following a prior version of the rule being largely invalidated by a federal court because an important provision of the rule lacked a "reasoned basis" (APSCU v. Duncan, 2012, pp. 1, 30). Respondents said that this history weighed on federal officials who were involved in creating the later version of the rule. One federal official explained that in that rulemaking, the government was interested in: find[ing] whatever way has the best chance of holding up in court … And so that was like the thrust of our efforts, looking at research and evidence. It wasn't like, okay, what's the problem here, or what are we trying to do? It was like, what has the best chance of holding up in court, to meet this ultimate objective?
Another respondent who had worked for the federal government confirmed: The original rule had been thrown out by the judge because … [the department] had kind of picked the percentage repayment rate below which programs wouldn't be eligible, and … the judge kind of argued that the standard that they had picked wasn't grounded in research; it was arbitrary. And so the rule had been kind of thrown out largely on that basis… So [during the 2014 rulemaking] there was … internally a pretty exhaustive review of the research literature on for-profits.
Given this history, it is unsurprising that the subsequent Gainful Employment rule and its rescission would include relatively more discussions of research than other rules in the dataset. The context surrounding the Teacher Preparation rule, which did not have an identical history to the Gainful Employment rules but had other characteristics in common with them, may also explain why research was cited on more pages of that regulation than was the case in Borrower Defense or REPAYE. Like Gainful Employment, the Teacher Preparation rule was controversial (Stratford, 2014). Also like Gainful Employment, Teacher Preparation included the possible loss of eligibility to participate in certain federal financial aid programs as a penalty for institutions that did not meet the rule's requirements (Program Integrity, 2014a; Teacher Preparation Issues, 2016). These contexts influenced policymakers as they developed the rules and used research in the rulemaking process. A federal official interviewed for this study explained: Particularly if you're going to remove eligibility from an entity or kind of punish a particular entity -I guess that was also kind of a feature of Teacher Prep, but it was really salient in the Gainful Employment rule -then you had to have a kind of an ironclad justification for your metric.
These contexts of the Gainful Employment and Teacher Preparation rules help to explain why research featured more prominently in those rulemakings than with the other case-study regulations. Another important context for research use in rulemaking involves the professional positions of the regulatory policymaking participants, as the networks and relationships of researchers, policymakers, and intermediaries can influence the extent and nature of research use in decision making (Cooper & Levin, 2010;Farley-Ripple & Grajeda, 2019;Ng-A-Fook et al., 2015). Table 3 indicates the professional positions of non-federal negotiators -that is, negotiated rulemaking participants who were not representing the Department of Education -in the five casestudy rules. The table distinguishes between primary and alternate negotiators, as these different types of participants play different roles in negotiated rulemaking. Generally speaking, alternates participate in consensus voting and debate only when the primary negotiator is absent or otherwise unavailable (Negotiated Rulemaking Committee, 2018). However, alternate negotiators have sometimes served as active participants in higher education rulemaking, 5 and as such, their professional backgrounds are important to include in this analysis. As indicated in the table, by far the most common professional positions of non-federal negotiators were administrators working in individual higher education institutions. Representatives of advocacy groups were also well represented, as were attorneys of various types and administrators working in higher education systems or networks. The fact that so many negotiated rulemaking participants were not employed by associations or formal networks is noteworthy, because such organizations may conduct research or serve as intermediaries between research and policymaking. But even when not serving as negotiators, associations provided support to individual administrators serving on negotiated rulemaking teams. For example, a respondent who was employed by an individual institution described receiving "information" and "research" from a Washington, DCbased association while participating in one of the case-study negotiating committees. Another respondent observed that institutional representatives tend "to stick with the talking points their associations send" during negotiated rulemaking, and that if research findings are not included in those talking points, research is less likely to be introduced by those negotiators.
Also noteworthy is the fact that very few college faculty participated in the case-study negotiated rulemakings. Higher education faculty often engage in research as part of their job responsibilities, yet as Table 3 indicates, college administrators have participated in negotiated rulemaking at a much higher rate than faculty. This observation was corroborated by a number of interviewees, who indicated that institutional personnel who participate in rulemaking are often not those who conduct research and are not likely to confer with their colleagues who do conduct research about their participation in the rulemaking process. As one interviewee said, "I think educational institutions are participating in rulemaking as like an industry that's being regulated. It's really the business side of educational institutions that are showing up to the table to do that." Sources of research used. Table 4 displays the sources of research discussed in each final rule. 6 The 2014 Gainful Employment rule, that rule's 2019 rescission, and the Teacher Preparation rule all cited at least 10 academic journal articles in the final regulations -10 for the 2019 Gainful Employment rescission, 12 for the 2014 Gainful Employment rule, and 20 for the Teacher Preparation rule. This is in contrast to REPAYE, which cited no academic journal articles in its final regulation, and the Borrower Defense rule, which cited one. Although the 2019 rescission of the Gainful Employment rule discussed research on a larger percentage of pages than any other casestudy rule (as shown in Table 2), that regulation cited fewer academic journal articles than either the 2014 Gainful Employment rule or the Teacher Preparation rule. The only source of research that was reflected in every case-study rule was federal government research, either from the Department of Education itself or another federal division. Reports from non-profit organizations (including think tanks and associations) were cited in all of the case-study final rules except REPAYE. Respondents offered some insights as to why government and non-profit organization reports were discussed prominently in the case-study regulations while research published in academic journals was cited less frequently. One reason was politics. A long-time observer of higher education rulemaking remarked that public officials tend to seek out research that will substantiate their policy positions, 7 and often ideological think tanks 8 produce reports on current policy issues that may be useful in that regard. Moreover, a federal official said that government actors are less likely to care about the source of the research as they are about the findings. This respondent said: In government you don't care who found the finding. You just care that it's true. And so if we're showing that the majority of people going to a … for-profit cosmetology program are defaulting on their loans, I don't really care that [a particular researcher] found that out to be true. What I care is that it's true.
Although the federal official quoted above appeared to be conflating research findings with truth, a crucial point of this respondent's statement is that sources and authors of research tend to be less important to policymakers than a study's findings and what the research claims to be "true." Another reason for the less frequent citation of academic articles in higher education rules relates to how academic reports are written. That is, the language often used to describe research methods and findings in academic articles can make them less understandable to an audience that is not well-versed in research methods or academic jargon. Also, academic articles do not always provide specific policy recommendations or examples of how research findings might be used in practice. As a respondent who worked for the Department of Education explained: I think a lot of studies in academic journals are highly specific to whatever they are studying, and if the researchers themselves have not kind of said something about how they might apply to a federal regulation situation, it's quite possible they'd never be noticed as possibly relevant.
By contrast, think tank reports are more geared toward a policy audience. One negotiated rulemaking participant said that think tanks "drill [research findings] down into these bullets. Boom, boom, boom." Another negotiated rulemaking participant indicated that think tanks sometimes send their reports directly to their contacts in the policy community, whereas such direct contact with policy actors is less likely in cases of "more scholarly, academic reports where the folks aren't as connected to the policymakers and advocates operating on the ground." As Table 4 also indicates, more research published in academic journals appeared in the Teacher Preparation rule than in any other case-study rule; indeed, the Teacher Preparation rule was the only case-study rule for which academic journals were cited more than federal government reports. One federal official offered an explanation for this finding: In Teacher Prep, we had looked at the published research. There was kind of a flurry of publications around 2011 or 2012… looking at trying to develop value-added measures for teacher prep programs and kind of the validity of those measures… There's a ban on the department collecting any student unit record data 9 … So we couldn't do anything like that ourselves. We couldn't run any models. We couldn't somehow get data from teacher prep programs or from states or anything like that because we're not allowed to have that data.
The same official said that the federal government had access to data that government researchers could use for issues related to the Gainful Employment and Borrower Defense rules -such as data and Lubienski (2015) recognize that some of these organizations may serve as "intermediaries that seek to facilitate the use of research by policymakers -albeit often only selected research that supports a particular agenda" (pp. 3-4). 9 This ban was included in the HEA following its reauthorization in 2008 (Miller, 2016). This issue is discussed in more detail in the section of this article below on barriers to research use in higher education rulemaking.
from the National Student Loan Data System -so federal officials did more of their own analyses in those rulemakings. These cases illustrate how the sources of research used in policymaking can have more to do with timing and available resources than a desire to use research from particular sources.
Other factors. Respondents also discussed factors other than research that they have observed as influential in higher education rulemaking. More frequently than any other factor, interviewees identified the use of data, statistics, and mathematical calculations apart from a systematic study as playing a role in the development of higher education regulations. This included data from the Bureau of Labor Statistics and the Integrated Postsecondary Education Data System as well as gainful employment and other Department of Education data. Also, non-federal negotiated rulemaking participants sometimes discussed data they had gathered themselves, such as the results of a poll conducted of an association's membership or Department of Education data that non-federal negotiators used to conduct their own statistical analyses.
Respondents also indicated that examples were presented in the rulemaking process. As one negotiated rulemaking participant explained, "Honestly, there isn't always strong quantitative information to support the issues, but sometimes you cite examples." Respondents also discussed how more subjective and personal factors -including experiences, beliefs, emotions, and political preferences -have influenced the higher education rulemaking process. One negotiated rulemaking observer said, "I would not say research was informing those conversations; there was much more of experience and sort of gut feelings, like philosophical feelings." Another respondent described how negotiated rulemaking participants' own past experiences played a role in the process: There's a lot of experience around those tables. I've been doing this for [more than 25] years, so I have, I think, a pretty broad perspective on what happens where the rubber meets the road. In one sense it's anecdotal, but it's from a lot of experience and working at different types of schools with all different types of students. Table 2 indicates that experience was discussed on a larger proportion of pages in the Gainful Employment rescission than any other case-study rule. This makes sense given that the gainful employment policy had a relatively long history, which included the 2011 version, a negative court action regarding that rule (see Nolter, 2012), and the development of the 2014 version. The 2019 rule provided reasons to justify the rescission of that rule (Program Integrity, 2019). Thus, the rescission had to reckon with a lot of past experience, particularly the rule's own regulatory history.
As an observer of these and other higher education rulemakings said, "A lot of times it's history and it's expertise. It's not data" that drive the rulemaking. Some interviewees also discussed how the law has influenced decision making in higher education rulemaking. As the authorizing statute, the HEA served as a key consideration in the development of these regulations, as evidenced by numerous citations to the Act in final regulations as well as statements from some interviewees. As one respondent who worked for the federal government said, "I think one constant question is legal authority, and of course any rulemaking must fill in the gaps of the statute, not contradict the statute." Besides the HEA, other laws and legal matters were sometimes considered in the rulemaking process. For example, participants in the Borrower Defense rulemaking said there was consideration of consumer protection and trade practice laws in the development of that regulation as well as its subsequent modification. Other legal issues discussed during the creation of the case-study rules included statutes of limitations, burdens of proof, and in the words of one negotiated rulemaking participant, information about "judgements and settlements that had occurred between students or student advocate groups and the for-profit sector."

Barriers to Research Utilization in Higher Education Rulemaking
Respondents also discussed their observations about barriers to using research in the higher education rulemaking process. Several barriers were identified repeatedly by respondents, each of which is described in this section in order of frequency with which they were reported. Although these barriers are presented separately, these descriptions show that there was often overlap between two or more of them.
Time constraints. More than three quarters of interviewees discussed issues relating to time constraints in higher education rulemaking as a barrier to using more research in the process. These shortages of time were present in different aspects of the rulemaking process. With regard to negotiated rulemaking, some respondents said the relatively short amount of time in which negotiated rulemaking occurs presented a barrier to research use because conducting original research can take much longer than the time allotted for negotiated rulemaking to take place. As one lending industry representative remarked, "Research, you don't do overnight." Relatedly, when data or research was presented during negotiated rulemaking, participants typically had little time to review and consider it. One public university representative said: A unique part of the negotiated rulemaking process is that you don't usually receive what you're asking for in enough time to actually use it… And so we got most of that data either after the session was over, so we would have to remember to go back and talk about it at the next session, or even after the rulemaking had concluded in its entirety.
Another way time constraints presented a barrier to research utilization was when research was presented or considered during later stages of the rulemaking process. Several respondents indicated that the later research was presented in the process, the more the policy and ideas behind the rule were already crystalized, and therefore it was less likely that research or other factors would prompt changes at that point. For example, one federal official said that by the time meetings with OMB personnel were held, the government's "ideas are pretty set in stone at that point, and so it would be hard for that kind of research to really have a big impact." Desired data not available. Also reported by more than three quarters of respondents was the fact that data desired by rulemaking participants were often not available. Several respondents said they observed negotiated rulemaking participants make requests for data from the Department of Education, and that those requests sometimes went unfulfilled or were fulfilled in an untimely manner. 10 One negotiated rulemaking participant described this issue: I felt like the difficulty with it was it took the Department of Education a long time to respond to data requests from the committee. I think there's a lot of reasons for that … but that was a challenge sometimes. So if certain people on the committee had requests, it did take a while to get those requests answered.
Notably, the fact that desired data were not readily available is related to the issue of timing described above: if it takes a long while to receive requested data, then there is less time for stakeholders to conduct research with that data during the rulemaking process. Also, the statutory prohibition on the Department of Education from collecting studentlevel data was discussed by several respondents as a barrier to research use in higher education rulemaking. The HEA states that, with some limited exceptions, the government shall not "authorize the development, implementation, or maintenance of a Federal database of personally identifiable information on individuals receiving assistance … or otherwise involved in any studies or other collections of data under" the Act (Higher Education Act, 2018, § 1015c). Some respondents said that this statutory prohibition was a barrier to research use in rulemaking because it prevented researchers from having access to relevant data. One negotiated rulemaking participant said that this prohibition meant that the department "doesn't have perfect data… So they don't have full information about every borrower. They don't know which borrowers all have private loans for example." A respondent who worked for the federal government said that this prohibition contributed to "a lack of available data … in the higher ed space." Highly politicized process. Policymaking tends to be highly political (Minor & Mutamba, 2017), and higher education rulemaking is no exception. About two-thirds of respondents indicated that the fact that higher education rulemaking is very politicized can present a barrier to the use of research in the process. One way that this barrier manifested itself was when policy actors were so committed to their political positions that they disregarded research that was contrary to these positions. A negotiated rulemaking participant said that in higher education rulemaking, "folks come to the table with very preconceived notions about what they want out of this process and what they're willing to accept." This same respondent observed that "not only is [research] produced sparingly [in the process], but when it is produced, it's mostly ignored." Another way that politics presented a hindrance to research use was that the political nature of higher education rulemaking caused some participants to be mistrustful of research that was presented in the process (see also Natow, 2020). One federal official said that individuals who attended OMB meetings were often "very interested parties… So the evidence they present is sometimes viewed through a very suspect lens." Another respondent explained how sometimes this skepticism is warranted: There's a whole cottage industry -it used to be concentrated in Washington, thanks to the Internet it's now widespread across the country -of what I call think-tanky stuff, which is nonpeer reviewed, nonrigorous, oftentimes very facade reportage with a preexisting point of view… So there is an ideological tilt to most of the stuff that I think does pass for scientific literature and even those, again, are highly susceptible to selective usage.
Thus, the barrier to research use presented by politics was more complicated than simply preventing the consideration of research in higher education rulemaking. Rather, due to politics, research presented in the process was often not trusted or relied upon by policymakers.

Mismatches between the research and policy communities.
Nearly two-thirds of respondents made statements demonstrating some fundamental differences between the research and policy communities, and they illustrated how these differences presented a barrier to research use in higher education rulemaking. Some of these mismatches overlap with other barriers to research use. For example, the time constraints barrier discussed above reflected a mismatch between the research and policy communities regarding timeframes: the policy world moves quickly, while rigorous research takes more time.
Also, research findings have sometimes been presented in ways that are not conducive to being understood by a busy policy audience. As a long-time higher education policy observer said, "Academics speak with great nuance and oftentimes in very turgid and highly technical, very specialized language that doesn't always -that the regulators can't necessarily fathom, let alone capture." Moreover, a negotiated rulemaking participant said that policymakers are more likely to appreciate "high quality talking points... that very busy people can digest, and articulate, and analyze." Academic research does not always present findings in such a format. A respondent who worked for the Department of Education said, "classic academic research is hard for people who are not in the field doing the research to see the connections. It's helpful for somebody to sort of find that for them." These mismatches are illustrative of the "two communities" perspective in the literature of research utilization, which posits that researchers and policymakers occupy separate "communities," and that this is a reason why more research is not used in policymaking (e.g., Caplan, 1979;Ness, 2010). Interestingly, respondents provided statements that demonstrated how this separation of the research and policy communities has not only occurred between organizations who participate in the rulemaking process, but also within them. While higher education institutions were represented on all of the negotiated rulemaking committees, it was not typically the research divisions of those institutions that participated. Rulemaking documents show that, although researchers and faculty members did participate in negotiated rulemaking occasionally, far more often it was financial aid or business officers, senior executives, and other non-research administrators. Also, for at least one of the case-study rules, the research and policy personnel within the Department of Education did not work closely together on the rulemaking until toward the end of the process. A respondent who worked on that rule as a Department of Education official said that one of the department's other divisions (the Policy and Program Studies Service) was "very research oriented." But that division provided research support in the development of the rule only "closer to the end of that process." The same respondent said, "It took us forever to figure out what they were." These data illustrate how the two-communities phenomenon may exist within a single organization -whether the Department of Education or a university -where the work of personnel active in policy and the work of those active in research do not routinely intersect.
Lack of government capacity. About one-fifth of respondents indicated that the federal government has lacked sufficient capacity to respond to data requests and to stay informed about recent research, and that this lack of capacity has presented a hindrance to research use in rulemaking. As explained above, mere data and statistics do not in themselves constitute research, because research involves a systematic inquiry conducted according to a methodological plan (Penuel et al., 2016). But access to relevant data is an important prerequisite to a researcher's ability to conduct an empirical study. Thus, a lack of government capacity to provide relevant data hinders researchers' ability to conduct studies relevant to the topics at issue in higher education rulemaking.
As mentioned above, stakeholders made data requests of the department that often went unfulfilled during the rulemaking process. Some interviewees said that this may be at least partially due to a lack of capacity within the government to respond to such requests. 11 One respondent who had worked for the Department of Education said, "Right now, the capacity constraints at [the department's Office of Federal Student Aid] are very significant, and getting a data run is not easy." In a more nuanced explanation, a respondent who had worked for the federal government said: I was going to say there's not a lot of expertise in the federal government to really incorporate [research], but that's just not quite what I mean. I think there's not enough expertise relative to the size of the magnitude of work that they have to process. There's sort of like a lack of capacity to be able to stay on top of the body of literature that's relevant for any given issue.

Discussion and Conclusion
This multi-case study was the first to examine the issue of research utilization in federal higher education rulemaking, an important educational policymaking process that has substantial implications for student financial aid, institutional accreditation, and other important policy matters affecting higher education. Federal law requires agencies to conduct regulatory assessments and cost-benefit analyses (Belfield et al., 2018), and to have a "reasoned basis" for their regulations (APSCU v. Duncan, 2012, pp. 1, 30-31). By examining the cases of five higher education regulations, this study found that research was used in the creation of these rules and that references to research appeared in the texts of their final regulations. However, factors other than research -that is, data apart from a formal study, experience, examples, law, and beliefs -were also influential and were discussed on a greater proportion of pages than research in all of the case-study final rules. Also, government research -whether the Department of Education's own research or that of another federal office -was the only research source cited in all five case-study rules.
This study also found that the contexts in which particular rules were created shaped the prevalence and sources of research that were used in the rules' development. Case-study rules that had a history of litigation or were otherwise controversial (i.e., the two Gainful Employment rules and the Teacher Preparation rule) discussed research on a greater proportion of pages than the other case-study rules. The three controversial rules also cited at least 10 academic journal articles each, while very few academic journals were cited in the other case-study rules. Statements from interviewees suggested that these patterns were related to the histories and subject matters of the regulations: the Gainful Employment rules had a lengthy history of controversy and litigation, the Teacher Preparation rule was also controversial (see also Stratford, 2014), and both policies involved noncompliance penalties that could theoretically jeopardize an institution's eligibility to receive federal student financial aid funds. Therefore, federal officials anticipated legal challenges for these rules, and they attempted to protect the rules from legal jeopardy by using relatively more research in the development of those regulations. Also, the Department of Education did not have access to student-level data while developing the Teacher Preparation rule, and around that same time, a number of relevant studies had been published in the academic literature. Consequently, the Teacher Preparation rule was the only case-study rule to cite more academic journal articles than government reports. Finally, the positions and networks of rulemaking participants were another important context for research use in this process. Although considerably more negotiators in the case-study rules were employed by individual colleges than by associations or advocacy groups, interested higher education associations sometimes gave support, including information about research, to negotiators, indicating these organizations' role as intermediaries in the rulemaking process.
Several barriers obstructed the use of more research in higher education rulemaking, including time constraints, the unavailability of desired data, the highly political nature of the process, and the lack of government capacity to conduct research and respond to all data requests. Certain disjunctions between the research and policy communities also impeded research use in the process. One disjunction involved timing incongruities between researchers and policymakers: policymakers attend to policy problems more quickly than research on those problems can typically be conducted. Moreover, policymakers tend to prefer reports that provide concise and distilled points and recommendations, whereas academic researchers often write in ways that are more nuanced, highly technical, and contain academic jargon. Although higher education rulemaking proceedings, and negotiated rulemaking in particular, provide a unique opportunity for research organizations to work closely with policymakers in the policy development process, this study found mismatches between researchers and policymakers that are reminiscent of the "two communities" perspective (e.g., Caplan, 1979;Ness, 2010), and these gaps existed despite the fact that higher education rulemaking brings research-oriented institutions and policymakers together. Indeed, this study discovered incidents of research-policy disjunctions even within an organization, such as where universities were represented on negotiated rulemaking committees by non-research administrators, or when policymakers at the Department of Education did not work with one of the department's research offices until near the end of a rulemaking procedure.
The findings of this study suggest that the veracity of the "two communities" perspective should not be dismissed, but the theory should be modified to better account for contextual and organizational influences on the willingness and ability of researchers and policy actors to make use of each other's work. Some prior literature recognized that context plays a role in whether and how research gets used in decision making (Farley-Ripple et al., 2018;Head, 2015;Holzer et al., 2007;Levin, 2013;Nutley et al., 2007). This study of research utilization in higher education rulemaking contributes to that literature by illustrating how localized elements of context can influence the extent to which research is used, the sources of research used, and the barriers to research use in policymaking. These elements of context include the subject matter of a policy (Does research exist on this topic, and if so, who conducted the research?), the resources available to the policymaking entity (Does this agency have access to relevant data? Does it have the capacity to conduct research or respond to data requests?), the history and possible consequences of the policy (Does this policy have a contentious history? Are lawsuits likely to result, and if so, is the policy likely to be jeopardized by not having an evidence-based foundation?), organizational structure and relationships (Is the work of policy personnel separated from the work of research personnel in this organization? Are policy actors and researchers in regular communication with each other? Do policy actors' professional networks contain "knowledge brokers" [e.g., Rodway, 2015, p. 4]?), and even timing (Has there been any recent research that may be relevant to this policy?). Contexts such as these can influence whether, how much, and what kinds of research are used in the creation of policy, and may also lead members of the often-separate policy and research communities to collaborate.
This study has implications for policy and practice that may help to improve the use of research in rulemaking and other educational policymaking processes. One recommendation is for the Department of Education to include a place for researchers on negotiated rulemaking committees (see also Kelchen, 2019). Respondents noted that although this has not occurred often, the times when a researcher was present at negotiated rulemaking were useful in helping the participants make sense of the data and research that were considered during negotiations. Just as the department selects representatives from a number of different stakeholder groups to participate in negotiated rulemaking, it should also appoint experienced researchers as primary and alternate negotiators on future rulemaking panels. Moreover, Congress should eliminate the part of the HEA that bans the Department of Education from gathering student-level data, and that data should be made available to government and outside researchers. This recommendation was made by some of this study's respondents, and it echoes similar calls from many in the policy community (Kreighbaum, 2017;Miller, 2016). Congress should allocate additional resources to the Department of Education to enhance its capacity to conduct research and respond to data requests via hiring additional research staff, upgrading relevant technologies, and providing enhanced training for staff on research and data use (see also Henig, 2008).
For their part, researchers should strive to study policy-relevant topics and include specific policy recommendations in their reports (see also Henig, 2008). Researchers may also provide more visibility to their findings by authoring opinion pieces and other concise publications that are suitable for policy and practitioner audiences. Higher education leaders can assist in this endeavor by providing editorial and public relations support for researchers at their institutions who seek to publicize policy-relevant findings. When institutional executives and other non-research administrators are preparing to participate in policymaking activities, they should consult with researchers within their organizations who have conducted studies on relevant issues. For example, a university administrator planning to participate in negotiated rulemaking should meet with education faculty or other researchers at their institution to discuss how research findings might be relevant and useful to the topics that will be the subject of the rulemaking. To incentivize researchers to connect more with the policy community, higher education leaders should reform faculty and research incentive structures to reward research that targets policymakers (see also Henig, 2008). Higher education institutions might also work with government agencies to provide training for government staff on research methodologies.
Despite persistent incongruities between the research and policy communities, one goal that researchers and policymakers often share is the desire to improve the use of research in policymaking. By illuminating the extent of and barriers to research use in higher education rulemaking, the findings presented here can help researchers and policymakers design effective strategies for bringing more and better research into policymaking for the purpose of creating policies that improve education.