Measuring , Rating , Supporting , and Strengthening Open Access Scholarly Publishing in Brazil

This study assesses the extent and nature of open access scholarly publishing in Brazil, one of the world’s leaders in providing universal access to its research and scholarship. It utilizes Brazil’s Qualis journal evaluation system, along with other relevant data bases to address the association between scholarly quality and open access in the Brazilian context. Through cross tabulation among these various data sets, it is possible to arrive at a reasonably accurate picture of journals, systems, ratings, and disciplines. The study establishes reliable measures and counts of Brazilian scholarly publications, the Open Access Scholarly Publishing in Brazil 2 proportion and types of open access, and journals ratings and by disciplinary field. It finds that the better the Brazilian journal, the more likely it is to be open access. It also finds that Qualis ranks Brazilian journals lower overall than the international journals in which Brazilian authors publish, most notably in the field of the biological sciences. The study concludes with a consideration of the policy implications for building on the country’s global leadership in open access to strengthen the quality of its global contribution to knowledge.


Brazil's Journal Rating System
Brazil has a research university system composed by a minority of public institutions accounted for the majority of the scientific research produced in the country.This Brazilian federal system and the national higher education strategy have been focused on improving scientific research quality by investing in public institutions programs, while private institutions are responsible for absorbing the expanding enrollment in undergraduate courses.
As observed by Alperin (2013), the Brazilian public higher education system has solid graduate programmes with qualified professors and produce a significant amount of research in a sufficient internal market to exchange ideas in locally academic journals.National policies have incentivised research and scholarship at these public universities, increasing the number of doctorate degrees, students and professors collaborating with international universities.The country consequently has shown considerable growth in the number of scientific papers published, having a large body of journals, therefore can be considered as a regional leader in terms of scientific publication in Latin America.
The Brazilian government funds the production of academic journals and researchers, increasing the number of published journals and attracting visibility to the Brazilian science.The two agencies in charge of allocating the majority of research funding, CAPES (from the Portuguese acronym for the Federal Agency for Support and Evaluation of Graduate Education) and CNPq (the National Council for Scientific and Technological Development) have focus on strengthening graduate education and national research culture.The national agencies have also implemented a comprehensive graduate programme valuation scheme.CAPES is also the agency responsible for the sistematic evaluations of the Brazilian graduate programmes, assigning a 7-point scale score based on criteria such as publishing in high-prestige journals, receiving international scholars, producing patents and organising international conferences.The Serials Qualis rating system established by CAPES is a crucial part of this evaluation system.
The Serials Qualis (also known as just Qualis) is the name adopted for the set of procedures used by CAPES for stratification the quality intellectual output of graduate programs.It evaluates and rates global and national journals with the purpose of assessing the Brazilian graduate education publication system.It provides a rating list of scientific journals that can be used by graduate programs for evaluating dissemination possibilities of their scientific production.The premise of Qualis is to estimate the quality of the articles based on the analysis of the quality of the scientific journals where they have been published.Qualis employs multiple committees of Brazilian researchers to rate periodicals from both Brazilian and foreign publishers, grouped into nine disciplinary fields (agricultural sciences, bioscience, health sciences, engineering, exact sciences, arts and linguistics, human sciences, social sciences and multidisciplinary disciplines).The function of Qualis is to be an evaluation tool for institutional scientific production of authors linked to the formal system of Brazilian graduate programs.The scientific production is assessed indirectly by ratings in a scientific journals list, categorized and prioritized by the ad-hoc committees, therefore we have to consider contextual variables of theses evaluation procedures.Qualis represents a complex process evaluation and it is not intended to be a generalized expression of the journals quality.This is not Qualis purpose, although it may be a significant and representative sample of the Brazilian publishing potential.We must then consider the scope and meaning of the hierarchies established in the classification of journals in the Qualis list, thus the same journal published within an area of knowledge can generate a relevant surplus, receiving and publishing articles from authors from other areas of knowledge without incurring inconsistency.
Our analysis of open access journal quality both reviews and relies on aspects of the CAPES Serials Qualis rating program.While not without its critics (Rocha-e-Silva, 2009), Qualis ratings make it possible to see where open access serials fit into the Brazilian scholarly publishing landscape.
One serial can be rated in several areas, with different ratings in each, expressing the serial's relative value in each area of evaluation.Qualis classification criteria vary somewhat according to field, with some common to all fields, mainly those based on citation indicators, such as the Impact Factor and the Scopus h-index (Figure 1).Rating criteria also include the number of indexing and abstracted services in which the journal is included, number of different author affiliations, presence of international co-authors, the use of blind peer-review system, and wide serial dissemination (Capes, 2013).This is consistent with other Latin American Science Councils, which are increasingly relying on bibliometrics in assessing the research quality of journals for their respective nations (Alperin, Fischman, & Willinsky, 2011).
Despite the fact that area coordinators have autonomy to establish their own criteria, all Qualis committees have to observe three generic proportion rules when rating serials, defined by the CAPES Technical-Scientific Council.The main rule is that the percentage of A ratings (A1 and A2 combined) in a given area should be no more than 25% of the total serials in that area, and the percentage of ratings A1 should be less than the percentage of A2.The total percentage of A1, A2 and B1 ratings in a given area should be no more than 50% of total serials in the area (Andrade & Galembeck, 2009;Mafalda et al., 2015).Combined, these three rules mean than no more than half the journals can be rated in the top three categories, and the higher categories have to be more selective than those below it.

Research Methods
To establish the number, proportion, and quality of open access Brazilian journals, we first sought to assess the comprehensiveness of the Qualis list by comparing it with other serials directories and bibliographical database indexes (Table 1).We collected data from the institutions websites and created a dataset containing several journals characteristics.The main source of data was WebQualis, the online web-based system from CAPES, used to access data from Serials Qualis.The CAPES evaluation system produces a list of periodicals from Brazilian and foreign publishers with ratings attributed by committees composed of academics from all fields of science.The Qualis rating classes are in an 8-point grading scale, with letters and numbers, A1 being the highest grade, attributed to top journals in each evaluation area, followed by A2, B1, B2, B3, B4, B5 and C being the lowest level with zero weight, attributed to serials considered non scientific journals.For our analysis purpose, we coded the ratings classes with numbers from 8 to 1, 8 corresponding to the A1 level and 1 to the C level.If the serial was not in Qualis the serial code rate was defined as 0 (zero).
This analysis was conducted from February 2013 to October 2013, and as such it captures a moment in time.In 2013, Qualis rated 24,406 serials, including both active and non-active titles.Of these, 32% were Brazilian and 68% non-Brazilian.Since a serial can be rated in multiple disciplinary areas (once by each area committee), the number of Qualis ratings was 87,178 (of which we used the highest rating in our analysis).
The objective of crossing data from several datasets was to establish a reasonable estimated number of OA journals in Brazil.First we face the question of how many serials are published in Brazil and we have to deal with the various challenges at arriving at the total number of periodicals published in the country.In some directories and indexers websites data were unavailable for download, so we contacted these institutions directly to obtain the data.After collecting data from the institutions websites, we combined all datasets to the list of journals extracted from Webqualis.As data extraction criteria for world serials and union of databases, we used variables such as country, institution, name, language, ISSN and e-ISSN.Three datasets were created, one with Qualis ratings, one with all Qualis serials (Brazilian and international serials rated in Qualis) and the third one with Brazilian serials only.This last dataset contained periodicals from all datasets joined by ISSN and eISSN (from Qualis plus other directories and indexers).
Next steps were data cleaning and browsing websites to confirm journals online.Data cleaning was done with the junction of print and online versions of the same serial (by ISSN, e-ISSN, Name and Publisher).The indiscriminate use of ISSN and e-ISSN numbers for the same journal in different datasets duplicated the journal on the final database.Extensive checking and data cleaning was performed, with the exclusion of duplicated journals and the reunion of the paper and online version of the same version in single cases.Data from Serials Qualis had several cases with two different ratings for printed and online versions of the same journal, and in this case, the higher value attributed was selected.
There were several ceased or merged periodicals on datasets.We excluded ceased, merged, suspended and non-Brazilian serials.Additionally, we excluded publications considered as nonperiodicals (proceedings, magazines, directories, newspapers and any other type of non-periodical).Several serials did not have ISSN or e-ISSN numbers.ISSN numbers and serials without ISSN were identified through a web-search and those not considered journals were deleted from the database.After all the process of cleaning and checking data, we created variables from others, to characterize each serial (highest rating, sum of ratings, maximum ratings, presence in indexers, etc).It has been observed that Qualis database contains a number of inconsistencies (Lopez, 2001).We found, for example, duplicate ISSNs, in which two different journals were registered with the same ISSN in Qualis (for examples, see Appendix A, Figure A1).The Qualis database also contained several cases in which the print and online versions of the same journal received different ratings, and in such cases (as with a title receiving ratings in multiple areas), the higher value was used.For example, the print edition of Cadernos de História da Educação, published by Universidade Federal de Uberlândia, received an A2 rating, while the online version was ranked B3.In 2013, the ratings were both changed to A2 in education, but a discrepancy remains in the journal's rating in the interdisciplinary area (where the print version is rated B1 and the online version B2) (Appendix A, Figure A2).
We sorted journals based on Serials Qualis ratings.Qualis periodicals list reflects a number close to the real number of periodicals currently existing in Brazil.However, Qualis journal list is incomplete, since it is only updated annually and may fail to include a title for a number of reasons.A web-based research performed in other databases completes the list of journals obtained from Qualis.
To study the Qualis rating of open access journals, a random sample of 450 Brazilian journals was created from the Qualis list.The sample was stratified, with 50 randomly selected journals from each of the eight Qualis rating classes (A1 to C), plus 50 journals that were not found in Qualis.These 50 journals not listed in Qualis were also randomly selected from a list of 1,143 serials obtained from other serials directories and bibliographical database indexes, presented on Table 1.
Whether a journal was OA, had an embargo, charged submission or processing fees was checked manually by the researchers.(While DOAJ and Ulrich identify open access titles, their coverage of Brazilian journals was inadequate).For each, we checked the first four open access features described in the Open Access Spectrum (Table 2), readers' rights, reuse rights, copyrights and author posting rights.
To check the Platform Systems most used by Brazilian Journals and verify the proportion of open access titles supported by the journals use of grant-supported software systems, we crosschecked information from original databases.We made a Venn diagram to compare serials in SciELO a successful 15-year Brazilian project of OA initiative engaged with the international movement of OA to scientific information (Packer & Meneghini, 2007), with serials from RedALyC, another OA journal publishing platform based in Mexico.And serials using the most used open source software Open Journal Systems (OJS), developed by the Public Knowledge Project, with support from the Brazilian Science and Technology Institute (IBICT).

Table 2 The Open Access Spectrum Used to Assess the Journal's Open Access Status
Readers Rights

A. The Database Count of Brazilian Serials
While the Qualis dataset lists 7,825 Brazilian serials (32% of all the serials in Qualis), only 6,634 titles proved to be valid journals, after removing duplicate, ceased, merged, suspended serials and non-periodicals publications (Table 3).This is twice the number of journals listed for Brazil in the Latindex Directory, the region's most comprehensive index service, and one in which Brazil accounts for 16%.Only 269 Brazilian journals were found in Scopus and 118 in the Web of Science (both around 1% of their total journal count), reflecting the much-noted under-representation of Latin American journals in external databases (Alperin, 2014;Cetto & Alonso-Gamboa, 2008).By comparing the Brazilian journals listed in Qualis and in the other indexes and services, an additional 1,143 journals were identified that were not in Qualis (Table 4).Of these, only 405 (35%) are available online, with the majority of them new journals, while the remaining 738 (65%) are only to be found in print or CD-ROM formats.This meant that according to relevant database sources, Qualis was perhaps missing 57 active and current journals, which is less than 1% of its filtered collection of 6,634 titles.
Qualis is clearly the most comprehensive, if not entirely complete, source for judging the state of Brazilian scholarly publishing.By the same token, it is fair to say that CAPES is doing a good job of reviewing the nation's scholarly journal literature.

B. Brazilian Serials Ratings
The Qualis ratings of Brazilian journals places 75.8% of the titles in the A and B categories, with less than 2% of the journals judged to be in the highest category of A1 (Table 5).The Qualis ratings fall well short of the CAPES guidelines (discussed above), which permit 25% to be awarded A's -with the 2013 ratings at 5.0% -while allowing 50% to be awarded to A1-B1, compared to the 11.5% awarded.The message to the journal community is that there is plenty of room to grow in terms of improved quality.At the same time, the Qualis committee found that less than a quarter of the Brazilian titles fail to meet the basic requirements of offering qualified editorial oversight, submission rules, authors affiliation, institutional affiliation, editorial council, bilingual description, ISSN number, periodical publishing, peer-review and publication by an educational institution or scientific society.
By assigning an 8-point scale to the Qualis A-C ratings (with A = 8 and C = 1), we were able to average ratings and compare Brazilian journals, which had an average rating of 2.8 (SD = 1.7), and international journals in Qualis which averaged 5.1 (SD = 2.1).The international journals in Qualis are those in which Brazilian scientists publish their research papers and, as such, are more likely to be higher quality journals in the Global North, with better research in Brazil being sent abroad to be published.We were also able to compare fields (Figure 2).Among Brazilian journals, mutltidisciplinary journals received higher ratings on average than other Brazilian journals.In the international comparison, the biggest gaps exists in the biological sciences, with the average for Brazilian journal dropping below 2.0 in their rating, compared to a 4.8 average rating for the international journals in the Qualis set.This suggest that the best work in biology goes out of country to be published, with room for a concerted effort on the part of the journals and federal agencies to attract, promote and serve well high quality submissions.

C. The Proportion and Quality of Open Access Journals in Brazil
To establish the proportion of open access journals published in Brazil, we used a stratified sample (described above) and manually verified what access model the journal was employing.In the resulting sample of 450 Brazilian journals, 68% proved to OA, a proportion similar to that found by Abadal in his analysis of Ulrich's database of journals (2012).The proportion deemed "open access" includes serials that were free to read online, whether with an embargo period for new issues (10%) or not (58%).The remaining 32% had some form of fee-based access.
However, if one considers the top two classes of journals (A1-2), the proportion that are OA (immediate and embargoed) jumps to 95%.In fact, we found that in all cases, the higher the rating class, the higher the proportion of serials providing free access to articles (Figure 3).The best journals in Brazil are far more likely to be open access.Without being able to determine cause and effect in this association, we can least confirm that open access is not a journal quality issue in Brazil, confirming what has been found for Ibero-American countries more generally, and without this level of analysis [18].What is also encouraging about the quality of open access journals is that this researh is being consulted by the public, with 20-25% of all article views coming from nonuniversity affiliated individuals reading for personal or professional practice reasons (Alperin, 2015).It should be noted that the open access journal copyright and self-archiving policies in the sample differed on PLOS's Open Access Spectrum (2013).Of the A1-B2 journal subsample, 85% (169/200) of the journals lack explicit self-archiving policies either on their website or in SHERPA/RoMEO (a database of such policies).Of the 31 with explicit archiving policies, 19 adopt the blue policy (authors can archive only the post-print version), ten the green policy (archive preand post-version of the article) and only two had a restrictive policies (yellow or white).
In addition, 30% of the journals rated A1-B2 do not appear to include information on copyright ownership in their website.The lack of information on copyright policies in journals appears to be widespread in Latin America, where a study of 292 journals from across the region found that a third of journals lacked any such information, although in some Latin American jurisdictions authors retain copyright by default [20].Only 20% of the journals in this sample made explicit mention of author's retaining copyright, while the remaining journals assert copyright of the published works.These numbers are again similar to the Cerda Silva and Lara study that found that 44% of journals assert copyright of the works they publish (Cerda Silva & Lara, 2011).
In relation to licenses, 112 (56%) of journals in the sub-sample of 200 do not specify a Creative Commons (CC) (or similar) license (including those for which there is no license information at all).These journals do not technically meet the full definition of Open Access, and may be said to be public access journals.The remaining 88 (44%) do specify a CC license, where CC-BY-NC is the most commonly found (22% of the journals in the subsample), followed by CC-BY (16%) and CC-BY-NC-ND (6.5%).Similarly, not all the journals that we coded as OA follow the same business or even access models.The Arts, for example, were more likely to use embargo periods for open access content (Table 6).Note: Includes sample from all Qualis levels and from those journals not in Qualis.Fields are not mutually exclusive and one serial can be rated in more than one field and was considered in the respective field.
As well, variations exist in the proportion of OA among the disciplines, with Agricultural Sciences, Engineering, Exact and Health Sciences exhibiting the higher proportion of OA journals in the sample, and for the sub-sample of top-tier journals.
The business models are also varied among the Brazilian open access journals.Only 8 (4%) of the highest rated journals (based on the sample of 200 A1-B2 journals) charged submission fees prior to the acceptance of a paper, while 16 (8%) levied "article processing charges" (APC), which is lower than the proportion of journals (26%) that charge in the international list of the Directory of Open Access Journals (Morrison et al., 2015).The business model of the Brazilian journals appears to be based on subsidies from partner institutions, grants from financing agencies, and subscriptions for the print version of the journals (Sandes-Guimarães & Diniz, 2014).

D. Platform Systems used by Brazilian Journals
Brazil's high proportion of open access titles is supported by the journals use of grantsupported software systems.Foremost among them is SciELO, which began in Brazil to support OA in 1998 (Packer & Meneghini, 2007;Packer et al., 2014).SciELO provides full open access to 1,030 journals from across Latin America, and more recently, South Africa (Packer, 2009).A second regional OA journal publishing platform is RedALyC, based in Mexico, having started a few years after SciELO with a focus on the Social Sciences.
RedALyC currently indexes close to 1,000 journals from across the Latin America.Both systems have served to professionalize journals from the region, and to strengthen the culture of openness in the region (Alperin, Fischman & Willinsky, 2011;Guedon, 2008).In a sense, SciELO and RedALyC have been among the most visible Latin American contributions to a globalized science composed mainly of international commercial publishers (Vessuri, Guedón & Cetto, 2013).
In addition, Márdero Arellano (Márdero Arellano, 2008) and Cyzyk and Choudhury (2013) describe proprietary and open source software systems that provide management and publishing platforms.Brazil was an early adopter of the open source (free) software Open Journal Systems (OJS), developed by the Public Knowledge Project, with support from the Brazilian Science and Technology Institute (IBICT), which translates, distributes, and provides training for the platform, which it has renamed SEER, from the Portuguese acronym (IBICT, n.d.).
In 2014, 1,747 journals were using OJS in Brazil or 26% of country's journals (Sandes-Guimarães & Costa, 2012).Among the journals rated A1-B2, the use of OJS increases to 54% (656/1,220).The use of OJS among journals also overlaps with their use of RedALyC and SciELO, suggesting the strength of the infrastructure for the national and global presence of Brazilian research through open access publishing systems (Figure 4).

Implications
In determining the extent and quality of open access among Brazilian journals, it first needs to be recognized how fluid and difficult it is to track the current state of the journal market.To Qualis' credit, and despite a number of inevitable data errors and inconsistencies, the Brazilian government has established what appears to be the most comprehensive list of scholarly journals for the country.The Qualis list, along with its rating system, enables a number of important points to be made about the place of open access in Brazilian scholarly publishing.The fact that the proportion of open access journals goes up with higher rated journals points, to above the 90% level for A1, points to the integral role of open access in quality publishing, as it increases global access and readership, for Brazilian journals.
At the same time, it needs to be noted that the Brazilian academic community can be judged to be unsatisfied with the quality of the country's journals.This can be seen in both its under utilization of the highest quality ratings and in its submitting work to higher rated journal abroad.In terms of what can be done to improve the perceived and actual quality of Brazilian journals, this study suggests a need for strategies to encourage better publishing within Brazil.
One such strategy may be to make more of the relative rarity of the APC among Brazilian open access journals.Consider Brazil's biology journals, to take a field with room for growth in quality, and the policy implications.The current rating disparity encourages researchers to publish abroad.In doing so, this work is either inaccessible in Brazil without a subscription or the work faces an APC fee prior to publication, given that APCs prevail among biomedical open access journals in the Global North, reducing the amount that can be spent on research in Brazil.In response to this situation, the Brazilian government may, for example, opt to work further with SciELO and the journal community as a whole to support Brazilian journal quality, enhance journal and article metrics, and improve publishing platforms.The goal would be to make Brazilian journals more attractive for the best research and a wider readership.Attracting better work is key to improving the quality of the journal, editors, reviewers, board members, and readership.This approach is in marked contrast to having Qualis send the signal to the academic community that Brazil has good journals but a smaller proportion of them than Brazilian researchers can find abroad.This only serves to perpetuate the current situation.This study establishes, for example, that the country's leading journals are committed as a whole to open access and are doing so largely without the reliance on APCs that otherwise reduces the amount spent on the actual research conducted in Brazil.
The most favorable assessment to international journals in Qualis list can be attributed to several factors.It could be occasionally case of deficit in the quality of national journals, but it seems more to be related with the effectiveness of policies that promote the internationalization of Brazilian science, encouraging the publication in international journals.Brazil is an example in Open Access with public universities funded by the government, creative journals with small budgets, no interest by international commercial publishers, pressure for impact in citations and the work of national agencies, but still has a research evaluation system pressuring researchers for publication in international traditional non-OA journals.The premise of measuring the quality of the articles based on the quality of the journals should be changed.We emphasize that the Brazilian educational policy should further enhance publications in open access, recognize and value its OA publishing system supported directly or indirectly by graduate programs and societies.
CAPES needs to recognize that establishing a rating scheme based on widely accepted scholarly criteria is only the first step in improving the state of scholarly publishing and what it gives back to the world.It has to seize opportunities to recognize and promote within the academic community the value of publishing in Brazilian journals.This need not hamper anyone's freedom to publish where they see fit.Rather, it enables authors to make informed decisions about where they place their work as a matter of extending the contribution of their work both globally and regionally, and in light of both scholarly and public use.It also enables CAPES to direct more of its considerable investment in Qualis to strengthening Brazil's existing global leadership in making universal access to research and scholarship.

Figure 1 .
Figure 1.Formally stated Qualis criteria for rating journals by nine disciplinary areas Source: Capes.Evaluation Board: Areas Documents and Committees.2013.Translated from Portuguese by authors.

Figure 2 .
Figure 2. Distribution of Brazilian and Non-Brazilian serials Qualis ratings by area

Figure 3 .
Figure 3. Proportion of Open Access journals published in Brazil by Qualis Rating NQ = Not in Qualis.

Figure A1 .
Figure A1.Screenshots of Qualis classification, with examples of the same journals with two ISSNs and of difference journals with the same ISSN.

Figure A2 .
Figure A2.Screenshots of Webqualis online Cadernos de História da Educação Qualis showing variations of ratings for the same journal.

Table 1
Data Sources Used to Assess Comprehensiveness of Qualis Journal List

Table 3
Number of and Proportion of Brazilian Serials by Major Journal Datasets a Filtered excluding non-periodicals.b Approximate.c Excluding duplicate, ceased, merged, suspended, and non-periodicals publications.

Table 4
Status of Brazilian Journals Not Found in CAPES Serials Qualis

Table 5
Number and Proportion of Ratings of Brazilian Journals in CAPES Serials Qualis

Table 6
Proportion of Open Access Journals by Full Sample and by A1-B2 Journals.