The role of arXiv, RePEc, ssrn and pmc in formal scholarly communication 1 Xuemei LI

Limitations

Scopus does not cover all research publications and it is possible that some important sources of publications are missing, for example perhaps book chapters and Chinese journals. In addition, the Scopus queries seem to return the majority SR citations but do not return all of them. Moreover, as the analysis of mathematics suggests, the results are likely to be due to some extent to the coverage and subject classifications of Scopus, so that comparisons between fields may be unfair if Scopus has wider coverage of one. The grouping of subjects into four broad disciplinary areas is an oversimplification to some extent. For example, biochemistry is important to PMC but was categorized within the natural sciences. The citing differences by subject, discipline and repository over the years are all based on citing documents rather than actual citations.

Most importantly, however, it seems likely that most citations to documents found in these repositories would not mention the repositories, especially for published articles, but would use a traditional citation instead. Hence, the figures reported here are likely to be substantial underestimates. In addition, articles seem to be commonly referenced in arXiv with identifiers instead of URLs, further undermining the figures, despite the use of the REFSRCTITLE command to catch some of these. Moreover, since RePEc does not have a single centralized archive, authors may also cite other archives that RePEc redirects them to.

Finally, the way in which the relatively new Scopus WEBSITE command indexes documents may have changed during the period studied, for example to be applied more comprehensively over time. Tests with this command suggested that it has been applied retrospectively to documents that were published long before it was introduced, however. For example, a search for WEBSITE(com) returned small numbers of (false) matches from as far back as 1977, before the web began and before Internet domain names were used.

Conclusions

In answer to Question 1, direct citations to arXiv, RePEc, SSRN and PMC in Scopus-indexed scholarly publications have all increased steadily from 2000 to 2013, although at different rates. The low initial number of citations to PMC is not surprising as it was launched later than the others, in 2000. The exponential growth in articles citing PMC after 2008 may have been caused by the NIH OA mandates since 2006. The small number of citations to RePEc may be caused by RePEc often linking to full-text versions of articles on external servers. The increasing number of citations to all of the SRs forms useful evidence that they all continue to be an important part of the scholarly infrastructure, despite publishers' apparent preferences for IRs. Hence, researchers in relevant disciplinary areas should continue to use them and policymakers do not yet need to encourage or plan for a wholesale migration to IRs. These findings are about the trends in uptake of the SRs, as evident from citations to them in published articles and are based upon the assumption that these citations reflect the much higher usage of them by researchers, even though the vast majority of articles found in SRs and cited in published work are presumably not cited via the SR. Perhaps most importantly, the findings assume that researchers cite a uniform proportion of papers discovered in SRs with SR references. This assumption is somewhat problematic because it seems possible that researchers have become increasingly likely to cite SRs to acknowledge their role or to help readers to find the articles.
In answer to Question 2, there are substantial disciplinary differences in citing the four SRs. At the broad disciplinary level, each repository was most cited within its own area. At the subject level, arXiv seems to be cited the most by mathematics, RePEc and SSRN are both cited most by economics, and PMC is cited the most by a group of biomedical subjects. Perhaps most importantly, however, the evidence of substantial use of each SR outside of its disciplinary area is valuable evidence of the utility of SRs for supporting this kind of wider uptake. Researchers seeking interdisciplinary audiences for their research can therefore use SRs for this.
The comparison between the SRs found some substantial differences. For example, 16% of the RePEc citations pointed to software components, showing that it is uniquely successful at hosting information about software, and other SRs might also wish to consider making provisions for hosting non-standard academic outputs. A total of 62% of the PMC citations pointed to gold OA journal articles, confirming that gold OA is particularly important for biomedical and life sciences researchers (Gargouri et al., 2012; Sotudeh and Horri, 2007).

In terms of methods, the new Scopus WEBSITE reference search facility has made it possible to investigate citations to online archives because it was possible to construct queries with few false matches. Nevertheless, it was not possible to identify all relevant citations with this method due to shorthand arXiv citation formats, which was only partially compensated for with the REFSRCTITLE command. Despite this and the differences in strategies of the different repositories in terms of whether to accept unrefereed articles and whether to present league tables based upon download statistics, arXiv, RePEc, SSRN and PMC clearly play an important and growing role in scholarly communication within their fields.

For future work, the WEBSITE reference search facility from Scopus can also be applied to other types of website, also following up previous studies of investigated web pages (Kousha and Thelwall, 2014) and YouTube (Kousha, Thelwall, and Abdoli, 2012).
References

Aguillo, I., Ortega, J., Fernández, M. and Utrilla, A. (2010), “Indicators for a webometric ranking of open access repositories”, Scientometrics, Vol. 82 No. 3, pp. 477–486.

Archambault, E., Amyot, D., Deschamps, P., Nicol, A., Provencher, F., Rebout, L. and Roberge, G. (2014). Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels—1996–2013, Science-Metrix, available at: http://science-metrix.com/files/science-metrix/publications/d_1.8_sm_ec_dg-rtd_proportion_oa_1996-2013_v11p.pdf

Bátiz-Lazo, B. and Krichel, T. (2012), “A brief business history of an on-line distribution system for academic research called NEP, 1998-2010”, Journal of Management History, Vol. 18 No. 4, pp. 445–468.

Bergstrom, T.C. and Lavaty, R. (2007), How often do economists self-archive?, Department of Economics, UCSB, available at: http://escholarship.org/uc/item/69f4b8vz (accessed 23 August 2014).

Björk, B.-C. (2013), “Open access subject repositories: An overview”, Journal of the Association for Information Science and Technology, doi:10.1002/asi.23021.

Björk, B.-C., Laakso, M., Welling, P. and Paetau, P. (2014), “Anatomy of green open access”, Journal of the Association for Information Science and Technology, Vol. 65 No. 2, pp. 237–250.

Björk, B.-C., Welling, P., Laakso, M., Majlender, P., Hedlund, T. and Guðnason, G. (2010), “Open Access to the Scientific Journal Literature: Situation 2009”, PLoS ONE, Vol. 5 No. 6, doi:10.1371/journal.pone.0011273.

Black, B. and Caron, P. (2006), “Ranking law schools: Using SSRN to measure scholarly performance”, Indiana Law Journal, Vol. 81 No. 1, pp. 83–139.

Brown, C. (1999), “Information seeking behavior of scientists in the electronic information age: Astronomers, chemists, mathematicians, and physicists”, Journal of the American Society for Information Science, Vol. 50 No. 10, pp. 929–943.

Brown, C. (2001), “The E-volution of preprints in the scholarly communication of physicists and astronomers”, Journal of the American Society for Information Science and Technology, Vol. 52 No. 3, pp. 187–200.

Brown, C. (2003a), “The changing face of scientific discourse: Analysis of genomic and proteomic database usage and acceptance”, Journal of the American Society for Information Science and Technology, Vol. 54 No. 10, pp. 926–938.

Brown, C. (2003b), “The role of electronic preprints in chemical communication: Analysis of citation, usage, and acceptance in the journal literature”, Journal of the American Society for Information Science and Technology, Vol. 54 No. 5, pp. 362–371.

Brown, C. (2010a), “Communication in the sciences”, Annual Review of Information Science and Technology, Vol. 44 No. 1, pp. 285–316.

Brown, D.J. (2010b), “Repositories and journals: are they in conflict?: A literature review of relevant literature”, Aslib Proceedings, Emerald Group Publishing Limited, Vol. 62, pp. 112–143.

Chang, C.-L. and McAleer, M. (2013), “Ranking leading econometrics journals using citations data from ISI and RePEc”, Econometrics, Vol. 1 No. 3, pp. 217–235.

Chu, H. and Krichel, T. (2007), “Downloads vs. citations: relationships, contributing factors and beyond”, available at: http://eprints.rclis.org/handle/10760/11085 (accessed 21 August 2014).

Cohen, N. (2008), “Now Professors Get Their Star Rankings, Too”, The New York Times, available at: http://www.nytimes.com/2008/06/09/business/media/09link.html (accessed 28 August 2014).

Craig, I.D., Plume, A.M., McVeigh, M.E., Pringle, J. and Amin, M. (2007), “Do open access articles have greater citation impact?: A critical review of the literature”, Journal of Informetrics, Vol. 1 No. 3, pp. 239–248.

Creaser, C., Fry, J., Greenwood, H., Oppenheim, C., Probets, S., Spezi, V. and White, S. (2010), “Authors’ awareness and attitudes toward open access repositories”, New Review of Academic Librarianship, Vol. 16 No. S1, pp. 145–161.

Cullen, R. and Chawner, B. (2011), “Institutional Repositories, Open Access, and Scholarly Communication: A Study of Conflicting Paradigms”, The Journal of Academic Librarianship, Vol. 37 No. 6, pp. 460–470.

Cybermetrics Lab. (2014), “WORLD | Ranking Web of Repositories”, available at: http://repositories.webometrics.info/en/world (accessed 24 August 2014).

Donovan, J.M. and Watson, C.A. (2011), “Will an Institutional Repository Hurt my SSRN Ranking: Calming the Faculty Fear”, AALL Spectrum, Vol. 16, p. 12.

Edelman, B.G. and Larkin, I. (2014), “Social comparisons and deception across workplace hierarchies: Field and experimental evidence”, Organization Science, available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1346397 (accessed 28 August 2014).

Elleby, A. and Ingwersen, P. (2011), “Do open access working papers attract more citations compared to printed journal articles from the same research unit?”, Proceeding of the ISSI 2011 Conference, Presented at the 13th International Conference of the International Society for Scientometrics & Informetrics, Durban, South Africa, July 4-7, 2011, pp. 327–332.

Elsevier. (2014), “Content Overview: Scopus”, available at: http://www.elsevier.com/online-tools/scopus/content-overview (accessed 21 February 2014).

Finch, D.J. (2012), Accessibility, sustainability, excellence: how to expand access to research publications, available at: http://www.researchinfonet.org/publish/finch/ (accessed 2 January 2015).

Fowler, K.K. (2011), “Mathematicians’ views on current publishing issues: A survey of researchers”, available at: http://conservancy.umn.edu/handle/11299/109309 (accessed 9 March 2014).

Frandsen, T.F. (2009), “The effects of open access on un-published documents: A case study of economics working papers”, Journal of Informetrics, Vol. 3 No. 2, pp. 124–133.

Gargouri, Y., Larivière, V., Gingras, Y., Carr, L. and Harnad, S. (2012), “Green and gold open access percentages and growth, by discipline”, Proceedings of STI 2012, Presented at the 17th International Conference on Science and Technology Indicators, 5-8 September, Montreal, Canada, available at: http://arxiv.org/abs/1206.3664 (accessed 10 August 2014).

Gavel, Y. and Iselid, L. (2008), “Web of Science and Scopus: a journal title overlap study”, Online information review, Vol. 32 No. 1, pp. 8–21.

Gibson, J., Anderson, D.L. and Tressler, J. (2014), “Which Journal Rankings Best Explain Academic Salaries? Evidence from the University of California”, Economic Inquiry, Vol. 52 No. 4, pp. 1322–1340.

Hahn, S.E. and Wyatt, A. (2014), “Business Faculty’s Attitudes: Open Access, Disciplinary Repositories, and Institutional Repositories”, Journal of Business & Finance Librarianship, Vol. 19 No. 2, pp. 93–113.

Hemminger, B.M., Lu, D., Vaughan, K.T.L. and Adams, S.J. (2007), “Information seeking behavior of academic scientists”, Journal of the American Society for Information Science and Technology, Vol. 58 No. 14, pp. 2205–2225.

Jensen, M.C. (2012), “ABOUT SSRN: From The Desk of Michael C. Jensen, Chairman”, available at: http://www.ssrn.com/update/general/mjensen.html (accessed 28 August 2014).

Karlsson, S. and Krichel, T. (1999), “RePEc and S-WoPEc: Internet access to electronic preprints in Economics. presented at the Third ICCC”, IFIP Conference on Electronic Publishing in Ronneby, May, pp. 10–12.

Kim, J. (2010), “Faculty self-archiving: Motivations and barriers”, Journal of the American Society for Information Science and Technology, Vol. 61 No. 9, pp. 1909–1922.

Kling, R. and McKim, G. (2000), “Not just a matter of time: Field differences and the shaping of electronic media in supporting scientific communication”, Journal of the American society for information science, Vol. 51 No. 14, pp. 1306–1320.

Kling, R., Spector, L.B. and Fortuna, J. (2004), “The real stakes of virtual publishing: The transformation of E-Biomed into PubMed central”, Journal of the American Society for Information Science and Technology, Vol. 55 No. 2, pp. 127–148.

Kousha, K. and Thelwall, M. (2013), “Disseminating research with web CV hyperlinks”, Journal of the American Society for Information Science and Technology, available at: http://www.researchgate.net/publication/256433340_Disseminating_Research_with_Web_CV_Hyperlinks/file/3deec5228643f90bff.pdf (accessed 4 March 2014).

Kousha, K. and Thelwall, M. (2014), “Web impact metrics for research assessment”, in Cronin, B. and Sugimoto, C.R. (Eds.),Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact, MIT Press.

Kousha, K., Thelwall, M. and Abdoli, M. (2012), “The role of online videos in research communication: A content analysis of YouTube videos cited in academic publications”, Journal of the American Society for Information Science and Technology, Vol. 63 No. 9, pp. 1710–1727.

Kurtz, M. J., and Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, pp. 1-64.Laakso, M. (2014), “Green open access policies of scholarly journal publishers: a study of what, when, and where self-archiving is allowed”, Scientometrics. In Press, available at: http://dx.doi.org/10.1007/s11192-013-1205-3 (accessed 1 March 2014).

Larivière, V., Sugimoto, C.R., Macaluso, B., Milojević, S., Cronin, B. and Thelwall, M. (2014), “arXiv E-prints and the journal of record: An analysis of roles and relationships”, Journal of the Association for Information Science and Technology, doi:10.1002/asi.23044.

Lyons, C. and Booth, H.A. (2011), “An Overview of Open Access in the Fields of Business and Management”, Journal of Business & Finance Librarianship, Vol. 16 No. 2, pp. 108–124.

Más-Bleda, A., Thelwall, M., Kousha, K. and Aguillo, I.F. (2014), “Successful researchers publicizing research online: An outlink analysis of European highly cited scientists’ personal websites”, Journal of Documentation, Vol. 70 No. 1, pp. 148–172.

McVeigh, M.E. (2004), “Open Access Journals in the ISI Citation Databases: Analysis of Impact Factors and Citation Patterns A citation study from Thomson Scientific”, Retrieved September, Vol. 20, p. 2008.

Miguel, S., Chinchilla-Rodriguez, Z. and de Moya-Anegón, F. (2011), “Open access and Scopus: A new approach to scientific visibility from the standpoint of access”, Journal of the American Society for Information Science and Technology, Vol. 62 No. 6, pp. 1130–1145.

Moed, H.F. (2007), “The effect of ‘open access’ on citation impact: An analysis of ArXiv’s condensed matter section”, Journal of the American Society for Information Science and Technology, Vol. 58 No. 13, pp. 2047–2054.

Morris, S. (2009), “Journal Authors’ Rights: perception and reality”, Publishing Research Consortium, available at: http://www.publishingresearch.org.uk/documents/JournalAuthorsRights.pdf (accessed 1 March 2014).

Mulligan, A., Hall, L. and Raphael, E. (2013), “Peer review in a changing world: An international study measuring the attitudes of researchers”, Journal of the American Society for Information Science and Technology, Vol. 64 No. 1, pp. 132–161.

Nariani, R. and Fernandez, L. (2012), “Open Access Publishing: What Authors Want”, College & Research Libraries, Vol. 73 No. 2, pp. 182–195.

Neuendorf, K.A. (2002), The Content Analysis Guidebook, SAGE Publications, Inc, London.

Novarese, M. and Zimmermann, C. (2008), “Heterodox economics and dissemination of research through the internet: the experience of RePEc and NEP”, On The Horizon-The Strategic Planning Resource for Education Professionals, Vol. 16 No. 4, pp. 198–204.

OpenDOAR. (2015), “OpenDOAR - Charts - Worldwide”, available at: http://www.opendoar.org/find.php?format=charts (accessed 2 January 2015).

Pinfield, S., Salter, J., Bath, P., Hubbard, B., Millington, P., Anders, J.H.S. and Hussain, A. (2014), “Open-access repositories worldwide, 2005-2012: Past growth, current characteristics and future possibilities”, Journal of the American Society for Information Science and Technology, available at: http://eprints.whiterose.ac.uk/76839/ (accessed 16 February 2014).

PMC. (2014), “PMC International”, available at: https://www.ncbi.nlm.nih.gov/pmc/about/pmci/ (accessed 3 March 2014).

Poynder, R. (2012), “Open Access Mandates: Ensuring Compliance”, Open and Shut, available at: http://poynder.blogspot.fi/2012/05/open-access-mandates-ensuring.html (accessed 2 January 2015).

RePEc. (2014), “IDEAS: Rankings”, available at: http://ideas.repec.org/top/ (accessed 9 August 2014).

Schwarz, G.J. and Kennicutt Jr, R.C. (2004), “Demographic and Citation Trends in Astrophysical Journal papers and Preprints”, arXiv:astro-ph/0411275, available at: http://arxiv.org/abs/astro-ph/0411275 (accessed 2 March 2014).

Skeels, M.M. and Grudin, J. (2009), “When social networks cross boundaries: a case study of workplace use of facebook and linkedin”, Proceedings of the ACM 2009 international conference on Supporting group work, ACM, pp. 95–104.

Sotudeh, H. and Horri, A. (2007), “The citation performance of open access journals: A disciplinary investigation of citation distribution models”, Journal of the American Society for Information Science and Technology, Vol. 58 No. 13, pp. 2145–2156.

Spezi, V., Fry, J., Creaser, C., Probets, S. and White, S. (2013), “Researchers’ green open access practice: a cross-disciplinary analysis”, Journal of Documentation, Vol. 69 No. 3, pp. 334–359.

SSRN. (2014), “Home :: SSRN”, available at: http://www.ssrn.com/en/ (accessed 9 August 2014).

Suber, P. (2012), Open Access, MIT Press, Boston, available at: http://cyber.law.harvard.edu/hoap/Open_Access_(the_book).

Swan, A. (2010,February), “The Open Access citation advantage: Studies and results to date”, available at: http://eprints.soton.ac.uk/268516/ (accessed 2 March 2014).

Swan, A. and Brown, S. (2005), “Open access self-archiving: An author study”, available at: http://cogprints.org/4385 (accessed 2 March 2014).

Tenopir, C., Mays, R. and Wu, L. (2011), “Journal article growth and reading patterns”, New Review of Information Networking, Vol. 16 No. 1, pp. 4–22.

Thelwall, M. (2004), Link analysis: An information science approach, Emerald Group Pub Ltd.

Thelwall, M. and Kousha, K. (2014), “Academia. edu: Social Network or Academic Network?”, Journal of the Association for Information Science and Technology, Vol. 65 No. 4, pp. 721–731.

Thomson Reuters. (2014), “Web of Science Core Collection Help”, available at: http://images.webofknowledge.com/WOKRS517B4/help/WOS/hp_database.html (accessed 1 March 2014).

Wagner, B. (2010), “Open access citation advantage: An annotated bibliography”, Issues in Science and Technology Librarianship, No. 60, p. 2.

Walshe, E. (2001), “Creating an academic self‐documentation system through digital library interoperability: The RePEc model”, New Review of Information Networking, Vol. 7 No. 1, pp. 43–58.

Xia, J. (2008), “A Comparison of Subject and Institutional Repositories in Self-archiving Practices”, The Journal of Academic Librarianship, Vol. 34 No. 6, pp. 489–495.

Zimmermann, C. (2013), “Academic rankings with RePEc”, Econometrics, Vol. 1 No. 3, pp. 249–280.

Appendix 1. Scopus citing documents queries.

SR	Query
arXiv	(WEBSITE(*arxiv*) OR WEBSITE(*xxx.lanl.gov*) OR REFSRCTITLE(arxiv)) AND (PUBYEAR > 1999) AND (PUBYEAR < 2014)
RePEc	WEBSITE(*repec.org*) AND (PUBYEAR > 1999) AND (PUBYEAR < 2014)
SSRN	(WEBSITE(*ssrn*) OR REFSRCTITLE(ssrn)) AND (PUBYEAR > 1999) AND (PUBYEAR < 2014)
PMC	WEBSITE("*ncbi.nlm.nih.gov/pmc*") AND (PUBYEAR > 1999) AND (PUBYEAR < 2014)

Appendix 2. Scopus subject area codes used in the SUBJAREA() command.

Subject area code	Subject area description
agri	Agricultural & Biological Sciences
arts	Arts & Humanities
bioc	Biochemistry, Genetics & Molecular Biology
busi	Business, Management & Accounting
ceng	Chemical Engineering
chem	Chemistry
comp	Computer Science
deci	Decision Sciences
dent	Dentistry
eart	Earth & Planetary Sciences
econ	Economics, Econometrics & Finance
ener	Energy
engi	Engineering
envi	Environmental Science
heal	Health Professions
immu	Immunology & Microbiology
mate	Materials Science
math	Mathematics
medi	Medicine
neur	Neuroscience
nurs	Nursing
phar	Pharmacology, Toxicology & Pharmaceutics
phys	Physics and Astronomy
psyc	Psychology
soci	Social Sciences
vete	Veterinary
mult	Multidisciplinary

1^ Li, X., Thelwall, M. & Kousha, K. (2015). “The role of arXiv, RePEc, SSRN and PMC in formal scholarly communication”, Aslib Journal of Information Management, Vol. 67 No. 6, pp. 614-635.