Citation analysis is a bibliometric method of analysis that measures the impact of an item (typically a journal article or a book) by counting how frequently the item is cited in other articles, books and resources. It can also demonstrate patterns of use by categorizing the platforms or journals in which the citation appears. Citation analysis is most commonly used for scholarly objects such as journal articles and the same processes can also be applied to non-scholarly digital objects, such as images or primary source documents. However, the success rate is often limited due to a lack of standardized citation formats and practices for referencing non-scholarly digital objects. Likewise, the idea of truly measuring “impact” has been debated due to widely documented concerns over citation pollution and abundant self-citations within scholarly publishing. Despite these debates, it remains the most widely accepted and traditional form of measurement in assessing scholarly impact.
Citation analysis relies on how a citation is formatted and where a citation occurs (such as a works cited or reference section). For scholarly digital objects, this allows scholarship aggregators — such as Web of Science, Scopus, and Google Scholar — to regularly parse the works cited or references lists of scholarly works and update the number of times an article is cited, along with their authors and the journals in which they appear.
Citation managers such as Mendeley can also be used to demonstrate the impact of digital objects in the future. These tools track the number of times that users collect articles or other digital objects in their personal account libraries. While these digital objects are not yet included in the references of a published scholarly work, they indicate utility to a research process and the potential inclusion in a future publication.
For non-scholarly digital objects included in scholarly works, citation analysis is often a manual process where practitioners search scholarship aggregators (Google Scholar) or citation managers (Mendeley) for mention of specific digital objects. Likewise, searching general web indexes such as Google or Bing can help to find non-scholarly use of a digital object in web pages, blogs, or social media.
Commonly used tools include:
Practitioners should follow the practices laid out in the “Ethical considerations and guidelines for the assessment of use and reuse of digital content.” The Guidelines are meant both to inform practitioners in their decision-making, and to model for users what they can expect from those who steward digital collections.
Many of the citation tools available for citation analysis require personal accounts, including Google, Mendeley (Elsevier), and Altmetric (Elsevier), which are known to mine a significant portion of private information from their users. It is recommended users review the privacy policies for each tool (or their parent company) to understand the information that will be collected, and how it might be used, before making a decision.
Chao, T. C. (2011). Disciplinary reach: Investigating the impact of dataset reuse in the earth sciences. Proceedings of the American Society for Information Science and Technology, 48(1), 1-8.
Piwowar, H. A., & Vision, T. J. (2013). Data reuse and the open data citation advantage. PeerJ, 175(1).
Kousha, K.; Thelwall, M. (2019). Can Google Scholar and Mendeley Help to Assess the Scholarly Impacts of Dissertations? Journal of Informetrics,13(2), 467–484.
Aery, S. (2015, June 26). The Elastic Ruler: Measuring Scholarly Use of Digital Collections. Bitstreams: The Digital Collections Blog.
Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 9(1).
He, L. & Nahar, V. (2016). Reuse of Scientific Data in Academic Publications: An Investigation of Dryad Digital Repository. Aslib Journal of Information Management, 68(4), 478–494.
He, L. & Han, Z. (2017). Do Usage Counts of Scientific Data Make Sense? An Investigation of the Dryad Repository. Library Hi Tech, 35(2), 332–342.
Hughes, L. (2014). Live and Kicking: The Impact and Sustainability of Digital Collections in the Humanities. In Proceedings of the Digital Humanities Congress 2012. Studies in the Digital Humanities. Sheffield: HRI Online Publications.
Meyer, E.T. (2011). Splashes and Ripples: Synthesizing the Evidence on the Impact of Digital Resources. London: JISC.
Sinn, D., & Soares, N. (2014). Historians’ Use of Digital Archival Collections: The Web, Historical Scholarship, and Archival Research. Journal of the Association for Information Science and Technology, 65(9), 1794–1809.
Sinn, D. (2012). Impact of Digital Archival Collections on Historical Research. Journal of the American Society for Information Science and Technology, 63(8), 1521–37.
Thelwall, M. (2018). Early Mendeley readers correlate with later citation counts. Scientometrics, 115, 1231–1240.
Contributors to this page include Ali Shiri, Liz Woolcott, and Heidi Blackburn,
Blackburn, H., Shiri, A., Woolcott, L. (2023). Citation Analysis. Digital Content Reuse Assessment Framework Toolkit (D-CRAFT); Council on Library & Information Resources. https://reuse.diglib.org/toolkit/citation-analysis/
Helping digital collections measure impact
Hashtag: #digreuse
