Digital Libraries or Digital Objects: Measuring the Impact and Value of Use and Reuse

This invited blog post was contributed by Ali Shiri, Professor at the School of Library and Information Studies, University of Alberta. 

The emergence of large scale digital libraries and repositories such as HathiTrust, the Internet Archive, the Open Library, the Digital Public Library of America, Europeana and the World Digital Library, provide new opportunities for digital information users to openly and freely interact with a broad range of digital objects. The unprecedented availability of massive digital collections of books, manuscripts, images, photos, and maps offers new, individual and collective ways of making sense of information and of creating digital content.

The implications of using, reusing and repurposing digital content and collections, in this open and information rich environment, are profound and multifaceted, cutting across many different institutional
contexts such as libraries, archives and museums, as well as numerous disciplines and a wide range of user communities and audience types. Access to digital libraries, in particular, has been closely associated with and discussed in regards to such measures of impact, value and usefulness. In fact, a cursory glance at many digital library evaluation models that have been developed in the past two decades, demonstrates the importance of impact assessment and usability of digital libraries. In today’s world, digital information users, academic as well as the general public, are able to make use of digital libraries to read, explore, entertain, write, research, create and contextualize. Users engage in a diverse range of information practices and tasks, including searching, retrieving, using, learning, conceptualizing, synthesizing, presenting and disseminating. The use and reuse of digital objects is at the centre of all these activities. Teachers use digital libraries to support learning and instruction. Researchers make use of digital information and digital research objects to support the production and dissemination of new knowledge. Digital humanists make use of digital objects both as learning and research artifacts.

In line with these developments, the area of digital public scholarship is emerging as many disciplines within the humanities and social sciences promote the creation and use of digital artifacts and objects, not only by researchers and scholars but also by the general public. Community archives and numerous digital humanities projects that generate digital data and artifacts open a new horizon for the general public to develop digital literacy and digital fluency skills. Thanks to the availability of a diverse range of tools for using and modifying digital objects, digital information users are now capable of creative, innovative and novel use of digital objects to support discovery and exploration.

Assessment of the value, impact and usability of digital objects requires a holistic and multidimensional framework that takes into account use, reuse and repurposing of digital objects. While user evaluation studies have contributed significantly to how we measure, assess and evaluate the value of digital libraries as a whole, we need to be able to demonstrate the value of digital libraries at the digital object level as well. There are quantitative measures associated with the use and reuse of digital objects such as the number of clicks, downloads, bookmarks, views, likes, and items shared and saved. However, these measures only provide a general indication of impact and value. We need measures that document and demonstrate the quality, extent and nature of the use and reuse of digital objects in relation to such facets as the context, discipline, collection that the object belongs to, geographic origin, time period and the nature of task or the information-bearing object that contains the used or reused item. This will call for a holistic assessment framework that addresses various aspects and components of use and reuse as well as techniques, tools and technologies that support digital object use and reuse. Development of such a framework should be evidence-based, empirically-supported and should be based on qualitative and quantitative data, use cases, and user evaluation studies that involve diverse user groups and target audiences. A typology of reuse cases may, for instance, include digital object reuse in the context of educational, cultural, linguistic, artistic, historical, chronological, geographical, and genealogical research and exploration. The value and impact of digital objects and their reuse should be conceptualized not only as part of the scholarly communication lifecycle but also as part of lifelong learning and recreational experiences and activities of digital information users and searchers. It is timely for the digital library community to ensure and promote the relevance and usefulness of digital libraries by providing new frameworks and measures that evaluate and assess the impact of use and reuse of digital objects in relation to intellectual and artistic creativity as well as to informed citizenry, social responsibility and democracy.

Let the (early) data analysis begin!

The project team is fresh off its first successful presentation at the 2017 DLF Forum in Pittsburgh. Genya O’Gara formally introduced our team and project during a session of the DLF Assessment Interest Group’s yearly accomplishments. Following the forum the project team convened our first in-person focus group. Eight individuals kindly shared their thoughts and perspectives on the complexities and challenges of understanding content reuse and the promising possibilities that reuse brings to digital cultural heritage assessment. On the heels of this first focus group was also the completion of our survey. We were delighted to receive over 200 responses (and were equally excited to provide $25 Amazon gift cards to the first 50 respondents). As the team continues to plan future focus groups, we will also be combing through the data and, with guiding insights from our Advisory Board, formulating initial results. This work will ultimately lead to a series of use cases and functional requirements for a future assessment toolkit.

The results of all of this analysis will be shared widely with the community. The earliest reveal of our analysis will be discussed at several upcoming conferences, including ALA Midwinter in Denver. The team will update you all on other presentations as they are confirmed.

Call for Survey Participation

Our first step in our needs gathering process is to release a survey that seeks to identify how cultural heritage organizations currently assess digital library reuse, any barriers to assessing reuse of digital objects, and to begin pulling community priorities for potential solutions and next steps together.

If you are a cultural heritage or research data professional interested in methods of evaluating reuse of your institution’s digital object collections, consider taking our survey:

Please see the survey itself for full eligibility criteria and informed consent. The survey ends at 5:00pm EDT on Wednesday, October 11, 2017.

Please do share on your social media networks! It would be very much appreciated.


We’re growing!: Welcoming 6 Advisory Board Members

We are delighted to announce the formation of our new Advisory Board, comprised of expert stakeholders from cultural heritage organizations throughout the United States of America and Canada.  The Board will provide critical feedback on methods used for data collection analysis and guidance in drafting final recommendations. The Board will also provide direction in identifying digital libraries and other institutions that could benefit from a “reuse toolkit.” (The functional requirements and use cases for a toolkit is the primary output of this grant.)

The project team identified Advisory Board members based on their experience in building and administering digital collections as well as in researching and teaching on digital library issues, and their ability for a granular understanding of the functions of digital libraries as well as a broad vision to shape this toolkit.

Advisory board members will serve for one year. During that year, the project team will host three virtual advisory board meetings, with the remainder of the feedback gathered through email and conference calls.

You can read more about the interests and professional accomplishments of each board member on the Advisory Board page.

Please join us in welcoming the Advisory Board!

Advisory Board members:


Launching the project

Welcome to the IMLS funded grant project Developing a Framework for Measuring Reuse of Digital ObjectsYou can find out more about the project aims on the About page. Over the next year, readers can expect to see a series of posts to the Measuring Reuse website that follows the team’s progress. Along the way, we hope to include reflections and perspectives on reuse from digital library practitioners. Readers are encouraged to watch this space to learn more about the work and to explore the boundaries of reuse with the team.

The grant aims to gather input from those working in digital libraries by conducting a formal needs assessment. This work will inform the development of a “reuse toolkit” to promote the adoption of standardized reuse measurement in digital libraries. The toolkit will highlight sustainable assessment techniques and best practices for communicating the impacts of digital collections. The grant outputs are intended to support cultural heritage institutions in developing collections through a nuanced understanding of how digital library materials are being used.  

For our first post, we wanted to share our early efforts. The grant project began on July 1, 2017. Since then, we have assembled an advisory board (more soon), navigated the complex world of institutional review board (IRB) submission and approval, and compiled a survey that the team will use to develop a baseline for understanding how cultural heritage organizations view and incorporate reuse into their assessment practices. The team expects to solicit survey responses  in the coming weeks using a variety of listservs. Targeted lists will be those that intersect with the topics of digital libraries/repositories and cultural heritage organizations and their communities’ interests. The team will be offering $25 to the first 50 individuals who complete the survey.  If you are interested in participating in the survey please watch for the invitation or feel free to contact any member of the project team for more information.

With our first month completed, the team is looking forward to planning and executing other elements of the grant – including focus groups and conference presentations – over the next year. Check back to learn more about these efforts in future posts.