Partners' Workspace (password protected)

EQUINOX

Library Performance Measurement and Quality Management System

Performance Indicators for Electronic Library Services

Peter Brophy, Manchester Metropolitan University
Zoë Clarke, Manchester Metropolitan University Monica Brinkley, Dublin City University
Sebastian Mundt, Universitäts- und Landesbibliothek Münster
Roswita Poll, Universitäts- und Landesbibliothek Münster

November 2000

1. Introduction - 2. Electronic Library Performance Indicators

3. Consolidated list of datasets - 4. Definitions of terms

5. PI Collection Methodologies - Appendix 1 - Appendix 2

1. Introduction

This deliverable report presents the finalised set of electronic library performance indicators (PIs) devised during the EQUINOX project after extensive consultation with the professional community.

The purpose of this set of indicators is to enhance and complement the indicators for traditional library services presented in ISO 11620: Library Performance Indicators. They are not intended to replace these indicators and indeed it is strongly recommended that these indicators be collected together.

Great care is needed when using any performance indicator to make comparisons between libraries. Indicators which are based on the target population are particularly problematic, because each library will define that population to reflect its own operational environment. Careful analysis of the definitions used to establish each dataset is therefore essential if valid comparisons are to be made. It is also recommended that sampling methods be agreed in advance between libraries where comparison is to take place. For example, academic libraries in a region or public libraries in a national context may agree to use the same methods and instruments. It is important that the level of confidence be stated explicitly when sampling methods have been used (see Annex to ISO 11620 for methodologies).

At this present point in time it is not possible to extract all of the datasets required for these performance indicators from all electronic library services. Some service suppliers continue to provide very little or no usage statistics in relation to their services. Even for those services where usage statistics are made available, different data may be available from each supplier. It is hoped that the datasets required for these performance indicators may give service suppliers some indication of the sort of data that libraries would like to be able to extract from electronic library services, so that comparable data would be available from all systems. While the current situation pertains, some datasets may be impossible to collect for all services, although some data may be collected by user survey, where this is appropriate. Where a particular dataset is not available for a service, this service must be excluded from the indicator. Alternatively the data may be collected by user survey, if appropriate.

This Report is structured in the following way:

A summary of the finalised set of electronic library PIs (Section 2)
A consolidated list of the datasets needed to calculate the PIs (Section 3)
Definitions of terms (Section 4)
Dataset collection and PI calculation methodologies (Section 5)

In addition Appendices are provided to present a short bibliography, a note on the deliberations which led to the production of the current document and the XML DTD specification.

2. Electronic Library Performance Indicators

Performance Indicators

Percentage of the population reached by electronic library services

Number of sessions on each electronic library service per member of the target population

Number of remote sessions on electronic library services per member of the population to be served

Number of documents and entries (records) viewed per session for each electronic library service

Cost per session for each electronic library service

Cost per document or entry (record) viewed for each electronic library service

Percentage of information requests submitted electronically

Library computer workstation use rate

Number of library computer workstation hours available per member of the population to be served

Rejected sessions as a percentage of total attempted sessions

Percentage of total acquisitions expenditure spent on acquisition of electronic library services

Number of attendances at formal electronic library service training lessons per member of the population to be served

Library staff developing, managing and providing ELS and user training as a percentage of total library staff

User satisfaction with electronic library services

3. Consolidated list of datasets

[ELS = Electronic Library Services. LCW = Library Computer Workstations.]

Total number of persons in the target population
Total number of persons in the population to be served
Number of persons in a sample who have used ELS during a specified time period
Total sample size used to establish the percentage of the population reached by ELS
Total sample size for user satisfaction survey
Sum of the values for each ELS indicated by the users in the satisfaction survey
Number of sessions on each ELS by members of the target population during a specified time period
Number of sessions on ELS by members of the population to be served during a specified time period
Number of sessions on each ELS during a specified time period
Number of remote sessions on ELS by members of the population to be served during a specified time period
Proportion of total sessions which are remote for services where this information is available
Number of rejected sessions on a licensed ELS during a specified time period
Number of documents viewed from each ELS by members of the population to be served during a specified time period
Cost of each ELS for a specified time period
Acquisition expenditure on electronic library services
Total acquisitions expenditure

Number of information requests submitted electronically during a specified time period
Total number of information requests received during a specified time period
Mean number of LCW at a specific point in time
Number of LCW provided at a specific point in time
Number of LCW provided during a specified time period
Number of hours the library is open during a specified time period

Number of full-time equivalent library staff providing, maintaining and developing ELS and providing user training
Number of full-time external staff providing, maintaining and developing ELS and providing user training
Total full-time equivalent library staff

4. Definitions of terms

The term ‘library’ is used throughout this document but can be taken to refer to an information service irrespective of its title.

Standard definitions have been used where possible. Whether the definition is taken from an ISO document or has been defined by the EQUINOX project is indicated in bold after the definition.

Cost: Acquisition, subscription, license and pay-per-view costs for electronic library services made available by the library. Network and hardware costs should not be included. EQUINOX

Database: Collection or file of electronically stored data or unit records with software for the retrieval and manipulation of the data. ISO/DIS 2789

Document: Recorded information or material object which can be treated as a unit in a documentation process. ISO/FDIS 5127
[NOTE: Documents can differ in their physical form and characteristics.]
Document or record viewed: Any full text of a digital document or electronic resource that is uploaded, or any catalogue record or database entry that is fully displayed during a search.
[NOTE: Visits to the library website are excluded.] ISO/DIS 2789

Downloading transaction: Any procedure that aims to reproduce electronic data onto a local storage medium or printing facility. ISO/DIS 2789

Electronic library resources: Every document in electronic form which needs special equipment to be used.
[NOTE: Electronic resources include digital documents, electronic serials, databases, patents in electronic form and networked audiovisual documents.] ISO/DIS 2789
Electronic library services: A service which is either supplied from local servers or accessible via networks.
[NOTE: Electronic library services comprise the OPAC, the library website, electronic resources, electronic document delivery and internet access offered via the library.] ISO/DIS 2789 [NOTE: referred to as ELS throughout]

Entry: Bits of information, especially from reference databases or directories e.g. records, abstracts etc. ISO 5127-1

Information request: Information contact that involves the knowledge or use of, or recommendations, interpretation or instruction in the use of, one or more information sources (such as printed and non-printed materials, machine-readable databases, the library’s own and other institutions’ catalogues) by library staff. May also involve recommendations, interpretation or instruction in the use of such sources.
[NOTE 1: The request can be delivered personally or by means of telephone, regular mail, fax or electronic media.
NOTE 2: It is essential that libraries do not include directional or administrative enquiries e.g. for locating staff or facilities, regarding opening times or about handling equipment such as reader printers and computer terminals.
NOTE 3: Enquiries are also excluded if asked for the purpose of locating items of stock that have already been identified bibliographically.] ISO/DIS 2789
Library collection: All documents provided by a library for its users.
[NOTE 1: Comprises documents held locally and documents on remote resources for which access rights have been acquired at least for a certain period of time.
NOTE 2: Access rights may be acquired by the library itself, by a consortium and/or external funding.
NOTE 3: Acquisition is to be understood as deliberately selecting a document, securing access rights and including it in the OPAC or other databases of the library. Interlibrary lending and document delivery and excluded.
NOTE 4: Does not include links to Internet resources for which the library has not secured access rights by license or other contractual agreement. ISO/DIS 2789

Library computer workstation: Public access networked and stand alone computers, provided in the library, though not necessarily by the library, offering access to electronic library services. EQUINOX

Session: An established connection to an electronic service, usually by a log-in.
[NOTE 1: Connecting to a website is regarded as a session if the referring link is external to the website.
NOTE 2: Connections to a general entrance or gateway page should be excluded.] ISO/DIS 2789
Population to be served: Number of individuals for whom the library is set up to provide its services and materials. ISO 11620
[NOTE: For public libraries this will normally be the population of the legal service area; for academic libraries this will normally be the total of academic and professional staff plus students. Each library must decide who to include in its population to be served and this must be carefully recorded to facilitate benchmarking.]
Rejected session: Unsuccessful attempt to connect to an electronic library service because of requests exceeding the simultaneous user limit.
[Note: Rejection through entry of wrong passwords is excluded.] ISO/DIS 2789

Remote session: A session established from outside the library building. EQUINOX

Target population: Groups of actual and potential users appropriate to an individual library as the object of a specific service or as the primary users of specific materials. ISO 11620

1. Introduction - 2. Electronic Library Performance Indicators - 3. Consolidated list of datasets - 4. Definitions of terms - 5. PI Collection Methodologies - Appendix 1 - Appendix 2

This page was last updated on July 22, 2002