What is OAI-PMH?

OAI-PMH is the Open Archives Initiative Protocol for Metadata Harvesting. Essentially, OAI-PMH is a set of rules that allow Data Providers to share their metadata with Service Providers.

What is a Data Provider?

A Data Provider is any entity using OAI-PMH to allow access to their data. In the context of Spydus, a Data Provider is a library using OAI-PMH to allow access to bibliographic (and archival description) information.

What is a Service Provider?

A Service Provider uses metadata harvested from a Data Provider to perform services for the provider. In the context of Spydus, a Service Provider might be a 3rd party that is harvesting a library's bibliographic (and archival description) data in order to maintain a database of holdings at institutions at state and national levels (e.g. Libraries Australia or EBSCO).

How is harvesting performed?

OAI-PMH is performed using a client application with a specific URL syntax, and uses a number of request and response "verbs" in order to facilitate the retrieval of information from a Data Provider. Some verb examples include:

• GetRecord - to retrieve a single record from a data provider
• ListRecords - retrieve all records that match criteria (also specified in the URL)
• ListSets - retrieve a list of the available records sets which can be invoked in the ListRecords URL (e.g. recently deleted)

More information on specific URL syntax will be provided later in this factsheet. For full details on specifications of the OAI-PMH standards, or full verb dictionary and explanations, see the OpenArchives website. 

Getting started with OAI-PMH and Spydus

How is OAI-PMH implemented in Spydus?

Spydus currently supports the ability of Data Providers (libraries) to provide bibliographic (and archive) information to Service Providers in the MARCXML or DublinCore formats. MARCXML is supported by services such as Libraries Australia and the EBSCO Discovery Service.

Preparing record sets for OAI-PMH

The most common application of OAI-PMH in libraries is for discovery and maintenance of collections. That is, allowing new material to be indexed, and unavailable content to be removed, by the Service Provider.

As OAI-PMH allows the Service Provider to apply criteria to the harvesting based on the date that records were updated, libraries need only ensure that other criteria appropriately filters the data desired to be harvested (e.g. exclude stack locations, whether or not to include reference material and/or electronic resource collections).

The recommended way to prepare record sets in Spydus is to used the Saved Query functionality.

Example query for a record set

A useful query for many Service Providers is one capturing all content that is available in the library's catalogue. Below is an example of how such a query might be composed:

1. Perform an Item Search (Enquiry > Circulation > Item > Item Search) matching the Locations and Collections of bibliographic records desired to be updated by the Service Provider.
   • To include multiple specific Locations or Collections, hold down the Ctrl key and click the desired Locations/Collections before clicking Submit to perform the search.
   • If desired or required, this set may be further filtered using the Refine Set option before proceeding to the next step.
     • Using the NOT option with Refine Set might be useful to exclude items with a specific status e.g. Lost, Being repaired etc, which might signify that an item is not currently available.
2. From the Brief Display of the search result set, click the Translate Set button in the action bar.
3. Ensure that Bibliographic records linked to these items is selected, then click Translate Set.
4. Click Save Query. Enter an appropriate Saved Query Description, then click Save Query.

'Exposing' a set for OAI-PMH 

For a record set to be available to a Service Provider via OAI-PMH, the set must be 'exposed' by adding it to the OAI Set Editor. To do this:

1. Navigate to Maintenance > Cataloguing > OAI Set Editor.
2. Click Add to List. 
3. A Saved Query search will be performed automatically.
   • To switch to a Saved List search, mouse over the Search dropdown and click Saved List.
4. Check the boxes next to any queries desired to be exposed for OAI-PMH, then click Select Records.

What information does a Service Provider require?

Each vendor works slightly differently, but typically there are just two pieces of information that a Service Provider will require in order to harvest from a library's repository:

• the library's base URL
  • In Spydus, the base URL is the libraries Spydus domain with an additional path on the server. The additional path is the same for all Spydus libraries: /oai/data/oai2_0 e.g. https://libraryname.spydus.com/oai/data/oai2_0
  • for harvesting archival description records, the base URL should be https://libraryname.spydus.com/oai/data/oai2_0_arc
    
• the setSpec of any relevant record sets 
  • The setSpec will be displayed in the setSpec column of the OAI Set Editor in Maintenance once the Saved List or Query has been added (see instructions above).

Constructing URLS for OAI-PMH

Once the record sets have been exposed and made available to Service Providers, the content within those sets can be harvested. As mentioned previously, there is a specific syntax for OAI-PMH URLs that must be adhered to in order to successfully retrieve the set. The URL syntax for OAI-PMH in Spydus is:

• baseURL?verb&identifier(optional)&argument(optional)

Identifiers will usually only be used to retrieve an individual record, but arguments may be used to further refine sets. Some examples will be provided below under relevant headings. Multiple additional arguments may be concatenated to a URL using the & operator. e.g. baseURL?verb&argument1&argument2&argument3

Base URL

In Spydus, the base URL is the libraries Spydus domain with an additional path on the server. The additional path is the same for all Spydus libraries, but is slightly different depending on what record type is being harvested: /oai/data/oai2_0 for bibliographic records, or /oai/data/oai2_0_arc for archival description records.

e.g. https://libraryname.spydus.com/oai/data/oai2_0 for bibliographic records or https://libraryname.spydus.com/oai/data/oai2_0_arc for archival description records

Verbs

There are six OAI-PMH verbs that may be used to harvest information from Spydus. These verbs are consistent for both bibliographic records and for archival description records.

Identify

The Identify verb returns basic information about the repository being queried. An Identify URL would look like this:

https://libraryname.spydus.com/oai/data/oai2_0?verb=Identify

And the response would look like this:

ListIdentifiers

The ListIdentifiers verb will return a list of the headers and unique identifiers for all bibliographic records. In Spydus, the unique identifier is the BRN of a bibliographic record. A ListIdentifiers URL would look like this:

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListIdentifiers

And the response would look like this:

The syntax of the Identifier is oai:institutioncode:brn/BRN. i.e. where the institution code is ABCD, and the BRN of a record is 123456, the identifier will be oai:ABCD:brn/123456

ListMetadataFormats

The ListMetadataFormats verb returns the supported metadata formats in the repository. Currently, Spydus supports:

• MARCXML
• DublinCore

A ListMetadataFormats URL would look like this:

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListMetadataFormats

And the response would look like this:

Specify the metadata format

To output in a specified format, use the metadataPrefix argument.

To output in the MARCXML format, use the marcxml prefix. To output in DublinCore format, use the oai_dc prefix.

e.g. https://libraryname.spydus.com/oai/data/oai2_0?verb=ListRecords&set=SQRY:123456&metadataPrefix=oai_dc

ListSets

The ListSets verb returns the details of any sets (Saved Queries or Saved Lists) that have been made available for harvesting (by adding them to the OAI Set Editor in Maintenance).  A ListSets URL will look like this:

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListSets

And the response will look like this:

ListRecords

The ListRecords verb will return the details of records in a Data Provider's repository. Without additional arguments, the ListRecords verb can be invoked to return all bibliographic records in a library's repository. It is recommended to use the set argument to restrict to prepared record sets. Starting with the ListRecords verb URL:

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListRecords

..the setSpec value from a set is added to the argument. e.g. where the setSpec value is SQRY:123456, the ListRecords URL with the set argument will be:

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListRecords&set=SQRY:123456

The set may be further refined based on the datestamp field in the record header by using the optional from and until arguments. The datestamp field corresponds to the Last Updated date in Spydus.

 

From and until arguments would usually be added by the Service Provider, and based on the last time that a harvest was performed. e.g. If a harvest was last performed on the Deleted Items set (setSpec 123456) on 31/01/2020, the from argument would target the following day.

https://libraryname.spydus.com/oai/data/oai2_0?verb=ListRecords&set=SQRY:123456&from=2020-02-01

Additional arguments may be specified if desired (metadataPrefix, resumptionToken), and more information on these arguments is available in Cataloguing Maintenance help articles, and at the OpenArchives website. 

GetRecord

The GetRecord verb, together with an Identifier, can retrieve a single record from a repository. As mentioned above under the ListIdentifiers heading, the syntax of an Identifier is oai:institutioncode:brn/BRN. i.e. where the institution code is ABCD, and the BRN of a record is 123456, the identifier will be oai:ABCD:brn/123456

Using the getRecord verb and an identifier, the URL would look like this:

https://libraryname.spydus.com/oai/data/oai2_0?verb=GetRecord&identifier=oai:ABCD:brn/123456

More information on OAI-PMH

This fact sheet is only a brief introduction to OAI-PMH standards, and is far from exhaustive. Some additional information is available in the Cataloguing Maintenance help articles. For full details on the specifications, and extensive information on additional functions, visit the OpenArchives website.