Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

June 10, 2015

BASE – Bielefeld Academic Search Engine

Filed under: OAI,Search Engines — Patrick Durusau @ 1:48 pm

BASE – Bielefeld Academic Search Engine

From the post:

BASE is one of the world’s most voluminous search engines especially for academic open access web resources. BASE is operated by Bielefeld University Library.

As the open access movement grows and prospers, more and more repository servers come into being which use the “Open Archives Initiative Protocol for Metadata Harvesting” (OAI-PMH) for providing their contents. BASE collects, normalises, and indexes these data. BASE provides more than 70 million documents from more than 3,000 sources. You can access the full texts of about 70% of the indexed documents. The index is continuously enhanced by integrating further OAI sources as well as local sources. Our OAI-PMH Blog communicates information related to harvesting and aggregating activities performed for BASE.

One feature of the search interface that is particularly appealing is the ability to “boost” open access documents, request verbatim search, request additional word forms, and to invoke multilingual synonyms (Eurovoc Thesaurus).

I first saw this in a tweet by Amanda French

August 28, 2013

BASE indexed 50 million OAI-records

Filed under: Bibliography,OAI — Patrick Durusau @ 6:42 pm

BASE indexed 50 million OAI-records by Sarah Dister.

From the post:

BASE, a search engine for academic open access web resources, has indexed more than 50,000,000 OAI-records. The records are provided by about 2,700 repositories among which many are related to agriculture.

BASE is a multi-disciplinary search engine for academically relevant OAI-Sources worldwide, which was created and developed by Bielefeld University Library.

Take a few minutes (or longer) to explore BASE.

It is a remarkable resource. For example, users can invoke the Eurovoc Thesaurus as part of their search query.

February 11, 2013

ResourceSync Framework Specification

Filed under: NISO,OAI,Synchronization — Patrick Durusau @ 2:21 pm

NISO and OAI Release Draft for Comments of ResourceSync Framework Specification

From the post:

NISO and the Open Archives Initiative (OAI) announce the release of a beta draft for comments of the ResourceSync Framework Specification for the web consisting of various capabilities that allow third-party systems to remain synchronized with a server’s evolving resources. The ResourceSync joint project, funded with support from the Alfred P. Sloan Foundation and the JISC, was initiated to develop a new open standard on the real-time synchronization of Web resources.

Increasingly, large-scale digital collections are available from multiple hosting locations, are cached at multiple servers, and leveraged by several services. This proliferation of replicated copies of works or data on the Internet has created an increasingly challenging problem of keeping the repositories’ holdings and the services that leverage them up-to-date and accurate. The ResourceSync draft specification introduces a range of easy to implement capabilities that a server may support in order to enable remote systems to remain more tightly in step with its evolving resources.

The draft specification is available on the OAI website at: www.openarchives.org/rs/0.5/resourcesync. Comments on the draft can be posted on the public discussion forum at: https://groups.google.com/forum/?fromgroups#!forum/resourcesync.

For more on the ResourceSync Framework, see the article in the January/February 2013 issue of D-Lib.

For those interested in synchronization of resources. Say from or to topic maps.

September 18, 2012

Scholarly metadata from R

Filed under: Metadata,OAI,R — Patrick Durusau @ 7:59 pm

Scholarly metadata from R

From the post:

Metadata! Metadata is very cool. It’s super hot right now – everybody is talking about it. Okay, maybe not everyone, but it’s an important part of archiving scholarly work.

We are working on a repo on GitHub rmetadata to be a one stop shop for querying metadata from around the web. Various repos on GitHub we have started – rpmc, rdatacite, rdryad, rpensoft, rhindawi – will at least in part be folded into rmetadata.

As a start we are writing functions to hit any metadata services that use the OAI-PMH: “Open Archives Initiative Protocol for Metadata Harvesting” framework. OAI-PMH has six methods (or verbs as they are called) for data harvesting that are the same across different metadata providers:

  • GetRecord
  • Identify
  • ListIdentifiers
  • ListMetadataFormats
  • ListRecords
  • ListSets

OAI-PMH provides an updating list of data providers, which we can easily use to get the base URLs for their data. Then we just use one of the six above methods to query their metadata.

Re-using metadata is a lot easier than creating all new metadata.

Not to mention avoiding creating new metadata that is inconsistent with existing metadata.

September 28, 2010

Similarity and Duplicate Detection System for an OAI Compliant Federated Digital Library

Filed under: Duplicates,OAI,Subject Identity — Patrick Durusau @ 5:17 am

Similarity and Duplicate Detection System for an OAI Compliant Federated Digital Library Authors: Haseebulla M. Khan, Kurt Maly and Mohammad Zubair Keywords: OAI – duplicate detection – digital library – federation service

Abstract:

The Open Archives Initiative (OAI) is making feasible to build high level services such as a federated search service that harvests metadata from different data providers using the OAI protocol for metadata harvesting (OAI-PMH) and provides a unified search interface. There are numerous challenges to build and maintain a federation service, and one of them is managing duplicates. Detecting exact duplicates where two records have identical set of metadata fields is straight-forward. The problem arises when two or more records differ slightly due to data entry errors, for example. Many duplicate detection algorithms exist, but are computationally intensive for large federated digital library. In this paper, we propose an efficient duplication detection algorithm for a large federated digital library like Arc.

The authors discovered that title weight was more important than author weight in the discovery of duplicates. Working with a subset of 73 archives with 465,440 records. Would be interesting to apply this insight to a resource like WorldCat, where duplicates are a noticeable problem.

Powered by WordPress