:: Digital Libraries Columns


Library Journal "Digital Libraries" Columns 1997-2007, Roy Tennant

Please note: these columns are an archive of my Library Journal column from 1997-2007. They have not been altered in content, so please keep in mind some of this content will be out of date. :: Digital Libraries Columns

The Engine of Interoperability


   We don't think about standards when we replace a light bulb or use an
   electrical outlet. But we are surrounded by standards, and their
   invisibility is a testimony to their effectiveness. Without them we
   would be faced with an array of tools that couldn't work together in
   expected ways. On the other hand, where development is in ferment,
   creating standards too soon can have a chilling effect. It's a fine
   line between interoperability and innovation.

   Today is a fertile time for standards development in digital libraries.
   Many of these standards are still in their infancy and some are only at
   the idea stage.

   As an example, at the end of October the National Information Standards
   Organization (NISO) hosted a workshop in Washington, DC, on OpenURL and
   Metasearching (also known as federated searching). Although
   "metasearch" products have not been on the market for long, increased
   use of these tools--and their impact on database providers--make
   metasearching a prime area for standards development. At this meeting
   there was a call for an "0.1" standard for sending a search to a
   database and receiving back a structured result (as opposed to an HTML
   page of search results). This would help database providers know when
   they were being metasearched and format a response that would take
   fewer resources. From the metasearch side, a structured result set
   (instead of "screen-scraping" an HTML page) would be a huge advantage.
   Emerging metadata standards

   Dublin Core is an emerging metadata standard in development since 1995.
   The simple set of Dublin Core elements serves as either a basic element
   set for simple metadata purposes (e.g., adding metadata to web pages)
   or as a common meeting ground for more complex metadata schemes (e.g.,
   MARC). Earlier this year the standard was established as an
   International Standards Organization standard (DIS 15836). Dublin Core
   serves an important role for relatively simple uses, but we need more
   complex and powerful metadata standards as well.

   Others are gaining ground. The Metadata Object Description Schema
   (MODS) is being developed to carry selected data from existing MARC 21
   records and to enable the creation of original resource description
   records. The Metadata Encoding and Transmission Standard (METS) is an
   XML schema for describing digital objects or collections of digital
   objects with one or more metadata packages, including virtually any
   descriptive metadata schema (e.g., MODS, Dublin Core, etc.),
   administrative metadata, and structural metadata. It's considered the
   most useful way to encapsulate digital objects with the related
   metadata for those objects.

   Protocols are formal definitions of the rules that software must follow
   to exchange information effectively. These definitions often include
   how messages must be packaged and sent, appropriate responses to all
   possible situations, and even specific schemas to transmit metadata.
   One of the best known is the HyperText Transfer Protocol (HTTP) upon
   which the web is based.

   Emerging protocols & players

   A new protocol that is already providing useful services is the Open
   Archives Initiative (OAI) Protocol for Metadata Harvesting. It defines
   a method by which digital repositories can "expose" their metadata for
   harvesting and indexing in centralized discovery services such as
   OAIster at the University of Michigan. As was pointed out at the NISO
   workshop, we may also need a protocol for metasearch software to query
   dynamically databases and repositories that cannot be harvested.

   Not surprisingly, major professional institutions lead most standards
   development efforts. The Library of Congress is managing the
   development of MODS and comanaging the development of METS with the
   Digital Library Federation; OCLC leads the Dublin Core effort; and NISO
   is at work on standards in areas like networked reference services. The
   World Wide Web Consortium (W3C) develops Internet-related protocols and
   standards, such as XML. In a departure from the above, the OAI--which
   was initiated to improve access to scholarly e-print archives--is
   formed from a group of interested individuals who came together to
   solve a particular problem.

   Standards and us

   Individuals representing particular constituencies develop standards
   and protocols, but we can all help in the process. Most developing
   standards have discussion lists where people can pose questions and
   make suggestions. While voting on a draft standard is typically
   restricted to members, nonmembers can participate in dialog leading to
   the vote. To get involved takes as little as understanding the problem
   and thinking about how a standard can solve that problem (for more
   information, see the online version of this column at To comprehend the power of standards in a library
   context, just think where we would be if MARC hadn't been created some
   35 years ago.