Eleven years after its endorsement by the World Wide Web Consortium, XML has been widely
adopted within numerous, disparate communities and in a vast range of application
domains, from standards for electronic filing of federal income tax
For all of these reasons, XML-based content standards such as the Text Encoding Initiative (TEI) have seen wide adoption within the library community, and librarians have been actively engaged in the development of a number of XML-based metadata standards, including Encoded Archival Description (EAD), Metadata Object Description Schema (MODS), Metadata Authority Description Schema (MADS), Metadata Encoding and Transmission Standard (METS), Metadata for Images in XML (MIX), MPEG-21 Digital Item Declaration Language (DIDL), Open Archives Initiative Object Reuse and Exchange (OAI-ORE), Preservation Metadata Implementation Strategies (PREMIS) and many others. XML is now used throughout the research library world, and is a fundamental part of the infrastructure developed within the digital library community over the past decade.
Despite its success, however, XML has not lived up to many librarians' expectations
within one area, that of interoperability. Efforts to exchange information
employing XML-based metadata standards such as Dublin Core have fallen prey to a
number of encoding and semantic inconsistencies
Perhaps more surprising is the failure of XML to ensure interoperability at a
syntactic level.
While some of the failures experienced by the Library of Congress NDIIPP tests were the
result of incompatible repository infrastructure, others resulted from mismatched expectations regarding the appropriate use of METS, one of the XML formats employed for the test. DiLauro et al., discussing Johns Hopkins University's experience in the NDIIPP tests, state,
Stanford commented after their ingest of the JHU archive that they had expected one METS object for the entire archive. Because our approach resulted in many METS files – on the order of the number of items in the archive – the Stanford ingest programs experienced out-of-memory conditions. This situation may have been ameliorated had they used the reference code provided by JHU; however, this will be an area that we will look into for future archive ingest projects. This matter points to a broader issue observed during the various import
processes of this phase. Though three of the four non-LC participants (including
JHU) used METS as part of their dissemination packages, each of our approaches
was different. Clearly there would be some advantage to working toward at least some common elements for these processes
To date, the digital library community has treated these interoperability issues
surrounding structural metadata standards as a technical problem demanding a technical
solution. Most efforts to solve these interoperability problems have focused on the use
of a profiling mechanism to further constrain the creation of instance documents
conforming with a XML schema, sometimes in conjunction with a validation mechanism (such
as Schematron) to test instance documents conformance with the additional requirements
established in the profile
However, while these mechanisms may be successful in ensuring interoperability within
a narrowly defined local context, they are not in themselves any guarantee of
interoperability at the scale envisioned by digital library projects such as Aquifer
If we are to deal with the issues of interoperability that continually manifest themselves in the realm of structural metadata standards for digital libraries, we need to cease viewing this purely as a technical problem, and acknowledge that it is the result of the interplay of technical
One of the fundamental tenets of sociotechnical systems theory is that technological
design and technological evolution are not value neutral processes. Technological
design is both informed by, and seeks to inform, the social environment in which
technology is used, and the work of designers and engineers can be seen as being as much
social engineering as technical engineering. By providing a new means of accomplishing
a task, a technologist is also prescribing a new set of behaviors centered on the new
technology (and possibly proscribing others). This conceptualization of technology was
concisely summarized by ...when technologists define the characteristics of their
objects, they necessarily make hypotheses about the entities that make up the world
into which the object is to be inserted. Designers thus define actors with specific
tastes, competences, motives, aspirations, political prejudices, and the rest, and
they assume that morality, technology, science, and economy will evolve in
particular ways. A large part of the work of innovators is
We can learn a great deal about the viewpoints of a particular technology's designers from the documents they author which define the goals for the technology (e.g., use cases and user needs analysis), those which help implement the technology (e.g., specification documents), and those which attempt to explain or promote the new technology to potential users. If we look at the original specification document for XML, we find a relatively clear set of goals for the technology enumerated:
support a wide variety of
applications.
They also clearly believed that for XML to succeed it must be
easy to use, but they also recognized that the meaning of XML documents shall be easy to create,
XML documents should be human-legible and reasonably clear
) is a good deal different from ease of use for a software developer (It shall be easy to write programs which process XML Documents,
The design of XML shall be formal and concise
).
This relatively small set of goals for the XML language was further elaborated upon by
members of the original World Wide Web Consortium (W3C) XML Working Group in a variety of
papers they authored to introduce and clarify XML to its potential user community. extensibility, to define new tags as needed,
structure, to model data to any level of complexity,
validation, to check data for structural correctness,
media independence, to publish content in multiple
formats,
and vendor and platform independence, to
process any conforming document using standard commercial software or even
simple text tools.
The benefits adhering to XML's providing a
standardized format are identified as including complete
interoperability of both content and style across applications and platforms; freedom of
content creators from vendor control of production tools; freedom of users to choose
their own views into content; easy construction of powerful tools for manipulating
content on a large scale; a level playing field for independent software developers; and
true international publishing across all media.
Emancipatory language is invoked
repeatedly here through the use of the terms to
define new tags
and in so doing choose their own views into
content.
In the designers' world view, a key benefit to XML is the freedom it
provides users to define their own structure for documents and data, using their own
semantics, and to escape restrictions that software vendors (through their own inscriptions
in their own products) might wish to impose on their users. Other articles by members of
the XML Working Group (see, for example,
The XML 1.0 Recommendation bears the inscription of its designers' ideological stance towards appropriate mechanisms for data and document structuring as well as appropriate relationships between document creators and software and platform vendors. The effort to promote a metalanguage over any specific markup language, the adoption of Unicode as a basic character set, and the elimination of SGML features which proved difficult to implement (including CONCUR, OMITTAG and SUBDOC) are some of the technological means through which XML's designers sought to normalize and reify a particular set of social and technological relationships. Nor did this process stop with the release of the XML 1.0 recommendation in 1998. The period between February 1998 and October 2001 saw the development and release of a plethora of additional XML specifications, including XML Namespaces, XSLT, XPath, XML Schema, XLink/XBase, XML Information Set and XSL-FO, as well as a variety of XML software tools including parsers, editors and stylesheet engines. All of these various technological objects presented their own opportunities for their designers to further refine the ideological inscription carried within the XML 1.0 Recommendation. One of these objects in particular, the Namespaces in XML Recommendation, deserves further examination due to its significant affect on structural metadata standards developed by the digital library community.
We envision applications of Extensible Markup Language (XML) where a single XML document may contain elements and attributes (here referred to as a Such documents, containing multiple markup vocabularies, pose problems of recognition and collision. Software modules need to be able to recognize the tags and attributes which they are designed to process, even in the face of
A strong motivating force for the
XML's designers have inscribed two overarching messages within the technology they have created. The first is that XML is about establishing a new
Libraries' exploration of the use of markup languages for encoding of library data
predates the origin of XML by several years. Collaborations with the digital humanities
community on the development of the TEI Guidelines, the development of Encoded Archival
Description, and early efforts to apply SGML to bibliographic data
Most of these languages employ a similar pattern for structuring content and metadata. They provide an encoding mechanism which allows the author to record a hierarchical structure defining the object, and then associate both content and metadata with various nodes within that structure. Figure 1 depicts a very simple version of such a structure, a book with a single chapter; metadata and content files (and metadata
The goal of representing the structure of a work as a hierarchy of nested <div> elements with TYPE attributes was to have a relatively simple, abstract hierarchical structure that could be readily applied to a variety of materials. This was intended to promote the adoption of the standard (a single, simple standard is more likely to be adopted than a variety of complex ones), which in turn was seen as promoting interoperability. Having all of the digital library community using a single standard for structuring content and metadata was seen by METS' designers as preferable to the community adopting a disparate set of standards.
It should be noted that this move towards abstraction was a relatively significant break from the SGML design practices that many research libraries had been using to date. While it is true that the notion of using nested <div> elements was derived from the TEI text apparatus, TEI does not rely on pure abstraction; one does not expect to encounter encoding such as <div type="figure"> in a TEI document, when a <figure> element is available to use. Just as XML itself, the METS schema carries an inscription of its designers' world view, that it was preferable to develop a single, simple, generalizable, highly abstract model and encoding mechanisms to structure content and metadata for digital library objects, rather than to pursue the development of a variety of highly specific schemas (one for photographs, one for journals, etc.), or a grand encompassing schema that contained elements appropriate to different genres that could be combined as needed (e.g., the TEI model). Despite its use of the abstract <div> element with a TYPE attribute to represent the structural components of a digital library object, however, the METS schema insisted on the use of more specific concrete elements to identify different forms of metadata, with the <dmdSec> element used for descriptive metadata and the <amdSec> element used for administrative metadata, along with a series of subelements for different forms of administrative metadata (technical, rights, provenance and source). This typification of different forms of metadata was itself an effort to promote both modularity in further metadata schema development and the creation of certain types of metadata schema. By identifying specific subclasses of metadata within the METS schema, METS' designers hoped to encourage XML developers in the digital library world to create discrete, specialized metadata standards that would align with those subclasses, and that those creating digital library objects could then select from a set of such modular XML metadata standards in composing a particular object. Through METS' design, its implementers consciously sought to encourage the adoption of modular schema design practices within the digital library community.
Other XML-based markup languages adopted by the digital library community have taken a similar approach. The MPEG-21 Digital Item Declaration Language also employs a rather abstract hierarchical structural mechanism for ordering content and metadata. It differs inasmuch as non-structural metadata (<Descriptor> elements in MPEG-21 parlance) are not typed, and structural metadata elements, while still rather abstract, are of three different types: <Container>, <Item> and <Component>. The Open Archives Initiative Object Reuse and Exchange specification is perhaps the most abstract of all the structural metadata standards adopted within the digital library community; while it has multiple serialization syntaxes, all of them employ a single mechanism to link an abstract aggregation with a set of aggregated resources (although the specific linking mechanism varies according to the serialization syntax employed). Those aggregated resources may in turn be aggregations, and any aggregation may be associated with a variety of additional metadata.
If we examine these other standards to determine what inscriptions their designers have placed within them regarding their use, we find messages very similar to that of METS. Structural metadata should be highly abstract, so that a very small set of elements can be employed to structure widely disparate content genres. While METS was perhaps more vocal in trying to push the message that further development of metadata schemas should try to create small, focused and modular metadata sets that could be drawn upon as needed to encode a particular object, the other standards convey the same message (through the use <Descriptor> elements to associate metadata with other elements in the case of MPEG-21, and through RDF mechanisms in the case of OAI-ORE). Other structural metadata standards of interest to the digital library community employ similar mechanisms. The XFDU standard for data archiving uses hierarchies of <ContentUnit> elements that may be associated with <dataObjects> and <metadataObjects>. The IMS Content Packaging standard for learning objects uses hierarchies of <item> elements that may be associated with <resources> and <metadata>. While implementations differ in details, the pattern is similar and widespread across the various structural metadata standards of interest to the digital library community. Again and again we see designers seeking to achieve wide adoption of their standard in order to promote interoperability between differing institutions; to secure this goal, they favor a highly abstract structural mechanism which can be applied to a wide variety of content, and mechanisms to allow a variety of additional metadata schemas to
While of perhaps some limited interest to social researchers of technology, none of the preceding seems particularly surprising or problematic. That the designers of XML itself, and the designers of encoding standards for digital library metadata and content, should favor flexibility, extensibility, modularity and the use of abstraction to support the generalizability of their standard, and hence promote its widespread adoption to help achieve interoperability, would not be a great shock to anyone who has spent more than five minutes in the company of computer scientists. These are all considered almost innate goods among software engineers in general and markup language enthusiasts in particular. Yet the NDIIPP tests cited previously would seem to indicate that flexibility, extensibility, modularity and abstraction are not in and of themselves sufficient to achieve interoperability. So what, specifically, is the problem that METS and other structural metadata standards are encountering?
One of the earliest discussion points in the development of the METS standard was which of the various elements within the schema should be declared mandatory and which optional. After some discussion among the members of the working group that established METS' original design, it was decided that the <structMap> element, which records the basic tree structure on to which content files and metadata are mapped in METS, would be the only required element. METS, in the group's opinion, was fundamentally a
It was a matter of some surprise for many in the METS community, then, when the Library
of Congress, which serves as the maintenance agency for the METS standard, began to
produce METS files for digital versions of certain kinds of audio recordings which
placed the logical structure of the works in MODS records contained within the METS
descriptive metadata section (<dmdSec>) rather than in the structural map, and
registered a profile of METS establishing this as their formal internal practice for
Meanwhile, the structural map would record the physical structure of the work as follows:
This represents a
This example demonstrates two problems that have impeded the development of interoperable
content within the digital library community. The first is that the implementation of
highly abstract elements for the definition of structure provides a tremendous amount of
flexibility to document encoders; there are a vast number of potential encodings of any
given object in METS, with variations possible in depth of structure (do I limit my
structure to musical movements or do I provide structural information down to the
measure level?), labeling (you say TYPE="book", I say
TYPE="monograph"), and arrangement (should the Lord of Rings film trilogy be
encoded as a single METS file? Three METS files? Three METS files for the individual
films and a fourth representing the abstract notion of the Trilogy?). This can lead to
significant variation in encoding practices, even between two institutions dealing with
remarkably similar material and using the same metadata standards, as noted by
The second problem is what we might call the problem of standards independence. Many of the XML metadata schemas that have been developed with the help of the digital library community have been created with the understanding that ensuring their usefulness in a variety of application environments requires that they not contain inherent dependencies on other schemas; they need to be able to express all the relevant information within their particular domain on their own. In many of these XML standards, the designers recognized a need to be able to account for relationships between various content objects being described, whether the description being applied was the more traditional form of intellectual description you would expect in a library catalog, or a technical description of the composition of a TIFF image. The result has been that a number of common metadata schemas within the digital library field contain elements for expressing structural metadata, even schemas that are not primarily intended for recording structural metadata. Dublin Core has its <relation> element, MODS has its <RelatedItem> element, the PREMIS schema for preservation metadata has a <relation> element, even the MIX standard for still image technical metadata contains an element for referencing previous image metadata. As the standards' developers felt they should not make their efforts dependent on structural metadata mechanisms in other standards, they implemented their own. Unfortunately, with the addition of each new metadata standard containing structural metadata capabilities, the potential for difficulties with our first problem increases. Every new metadata standard created within the digital library community seems to add another mechanism for describing the structural relationships between content objects, and hence greater potential for variation in object encoding practices.
The irony is that both these problems derive from the flexibility, extensibility, modularity and use of abstraction to create structural metadata elements that the designers of the metadata schemas hoped to promote. The potential range of variation in encoding structural metadata is the result of each of these factors. The use of abstraction in METS (i.e., the <div> element) was an attempt to make the standard flexible in application; however, it opens up a tremendous degree of play in encoding practice. If you ask two different individuals how many page breaks there are in a text, the likelihood that they will give the same answer is a good deal greater than if you ask them how many divisions there are in a text. The use of abstraction opens up encoding to a much greater degree of personal interpretation, and hence variation. The extensibility of METS, and the hope to promote a modular system of metadata schema reuse that its authors inscribed within it, opens up the possibility of using other metadata schemas to encode structural metadata. And it was this same desire for flexibility and modularity that has led other metadata schema designers to include structural metadata components in their own schemas; they wanted to ensure that their own efforts were flexible enough to be applied in a variety of settings, and with a variety of others. But having to design their own schemas without knowing the specific supporting capabilities to be found in other schemas with which their own might be used, they are inevitably forced to create structural metadata capacities of their own within their schemas. The designers of metadata schemas (structural or otherwise) within the digital library community have sought to adhere to a particular set of design practices, seeking to create flexible, extensible, modular and generalized tools, and to promote like practice in others through inscription of their view of appropriate XML design within their technological artifacts. Unfortunately, promoting such good practices has been a death blow to one of the principle reasons for adopting XML in the first place: to ensure interoperability of digital library materials across systems. Wide-scale interoperability requires wide-scale adoption, but the design practices of schema implementers intended to promote wide-scale adoption run directly counter to wide spread interoperability.
Hence XML's similarity to a rope. Like a rope, it is extraordinarily flexible; unfortunately, just as with rope, that flexibility makes it all too easy to hang yourself.
The digital library community seems to face a dilemma at this point. Through its pursuit of design goals of flexibility, extensibility, modularity and abstraction, and its promulgation of those goals as common practice through their inscription in XML metadata standards, it has managed to substantially impede progress towards another commonly held goal, interoperability of digital library content across a range of systems. How then, should the community respond?
One possible response to this situation would be to say that perhaps our community cares
less about interoperability than we thought. Despite projects intended to promote
interoperability, such as the Digital Library Federation's Aquifer, it may be that
interoperability is actually a lower priority for the digital library community than it
likes to believe, and the adoption of metadata standards that impede interoperability is
merely a reflection of that underlying reality, and not a major problem to resolve.
There is at least some reason to suspect this may be the case. Research libraries
typically have a clearly defined local clientele, and while voices within the digital
library community have been calling for some time for the liberation of content from
local silos to enable their use by a larger community The University of Illinois at Chicago (UIC) Library strives to meet the information needs of UIC students, faculty, and staff.
If libraries do wish to make progress on the issue of interoperability of structural
metadata, they will need to recognize that, as every significant information processing standardization effort
must skillfully negotiate competing and apparently irreconcilable objectives, [and]
serve a wide variety of stakeholders with many different interests.
In the
case of structural metadata, the particular competing objectives that the digital
library community does not seem to have successfully reconciled to date are what control vs. connection.
The structural metadata standards which have been developed to date, with their emphasis on flexibility, extensibility and modularity have sought to afford local institutions the greatest degree of control possible in their encoding practices. The standards are designed to allow any given institution to do what it wants. This has clear benefits in terms of easing adoption of the standard in any given context, and as a result insuring the standard's widespread adoption (obviously a good thing in a standard). However, increasing the amount of local control over the ways in which a language is used and developed is fundamentally at odds with a language's ability to serve as a means for connection with others outside the local context. It is, in essence, promoting the development of regional dialects at the expense of mutual intelligibility. The particular case of structural metadata standards reveals that sufficient local variation in syntax, the ways in which people structure their objects using a markup language, can be as fatal to communication as variation in semantics.
Given this fundamental tradeoff between internal control and external connection, libraries wishing to promote interoperability of digital library content have two possible strategies. The first, and most obvious, is to attempt to alter the balance currently struck between connection and control to more strongly favor connection. There are several mechanisms which the library community might employ in pursuit of this strategy, including the design and use of schemas which more significantly restrict both the means for recording the structure of objects and the ability to employ arbitrary additional schemas within instance documents (or developing profiles of existing schemas to achieve the same ends), establishing formal rules of structural description (equivalent to rules of description used in cataloging for creating bibliographic records) dictating aspects of object encoding not susceptible to enforcement through XML's validation mechanisms, and mandating the use of particular controlled vocabularies and ontologies within document instances to record information such as a <div> element's TYPE attribute in METS.
Decreasing the possibility for local variation in encoding of structural metadata will certainly help improve digital libraries' capability to interoperate with each other. However, removing local capacity for variation will also tend to reduce the number of institutions who are willing to use such a markup language. If the digital library community, for instance, was to revise the METS standard to forbid any use of a <relation> or <RelatedItem> element in a descriptive metadata section to express the logical structure of a work, it would assist in insuring interoperability of digital content, but it might also very well mean losing the Library of Congress's support for the standard. More importantly, however, such an approach overlooks one of the fundamental realities of the web environment: communities of practice no longer operate in isolation from each other (if indeed, they ever did). Even if libraries could agree on a structural metadata standard that enabled a significantly greater degree of support for interoperability than we find with today's standards, libraries must now interact with a variety of other communities (publishers, museums, archives, educational technology companies, etc.) that are also creating their own structural metadata standards. This is not to say that pursuit of this strategy is futile or even inappropriate in many instances; libraries' previous experience with standard efforts such as MARC demonstrate that with sufficient time and effort a particular community of practice can achieve widespread interoperability of metadata. However, the library community's interactions with other communities clearly indicates that this strategy by itself is insufficient to resolve the interoperability problems that libraries confront today.
To deal with these wider issues of interoperability, the library community must adopt a second strategy based on accepting that the need for community control over encoding practices is a valid one, that community
This is certainly not the case today, as can be seen if we examine the work of some of
the major agencies involved in metadata standardization in libraries such as the Library
of Congress. The Library of Congress currently serves as maintenance agency for a
variety of XML standards developed within the library community; if you examine the list
of standards that they are maintaining
A heightened emphasis on standardizing translation between markup languages will mean
further work on formalizing translations between markup languages using XSLT, and
treating those with the level of attention and care that the community has lavished on
metadata schemas. However, it might also be worth considering whether the notion of
formal rules of structural description mentioned earlier might be of benefit in trying
to achieve greater translatability between different markup languages. As an example of
what this might mean, consider the example of the 1:1 principle in Dublin Core
The rise of the network information society is presenting libraries with a variety of new challenges. Perhaps the most significant of these is the heightened degree of interaction with communities of practice that do not share libraries' standards, practices or values. If libraries are to survive and thrive in this new information society, they must alter their own value structure to prioritize communication with other communities to an equal, if not greater, extent than internal communication between libraries. If they pursue this course, they may find that issues of internal interoperability of library systems are more tractable than they have appeared to date.