TEI, one of the first and most widely adopted models of text used in developing information infrastructures in DH was based on the idea of text as an ordered hierarchy of content objects (OHCO) [DeRose et al. 1990]. In the OHCO model, texts are primarily comprised of “linguistic objects” in which form (sections, paragraphs, and quotation blocks) and content (the words that comprise these entities) are separate but related aspects. As a hierarchical model, bigger objects such as chapters or volumes “contain” smaller objects such as paragraphs, which in turn contain other smaller objects such as lists or quotation blocks. This model reflects a concern with procedural practices for verbal texts in typesetting and printing, which entails adding meaningful metadata to a text as meta-instructions for how to format a document [DeRose et al. 1990, 2]. With the OHCO model, computation is facilitated by the structure of these content objects since they can be consistently maintained across different formats or layouts.

Rationales of text provide for an understanding of text by which we can evaluate whether a model such as OHCO adequately represents textuality more generally. Critiques of OHCO are concerned with the many ways in which elements of a dynamic text overlap and are therefore poorly represented by a hierarchical, nested structure. Literary texts such as novels and poems, for example, often comprise multiple, overlapping themes or rhyming schemes that fall across lines or stanzas. Due to these debates about the model’s representation of text, the OHCO authors subsequently “retreated from the simple OHCO thesis” and “conclud[ed] that analytical perspectives do seem to exist and do seem to provide fundamental insights into the nature of texts and the methodology of text encoding”  [DeRose et al. 1990]. Subsequent refinements to the OHCO model stipulate that while the OHCO model is both practically and empirically valuable to the general community of text consumers, it is perhaps less valuable in the context of literary, cultural, and aesthetic texts where a text can be understood to reflect a rationale of text with a spectrum of determined and determinable dynamics.

Meant to serve as a model for standards, FRBR was developed primarily as a model or vocabulary for library catalogers and system designers by the International Federation of Library Associations and Institutions (IFLA) in order to standardize practices for finding, identifying, selecting, and obtaining cultural artifacts from libraries [Tillet 2004, 2]. In an attempt to standardize terminologies for abstract concepts, FRBR includes rules for the specific use of general terms including work, expression, manifestation, and item. In FRBR, work is described as “the story being told in the book, the ideas in a person’s head for a book;” expression is described as a particular translation or edition (scholarly or popular for example) of the work; manifestation is described as a particular publication of the work; and item is described as the “physical object that has paper pages and a binding and can sometimes be used to prop open a door or hold up a table leg” [Tillet 2004, 2]. In its focus on conceptual entity-relationships, FRBR’s entities reflect an attempt to disambiguate terms by grouping entities by their characteristics as well as by grouping them according to the relationships that tie them together. Thus, multiple editions or expressions of Hamlet can be associated with the same work, making all of the editions findable, identifiable, and obtainable according to that relationship.

Though FRBR entities are imagined as “universals”, Renear and Dubin show how rationales of text that include textual dynamics at play provide an opportunity for problematizing how the FRBR entities represent texts [Renear and Durbin 2007]. In particular, the authors use the example of the author of a TEI XML (eXtensible Markup Language) document. They discuss a TEI XML document as a complex object that can be simultaneously considered a manifestation (or a new publication of the same content) and an expression (in the event of a scholarly edition that expresses a version of the text that has not been articulated in this way previously). Due to the kinds of ambiguities these examples represent, Renear and Durbin argue that three of the four FRBR entities in Group #1 are problematic since they are not self identical or unique. They argue that the entities are, in fact, roles rather than types. That is, while a type remains a “rigid” property regardless of the context, a role “is brought about by contingent social circumstances” and seems to represent more readily the behaviors that Renear and Durbin observe.

Consequent to making a distinction between role and type for the FRBR entities, Renear and Durbin refactor the FRBR types in order to suggest new, more “rigid” or distinct types. Adopting John Searle’s distinction between natural objects, which are presumed independent of context, and social objects, which are contingent on the social dimension of knowing [Searle 1995], Renear and Durbin call these more distinct types symbol sequences (which play the role of realizing a work), physical kinds (which play the role of embodying expressions), and physical objects (which play the role of exemplifying manifestations) [Renear and Durbin 2007]. While Renear and Durbin admit to their suggestion’s inconsistencies, the important point here is the extent to which rationales of text are being used both to problematize as well as reimagine how we can develop rigorous text processing systems for the reproduction, transmission, dissemination, and analysis of texts that are both tractable by computers and authentic to humans.

A joint project of the School of Information (iSchool) at the University of Texas at Austin (UT) and the Illinois Informatics Institute (I3) at the University of Illinois at Urbana-Champaign (UIUC), the High Performance Sound Technologies for Access and Scholarship (HiPSTAS) project is developing the ARLO (Adaptive Recognition with Layered Optimization) software. ARLO is a web-based, machine-learning application originally developed for acoustic studies in animal behavior and ecology [Enstrom 1993]. In ARLO, the audio text is represented as a spectrogram. Designed to model a bank of hairs in the inner ear and to vibrate at different audio frequencies in response to sound waves, ARLO creates spectrograms by monitoring and then sampling the instantaneous energy of these “hairs” or tuning forks[3] and then using this data to create a 2D matrix of values (frequency vs. time). These matrices or spectrograms (see example in Figure 1) show a map of sonic energy across time with each row of pixels representing a frequency band and the color of each pixel representing the numerical value of total energy of that particular frequency (or how much the tuning fork trembles) for that point in time. ARLO uses these spectrograms to model sonic features for machine learning processes including clustering (unsupervised learning) as well as classification (prediction or supervised learning).

HiPSTAS was originally funded by the National Endowment for the Humanities as an Institute for Advanced Topics in the Digital Humanities between May 2013 and May 2014 for scholars who are interested in using machine-learning software to analyze audio texts of primarily verbal events such as poetry performances, speeches, and storytelling activities. Many of the HiPSTAS participants used ARLO to analyze poetry performances in PennSound, a web-based archive launched by Charles Bernstein and Al Filreis in 2005 as a noncommercial offering of approximately 30,000 downloadable MP3s — mostly as song-length singles. The PennSound recordings are already retrievable both from a library catalog by authors’ names and via Web search engines, but the HiPSTAS participants were primarily interested in analyzing “vocal gestures” in the performances. These gestures such as “the cluster of rhythm and tempo (including word duration), the cluster of pitch and intonation (including amplitude), timbre, and accent”  [Bernstein 2011, 126], are always dynamically at play and are especially significant for interpreting poetry performances.

Poets and scholars describe on online HiPSTAS project pages how searching PennSound “sonically” with ARLO allow them to consider new research questions about the performances including a focus on sonic “para-content” such as pitch and laughter. One participant writes, for example, “I have observed from small-scale analysis that timbre and sound duration are indicators of a stressed syllable. Pitch intensity also correlates with stressed syllables, although not in every instance — I would like to investigate further how much pitch intensity correlates with metrical stress”  [Boruszak 2014]. Another participant wants to focus on the material artifacts of the recording process, what he calls “para-content audio data” for certain audio texts in order to locate other recordings in PennSound that were recorded as part of a series; he wants “to confirm the suspected provenance of some recordings and to start to bring the recordings together for the first time, perhaps, ever” [Mustazza 2014]. A third participant is interested in moments of laughter that “reveal the presence of the audience, emphasize the construction of sounded poems as a the [sic] product of a dialogue between audience and poet, and change significantly the nature of the poem in question”; focused on “close listenings” of four versions of William Carlos Williams’s “This is Just to Say,” this participant argues “that a version in which Williams seemed to try to get the audience to laugh and failed showcases the power of laughter at poetry readings, transforming the poem from a whimsical delight to a highly serious, academic reflection on the nature of art and poetry” [Rettberg 2014]. Using ARLO’s spectrograms and machine learning processes — which seemed to offer access to these dynamic aspects of audio textuality — the participants began to articulate how these rationales for audio texts might be realized in a system like ARLO.

While provocative, systems like ARLO also provide an opportunity for interrogating whether or not they adequately represent both authentic and tractable texts. For instance, identifying sonic patterns in a system such as ARLO is a time-consuming task of finding and labeling training data that adhere to the listener’s understanding of the audio-text’s rationale or dynamic structures. Using these human-generated training examples, ARLO creates a model for labeling new, “unseen” audio examples. In the case of a scholar working with recordings, this could mean marking up a sound recording according to some of the sonic aspects the participants identified above such as variances in pitch, recording noises, or the presence of laughter or applause. For example, Figure 1 shows how one could mark up different voices on spectrogram such as “William Owens” (at point “A”) and “Robert Frost” (at point “C”) as well as silences (at point “B”) in order to teach the machine to model these patterns and to identify or classify more of these sonic patterns of interest in a collection.

Labeling examples for machine-learning is ultimately an ontological exercise that requires some standardization. Standardization allows for consistency across machine learning examples as well as results that can be more easily interpreted in terms of a particular discourse community. For instance, instead of compiling their own list of labels, the PennSound scholars chose to use terms provided in the “Transcriptions of Speech” section of the TEI P5 Guidelines because they wanted a set of terms that had been standardized by a community of scholars for whom “mark up” guidelines were the result of sustained study and a concern with representing dynamic rationales of text. Indeed, the elements or terms in the “Transcriptions of Speech” guidelines, which had been vetted by peers who were also versed in both verbal transcription and textual theory seemed to adequately describe the “vocal gestures” also described by Bernstein including Tempo, Rhythm, Loudness, Pitch, and Tension as well as vocal quality attributes such as Whisper, Breathy, Husky, Creaky, Falsetto, Resonant, Unvoiced Laugh or Giggle, Voiced Laugh, Tremulous, Sobbing, Yawning, and Sighing [TEI Consortium 2014].

After the poets and scholars had used these TEI terms in ARLO to label two-second samples from PennSound’s 5497 hours of audio, however, they determined that the list was insufficient for adequately describing the sonic aspects of their audio texts.

The issue, as described by the participants, was that ARLO’s prototype environment for labeling required them to label snippets of sound as though they were static objects while they understood the audio texts to be dynamic events that reflected a relational context. For instance, one participant might mark a two-second sample “Beatable” with a “High” pitch and another might classify the same sample as “Arrhythmic” with a “Low” pitch. This seeming discrepancy was the result of differing perspectives on the sounds since “High” is only perceived as “High” in relation to “Low” and “Arrhythmic” is only meaningful in relation to “Beatable.”[4] The participants also wanted to label larger contexts rather than snippets and to tag the recording scenario (such as the sound of the room) over time; the perceived gender of the speaker so that they could mark why they thought a pitch was higher or lower (in relation to how a voice perceived as male or female might register); and the genre of a noise (such as musical or lilting voice) as they perceived it. It quickly became clear that the TEI terms did not provide a way for the participants to label their audio texts as situationally contingent or in a way they felt adequately represented the various audio-textual dynamics they could perceive based on their perspectives in a particular moment in time. Consequently, while participants had rationales of audio-text they wished to access and analyze, their experience in this pilot study seemed to indicate that these rationales were computationally intractable or non-computable in the ARLO system.

While the TEI’s “Transcriptions of Speech” guidelines represent one of the few rich content models for reproducing, transmitting, and displaying the verbal content of audio texts in the digital humanities [Johansson 1995], it was originally developed to produce transcripts or written representations of a stretch of speech. According to extant meetings notes and draft versions, speech encoding guidelines were not originally included in the original TEI P1 guidelines [Sperberg-McQueen and Burnard 1990]. Indeed, when tasked with creating recommendations for its inclusion, the Spoken Text Working Group (STWG) purposefully restricted their guidelines, as much as possible, to the verbal aspects of a recording [Johansson et al. 1991a]. Consequently, they labeled prosodic and paralinguistic features such as the vocal gestures Bernstein describes including “quasi vocal things such as laughter, quasi lexical things such as ‘mm’” [Johansson et al. 1991b] as well as “speaker overlap, pauses, hesitations, repetitions, interruptions”, uncertainty, and context [Johansson 1995] as “problems.” Reflecting these early deliberations, the current guidelines still focus on the verbal content of recordings, stating conclusively that “speech regarded as a purely acoustic phenomenon may well require different methods from those outlined here”  [TEI Consortium 2014]. To date, these different methods have never been produced within the context of the TEI.

From the extant meeting notes surrounding the development of these guidelines, it appears that the TEI STWG found time-based sound dynamics such as pitch, speed, and tone meaningful, but impractical to represent given the TEI’s structure. Notes from the working group’s 1991 meeting reflect the group’s knowledge of the relationship between para-linguistic content and cultural studies in the works of [Svartvik and Quirk 1980] and [Tedlock 1983], which are heavily cited, but also the understanding that the STWG had to compromise representing these audio-text dynamics in order to stay consistent with the TEI’s underlying OHCO model of textuality and its subsequent use of hierarchical XML (EXtensible Markup Language) elements. The STWG call “performative features such as pitch, speed and vocalization...as analogous to rendition in written texts” and suggest that they be marked similarly in TEI-marked transcriptions, “using milestone tags such as <pitch.change>, <speed.change> etc.” [Johansson et al. 1991b]. Likening these performative features to type changes such as italicized or underlined text, the note suggest “milestone” tags that identify these sonic events as clear points in time with a start and an end point instead of as dynamic events that happen in concert with other patterns across time. The end result is similar to the instances in which structural tags produced in the TEI guidelines for text such as <p> for paragraph or <l> for line cut off themes across novels or truncate rhyming schemes across poems.

First marking these sonic phenomena as moments of “change” and, later, as spans to “highlight” [TEI Consortium 2004], the STWG eventually relegated the descriptors outlined above to attributes within the “shift” element (<shift>), where they still appear in the P5 TEI Guidelines today [TEI Consortium 2014]. Accordingly, the encoder is not only encouraged to mark these dynamic sound attributes in the encoded transcript as defined points in time but also as “shifts” outside the “normal” speaking mode [Johansson et al. 1991b]. While the guidelines were “not intended to support unmodified every variety of research undertaken upon spoken material now or in the future”, these choices show the prominence that models for verbal content in the form of transcriptions have had in the development of content model standards for audio texts in the digital humanities [TEI Consortium 2014]. Like the scholars before them who critiqued the OHCO model of textuality for poetry, our HiPSTAS participants have indicated that these rationales of audio-text cannot adequately represent rationales of audio-text that include the situationally contingent and time-based vocal gestures of poetry performance recordings.

The Bibliographic Framework Initiative (BIBFRAME) was founded by the Library of Congress (LC) as a movement to replace the LC’s MARC (Machine-Readable Cataloging) standard[5] and represents an example of current thinking concerning how cultural artifacts such as texts and audio-texts are described for discovery in large repositories. Like the FRBR model, the BIBFRAME model seeks to establish a standard model for standards in “all aspects of bibliographic description, data creation, and data exchange . . . includ[ing] accommodating different content models and cataloging rules, exploring new methods of data entry, and evaluating current exchange protocols”  [Bibliographic Framework Initiative]. Like FRBR, the BIBFRAME model also has a central imperative to create models that represent and maximize a user’s ability to find, identify, select, and obtain library holdings such as books and audio files by “differentiat[ing] the conceptual work (a title and author) from the physical details about that work’s manifestation (page numbers, whether it has illustrations)”  [Bibliographic Framework Initiative].

The aspect of the BIBFRAME initiative of concern for this piece is a recent report created by members of the initiative called “BIBFRAME AV Modeling Study: Defining a Flexible Model for Description of Audiovisual Resources” [Van Malssen 2014], which is markedly different than FRBR. While the author of the “BIBFRAME AV Modeling Study” is, like FRBR’s authors, very keen to incorporate alternate models that meet both public and scholarly needs for access and discovery methods, her primary model of reference, unlike FRBR’s or the TEI’s, is time-based audiovisual resources rather than verbal texts.

As the report indicates, AV resources have medium-specific characteristics that must be taken into consideration in any digital representation. First, they are by nature event-centric and carrier dependent. That is, what is central to the rationale of such recordings is that an event or action (or a continuum of events and actions) that took place in a time and at a place was recorded or “fixed” to a carrier such as a film reel or an audio tape. Van Malssen notes that “while machine dependency is not unique to time-based media, it is an unextractable attribute” that must be taken into account when considering provenance but also for access and discovery in terms of creating playback opportunities at the right speed or color resolution [Van Malssen 2014, 6]. Second, AV resources are more often than not created by and contributed to by many collaborators. Third, these materials are often unique in that they have never been published [Van Malssen 2014, 7]. Fourth, AV resources can be singular items but they are also manifest in the aggregate as collections and can appear on multiple carriers. For example, albums have multiple tracks; songs appear in multiple collections; and long interviews or films are often recorded across multiple tapes or reels [Van Malssen 2014, 8]. Fifth, the audio and moving image media we see today have complicated relationships to their original recordings. Most have been migrated across media formats as older media degenerates and new media evolves. Historical recordings migrated to reel-to-reel tapes, for example, may have first been recorded on glass discs that were shorter in length. Thus, the reel-to-reel tapes do not have a one-to-one relationship to the carriers that originally held the recording. Or, film reels may have been conserved or enhanced in later generations after decades of poor quality viewings. Because of these important characteristics of AV materials, Van Malssen argues that models for cataloguing (and thus discovering) AV must reflect a primary concern with modes of creation and access rather than simply a focus on particular types of content [Van Malssen 2014, 4].

Due to these issues, Van Malssen recommends an event-centric content model for AV resources that takes the physically situated context (e.g., the recording and playback circumstances) into account. If expressions can be described as either work- or event-centric, it would allow for “the inclusion of works, or some work elements as part of the event, but does not require a work be present”  [Van Malssen 2014, 24]. This distinction between event and work is exemplified by the difference between an audio recording of a battle that took place in World War II and a filmed reenactment of the same battle. A description of the battle recording would be event-centric since the event took place in space and time and is the object of description at the content level; the filmed re-enactment would be more like the traditional notion of the work in the mind of the creator [Van Malssen 2014, 18]. Van Malssen notes that not treating these catalog descriptions differently results in a blurring of resources: both event and the reenactment would employ the same subject terms (World War II) even though the first is the event while the second uses the event as content.

For audio texts such as recordings of poetry performances, an event-centric description could reshape processes for discovery and possibilities for research. For instance, poetry readings that are recitations of a written text could be marked differently than poetry performances. FRBR identifies an expression as a different edition or translation of a work and the manifestation as the physical embodiment of that expression with the item as “a single exemplar of a manifestation.” A marked difference between work and event would be salient in poetry performance studies where the performance of a poem that is also written could be very similar to or very different from the written text – either a new expression or a whole new work (that is the event). For example, in Ken Sherwood’s analysis of the recordings and transcriptions of Amiri Baraka, Kamau Brathwaite, and Cecilia Vicuña, he speaks of “emergent performance events” as different works in comparison to poetry recitations, which he considers oral versions (or variant expressions) of the written text [Sherwood 2006]. A performance, Bernstein argues “opens up the potential for shifting frames, and the shift of frame is itself perceived as a performative gesture . . . the implied or possible performance becomes a ghost of the textual composition”  [Bernstein 2011, 127]. A model that could be either event- or work-centric could serve to help identify these kinds of hybrid situations in which an event gives rise to a work or vice versa, creating an entirely new, but related, entity.

It is clear that our print-based, logocentric biases preclude us from facilitating new kinds of discovery and analysis with alternate media, but seeing these biases also makes it apparent that we are building models of textuality in general that remain insufficiently poor. The attempt in the HiPSTAS project to create an infrastructure for analyzing audio patterns in audio texts illustrates the importance in the humanities for rationales of audio text that engage audio as events that may make meaning differently by means of markedly different reception scenarios. Finding and making meaning with audio texts can include a consideration for sound dynamics that can only be perceived through time such as pitch, timbre, and intonation as well as the importance of material recording scenario artifacts, all of which could be considered “noise” and essentially rendered undiscoverable in the context of systems that model all texts as if they are static, verbal documents. As a culture, we have become adept at modeling textuality in terms of a print-based paradigm in which texts are discrete entities (in terms of both time and space) of verbal content. We are arguably less adept at modeling time-based or three-dimensional media, which we still insist on fixing in time and translating into words, either through metadata or through transcripts.[6] Discussions concerned with access and scholarship with AV materials provides for an opportunity for expanding our thinking about these “other” texts but also how representations of traditional (or book-like) verbal texts are equally insufficient in the digital environment.

With an event-centric model, textual events could be described as social objects that reflect collective intentionalities. Arguing that the role a “natural” object plays is determined by contingent social circumstances, Renear and Durbin cite J.R. Searle’s understanding of these circumstances as the “collective intentionality” of producers, transmitters, and consumers who shape the role an object plays in a system ([Searle 1995], quoted in [Renear and Durbin 2007]). This notion of collective intentionality is also essential in social text theories such as Martha Nell Smith’s ideas concerning triangular textuality (“the influence of biography, reception, and textual reproduction”) in Emily Dickinson’s work [Smith 1992, 2]; in the context of the Ivanhoe Game in which the reader’s interpretations are essential to the perceived rationale of text [McGann 2001] [Drucker and Rockwell 2003]; or in D.F. McKenzie’s insistence that texts serve as signposts for spatio-temporality, as they alert “us to the roles of institutions and their complex structures in affecting the forms of social discourse, past and present”  [McKenzie 1999, 15]. In defining each manifestation as the fixing or recording of socio-textual events, McGann, McKenzie, Smith, and others are discussing texts as social objects in much the same way as Renear and Durbin who are arguing that the social and cultural circumstances that help produce texts and our understandings of a text should be included in computational models for texts.

While it would seem that such social circumstances are beyond the codifications of a bibliographic standard for access, there are emerging models for discovering AV that are promising. Van Malssen cites indecs, a project of the European Community Info 2000 initiative, as an AV model that adequately “places emphasis on the role that events play in the creation of a resource” by using the expression entity to describe events as “‘creating’ events (an event which results in the making of a creation), ‘using’ events (events which results [sic] in the use of a resource), or ‘transforming’ events which involve the use of one creation in the making of another)”  [Van Malssen 2014, 19]. Marking the role of event is markedly different even than the promising W3C Provenance Data Model (PROV-DM), which is a framework that includes “information about entities, activities, and people involved in producing a piece of data or thing, which can be used to form assessments about its quality, reliability or trustworthiness”  [PROV-DM]. Designed to model the relationships involved in the creation of items, PROV-DM is organized into six components that include the following: “(1) entities and activities, and the time at which they were created, used, or ended; (2) derivations of entities from entities; (3) agents bearing responsibility for entities that were generated and activities that happened; (4) a notion of bundle, a mechanism to support provenance of provenance; (5) properties to link entities that refer to the same thing; and, (6) collections forming a logical structure for its members”  [PROV-DM]. Like the FRBR model, however, PROV-DM does not designate the role of events in terms such as “creating events,” “transforming events,” and “using events” but rather the manifestations such as texts, paintings, or photographs that result from these events. Thus, while a more robust model such as PROV-DM can provide a model for text that foregrounds the results of collective intentionalities (and thus records its occurrence), the event-centric information concerning the roles these events play in relationship to each other and to the objects they effect remains obfuscated.

A model of textuality that represents text as a spatial and temporal phenomenon might allow for interactions and representations in a digital environment that, rather than insisting on fixity, foreground principles of emergence. For example, the AV model recommendations for BIBFRAME note that FRBR does not provide a model for describing an aggregate manifestation that comprises two or more distinct expressions such as when two different parts of a recording are recorded or “fixed” at different times and different places or when different tracks on a recording are compilations of wholly different works, a scenario that is not far removed from similar instances of volumes of poetry or short stories, which have been republished and reissued. This inadequacy comes into greater relief as we begin to think about the creating, using, and transforming events that give rise to whole new works.

The recommendations for a BIBRAME AV model attempts to expand on the notion of fixity that is inherent to the FRBR model. Discussing a digital video file that might have singular or multiple video and audio tracks plus a subtitle track, the AV model recommendations cite the concept of essence as it is used in the PBCore (Public Broadcasting Metadata Dictionary Project) and EBU (European Broadcasting Unit) Core models for broadcast collections as a more robust concept for an event-centric model since it typically includes information about both the content and carrier as well as enabling the description of multiple instantiations of a given content item, such as a broadcast that is repeatedly aired at different times. The AV model recommendations favor the essence model as one that “is as much about the bitstream or signal as it is about the content carried in that stream” because it “is expressed in these models through sets of carrier sub-elements . . . which can be repeated for different essence types found in that carrier”  [Van Malssen 2014, 18].

This notion of essence corresponds to John Bryant’s definition of verbal texts as “fluid” or as a “flow of energy” rather than a product or a “conceptual thing or actual set of things or even discrete events”  [Bryant 2002, 61]. According to this definition, the aggregated text comprises multiple versions in manuscript and print, various notes and letters and comments of contemporaries or current readers, as well as an element of what I have called textual performance [Clement 2011] or a concept of textuality as it is manifested or performed in real time and space with a collaborative audience. By developing models that expand notions of fixity as an essential element, we allow manifestations to be aggregate and contingent, a representation for text that allows for new modes of discovery and access.

As discussed, according to the recommendations for a BIBFRAME AV model, an AV object should always be described in terms of the event of its realization, in terms of how it has been and could be realized on a particular carrier or format (such as a vinyl record or a CD), or in terms of a particular playback system (such as a record player or CD player). As such, every manifestation becomes indexical because each points to these events or technological protocols that generated the item. As such, the nature of an object’s identification as an “item” would be based in part on the information surrounding the contingent circumstances of its realization. For instance, in order to enable “authentic access and preservation,” the AV model recommendations cite a need for software systems that not only describe granular features of certain formats such as “audio reel, number of tracks, playback speed, reel size, etc.”, but also the technical characteristics of carriers in order “to enable identification of the appropriate playback equipment” that can respond to or make manifest these features [Van Malssen 2014, 24].

Accordingly, these aspects of an object’s indexicality are also performative in a digital environment since they create how we understand the structure of the event of an object’s instantiation. Wolfgang Ernst describes this kind of performativity in the digital environment as the extent to which code plays a role as text, as image, as audio, and so on; from this perspective, “rigid text is replaced by an operative mathematics”  [Enrst 2012, 87]. By drawing attention to the manner in which code effects commensurability among digital “items” such as text, images, and sound, Ernst considers “the structure of an archive whose essence, the closer one looks [as] less the archived material per se than a dynamic conception of the idea of the archive”  [Enrst 2012, 83]. This is to say that the very nature of digital materiality requires that a model for “item” (through attribute and values) be applied on the level of the bitstream in order for any text to be realized as such. This aspect of the item’s indexical performativity is also identified in Renear and Durbin’s assertion that “items are the things, whatever their nature (physical, abstract, or metaphorical), which play the role in bibliographic control that FRBR assigns to items”  [Floyd and Renear 2007]. In both cases, the attributes and relations of an item (whether on the level of its physical or digital instantiation) that effect its manifestation are also what would define its role as an item in a system like FRBR or BIBFRAME and our understandings of its “itemness”.

When modeled as event-centric, these attributes and relations of an item are directly affected by changing data models that demand updated software and hardware with changing protocols and algorithms for the constant realizing of items over time and across systems. For instance, as Jonathon Sterne argues, “the transformations effected by MP3 encoding are themselves heavily-directed cultural practices. MP3s contain within them a whole philosophy of audition [based on the limitations of human hearing] and a praxeology of listening . . . [that] emphasizes distraction over attention and exchange over use”  [Sterne 2006, 828]. Noting that the MP3 is a container for an audio digital recording that has been compressed with the use of a mathematical model that takes human auditory perception into account, Sterne writes that the “most compelling part of the MP3 is the psychoacoustic model encoded within it . . . It preemptively discards data in the sound file that it anticipates the body will discard later, resulting in a smaller file”  [Sterne 2006, 833]. Depending on what aspect of the sound recording one seeks to address, therefore, the MP3, which is typically categorized in traditional models such as FRBR as a derivative item of the same manifestation, could be understood as actually realizing a different event: one without the soundscape that exists beyond human hearing.

This notion of an event-centric indexicality can be made more broadly applicable to all kinds of digital textuality. Michael Witmore has written that “a text is a text because it is massively addressable at different levels of scale . . . one can query a position within the text at a certain level of abstraction” such as a word or a line, but also on the level of a theme or a character [Witmore 2010]. In a digital context in which we are attempting to model textuality for discoverability and analysis, a text can also be understood as massively and differently addressable and therefore indexical according to how the protocols we design in the system instantiate it at a given point and time. Consider Witmore’s definition of text addressability in relation to Sterne’s description of the MP3 and McGann’s “Rationale of Hypertext” in which a central identifying feature of the hypertext is that “[un]like a traditional book or set of books, the HyperText need never be ‘complete’”; rather, the hypertext by nature “will evolve and change over time, it will gather new bodies of material, its organizational substructures will get modified”; and, “[u]nlike a traditional edition, a HyperText is not organized to focus attention on one particular text or set of texts. It is ordered to disperse attention as broadly as possible”  [McGann 2001, 71]. With such a text, Witmore’s levels of abstraction are exponentially complicated by the digital text’s complexities. The digital text can become differently discernible and interpreted by different audiences with each different manifestation or code performance. In web pages created “on the fly” with PHP (Hypertext Preprocessor), for example, the same URI (Uniform Resource Identifier) or URL (Uniform Resource Locator) could take a reader to a different textual instantiation each time. As McGann reminds us, with hypertext, “[o]ne is encouraged not so much to find as to make order -- and then to make it again and again, as established orderings expose their limits . . . [the edition] will incorporate and then go beyond its initial design and management”  [McGann 2001, 71]. Event-centric attributes and relations have an indexical performativity across all texts (written, verbal and AV materials) in that the technological context in which the text has been created and “served” or delivered to the scholar is not only reflected in the item, it creates our understanding of the item’s itemness (such as “a page created on the fly”) and therefore directly impacts our understandings of its textuality more generally.