DHQ: Digital Humanities Quarterly
2014
Volume 8 Number 2
2014 8.2  |  XML |  PDF |  Print

A Design Methodology for Web-based Sound Archives

Annie Murray  <amurr_at_ucalgary_dot_ca>, University of Calgary
Jared Wiercinski  <jared_dot_wiercinski_at_concordia_dot_ca>, Concordia University

Abstract

Well-designed digital tools facilitate the creation of new knowledge in the humanities. Good design is user-centered, focused, and needs-driven, all of which depend on a rich understanding of the target audience or end user. Unsworth’s scholarly primitives [Unsworth 2000] and the work of Palmer, Teffeau and Pirmann [Palmer et al. 2009] on scholarly information practices provide a framework for understanding how humanities scholars do their work. We propose applying this framework to the design of a spoken word archive, with the aim of designing a digital tool that is optimized for the documented practices of scholars. We propose that listening and annotation are key activities of humanities scholars performing literary criticism of audio recordings. Taking the SpokenWeb poetry project as an example, we discuss how designing a web-based tool with these key activities in mind could facilitate close and critical engagement with recordings of spoken poetry. We present a methodology for designing a web-based sound archive for literary criticism and we propose features and functionalities that facilitate this criticism.

[P]roviding the collections and tools needed for producing new scholarship is arguably the most important role for cyberinfrastructure and will require a digital resource base “that is developed for specific scholarly purposes”.

Palmer, Teffeau & Pirmann, Scholarly Information Practices in the Online Environment: Themes from the Literature and Implications for Library Service Development  [Palmer et al. 2009]

Introduction

1
As more digital libraries and digital humanities projects are developed, it is crucial to ensure that they are designed with the user experience in mind so that they are useful, sustainable, and can help generate new methodologies and knowledge in the humanities. Academic and memory institutions increasingly recognize the importance of acquiring and making available unique or primary source materials that will stimulate new scholarship. As rich scholarly resources are increasingly made available on the Web, scholars and designers need to create user interfaces that take into account how humanities scholars work and behave. Instead of expecting researchers to conform to an interface that does little more than present a collection, we should design scholarly tools that conform and adapt to the detailed and documented practices of scholars. We should create tools to disseminate primary source materials, and these tools should be supportive of the formative stages of the research and writing practices of humanities scholars.
2
The researchers from the SpokenWeb project, based in Montreal, Canada, are designing a digital spoken word archive of digitized archival recordings of poetry readings given in Montreal from 1966-1974 by major North American poets. In doing so, we hope to develop new forms of critical engagement with literary recordings and to develop a comprehensive, modular and adaptable template for the handling of spoken word archives. Eventually, we hope that other scholars and institutions with analogous holdings of literary recordings can adopt and adapt SpokenWeb for their own recordings.
3
The primary goal of the SpokenWeb project is to create a sound archive that encourages scholars to engage with the sound recordings in ways that facilitate their research. Whereas for many sound archives the focus is accessibility (i.e. simply making a collection of sound recordings web accessible), SpokenWeb’s focus is interactivity and productivity. The aim is to include design features grounded in empirical research on the needs of humanities scholars, in order to facilitate the types of activities and tasks that are the building blocks of their research, and also to experiment with innovative features that help scholars engage with the sound archive in generative ways that they did not realize were possible.
4
UbuWeb is a web-based educational archive of avante-garde audio, visual and textual material, where the emphasis is strictly on accessibility: sound recordings on the site are accessible via a minimalistic playback interface where the only options are the playback and bookmarking of recordings. SpokenWeb, in contrast, aims to include features and tools which facilitate scholarship directly into the web interface through tools to compare multiple sound recordings or transcripts side-by-side, tools for sound visualization and playback manipulation, and the ability to annotate a poetic work with scholarly commentary.
5
While many scholars use sites like UbuWeb for their scholarly activity, most of that scholarly activity likely takes place outside the site. Users can download files from UbuWeb, for example, for closer analysis of a given piece. The minimal design of UbuWeb does not leave room for an extensive array of tools to help scholars engage with the site’s content. SpokenWeb’s aim is to design a sound archive that promotes in-site scholarly activity. Thus the goal is to have scholars both access and engage with the content on the web.
6
On a technical level, SpokenWeb is built using the WordPress content management system (CMS) and incorporates SoundCloud’s media player with its application programming interface (API). While we are evaluating several CMS systems, WordPress is the current front-runner based on its large user community and high degree of replicability. This ease of replication is important given our stated goal of creating a sound archive “recipe” that other cultural heritage institutions can follow, in order to open their own collections to scholarly activity. SoundCloud’s robust media player and API is particularly important, as it allows us to experiment with innovative features such as sound visualization, tethering the audio playback to a written transcript, and playback manipulation, features that all depend on an API. Archival copies of the sound recordings are stored offline in WAV format. Access copies are MP3s (encoded at 320 kilobits per second) that are streamed via the SoundCloud player. For copyright reasons, the majority of the sound recordings are password protected, with access limited to selected users for educational purposes. A small number of files are publicly accessible.
Screen capture of audio playback on the SpokenWeb site.
Figure 1. 
Screen capture of audio playback on the SpokenWeb site. Among the site’s features are sound visualization, in the form of a waveform display, and tethered playback, where the transcript is hyperlinked to different time points in the playback.
7
Using 89 recordings of poetry readings from a dynamic period in North American poetry, the SpokenWeb team of literary scholars, designers, computer scientists and librarians is developing a flexible and modular interface that tries to foster a critical practice of close listening. Close listening is an approach and practice of deep engagement with performed or recorded poems (sometimes referred to as audiotexts) that has been developed and encouraged by poet Charles Bernstein in his various writings and through his co-founding, with Al Filreis, of the PennSound website. Bernstein foregrounds the importance of sounded poetry, arguing that

To be heard, poetry needs to be sounded - whether in a process of active, or interactive, reading of a work or by the poet in performance. Unsounded poetry remains inert marks on a page, waiting to be called into use by saying, or hearing, the words aloud.  [Bernstein 1998, 7]

8
Bernstein affirms that the sound of poetry contributes to its meaning, and that “poets, especially twentieth-century innovative poets, work with sound as material, where sound is neither arbitrary nor secondary but constitutive”  [Bernstein 1998, 4]. He also cites the poetry reading in particular as

...one of the most important sites for the dissemination of poetic works in North America, yet studies of the distinctive features of the poem-in-performance have been rare (even full-length studies of a poet’s work routinely ignore the audiotext), and readings - no matter how well attended - are rarely reviewed by newspapers or magazines).....A large archive of audio and videos documents, dating back to an early recording of Tennyson’s almost inaudible voice, awaits serious study and interpretation.  [Bernstein 1998, 5]

9
The purpose of this article is to address a specific format (digital audio) used by researchers undertaking a particular humanist activity (literary criticism) when they engage with recordings of poetry readings. Drawing upon the literature of scholarly information use and behavior, we determined that two key activities in literary criticism - reading and notetaking - would likely find their corollary activities of listening and notetaking in the online audio environment. Our desire to develop a sound archive based on the actual behavior of humanities scholars is challenged by limited knowledge of how scholars of any kind conduct research that relies on listening instead of reading. Despite our goal to develop tools appropriate to the literary enterprise, criticism of sounded poems is new and consequently under-documented within the field of information studies. In this absence, we are (somewhat) hopefully adapting known behaviors of literary scholars and imagining how they might be replicated or adapted in an online audio environment. While we fully intend this to be an exercise in evidence-based design, we will rely on an iterative design process that includes user testing to elucidate user needs.
10
There is a need for research that explores how scholars engage with audio recordings in their work. Ideally, observation of researchers using audio formats would be the basis for a considered ethnography of scholarly listening practices. Historically, audio recordings have presented challenges as a research format. Recordings are often stored on fragile and degrading magnetic media, require specific and possibly obsolete playback equipment, and are hard to manipulate and annotate. As digital objects, sound recordings are still somewhat difficult to navigate, annotate and manipulate. Most media players in sound archives tend to offer very limited playback functionalities. Digital audio presents interesting options for the navigation and annotation of sound; options that are simply not possible in the analogue realm. With well-developed digital audio workstations, we may be able to “unlock” much media content from earlier format limitations.
11
We explore models of scholarly behavior and investigate the different phases of scholarly research in order to reveal what specific scholarly functions we could and should facilitate in an online sound archive. Taking into account other models of research processes developed by researchers who studied literary critics or music scholars, we speculate what system requirements or design features might best suit literary criticism of sounded poems. We draw upon cognitive and multimedia studies that ground learning in more than one sense modality at a time (summarized in [Murray and Wiercinski 2012]. For the sake of brevity, we focus on two activities that are central to literary criticism - reading and note taking - and suggest features to facilitate and enhance these core activities in a web-based sound archive. We are guided by the following idea expressed in Our Cultural Commonwealth that: “providing the collections and tools needed for producing new scholarship is arguably the most important role for cyberinfrastructure and will require a digital resource base ‘that is developed for specific scholarly processes’ ” (cited in [Palmer et al. 2009, 34]). In this spirit, we present a brief design methodology at the paper’s conclusion, with the goal of responding to known scholarly habits and processes.

Scholarly Primitives, Information Activities, and Practices

12
John Unsworth proposed the notion of scholarly primitives, which are “basic functions common to scholarly activity across disciplines, over time, and independent of theoretical orientation”  [Unsworth 2000]. These intentionally abstract primitives, the atomic elements of scholarly activities and processes, include the following:
  • Discovering
  • Annotating
  • Comparing
  • Referring
  • Sampling
  • Illustrating
  • Representing
13
Scholarly primitives “are basic to scholarship across eras and across media,” and arose from the need to build improved networked tools for humanities scholars. The goal was to better understand the scholarly process and its composite activities in order to develop tools that facilitate them, to “imagine some basic functions of scholarship that might be embodied in tools which, given a common architecture, could be combined to accomplish higher-order (axiomatic) functions”  [Unsworth 2000]. Unsworth’s list was offered in the spirit of starting a conversation, and was not meant to be definitive or exhaustive.
14
Palmer and her colleagues refined the scholarly primitives concept by emphasizing “the explicit role of information in the conduct of research and production of scholarship”  [Palmer and Cragin 2008] and by “emphasizing a sense of the primitive as something at the base or beginning of a larger process”  [Palmer et al. 2009]. Palmer et al. modify Unsworth’s vocabulary and propose a more elaborate framework that consists of scholarly information activities which include searching, collecting, reading, writing, and collaborating. The activities are then comprised of more granular activities, which they label as primitives. Further, they propose the idea of “cross-cutting primitives” which are component, granular activities that are not tied to any one particular information activity and which are typically applicable to more than one.
The five core scholarly activities and their primitives [Palmer et al. 2009]
  • 1. Searching
    • 1.1 Direct searching
    • 1.2 Chaining
    • 1.3 Browsing
    • 1.4 Probing
    • 1.5 Accessing
  • 2. Collecting
    • 2.1 Gathering
    • 2.2 Organizing
  • 3. Reading
    • 3.1 Scanning
    • 3.2 Assessing
    • 3.3 Rereading
  • 4. Writing
    • 4.1 Assembling
    • 4.2 Co-authoring
    • 4.3 Disseminating
  • 5. Collaborating
    • 5.1 Coordinating
    • 5.2 Networking
    • 5.3 Consulting
  • 6. Cross-cutting Primitives
    • 6.1 Monitoring
    • 6.2 Notetaking
    • 6.3 Translating
    • 6.4 Data Practices
15
Whereas Unsworth is deliberately format neutral, Palmer et al. are primarily grounded in a text-based world of information. Their framework includes the activity of reading - a text-based activity - but excludes listening and viewing, activities that are associated with audio, video, or image formats. An audio recording of an interview or a video recording of a news broadcast contain information in much the same way that a text document does. So for our purposes, we remedy this text-centric tendency by supplementing Palmer’s scholarly activity of reading with the additional activities of listening and viewing. With audio formats, the primitives tied to reading (i.e., scanning, assessing, and re-reading) also roughly translate to listening, with a slight adjustment. The activity of listening, for example, would include the primitives of scanning, assessing, and re-listening.
16
The multi-institutional and interdisciplinary Project Bamboo strikes a balance between Unsworth’s minimalist, abstract and format-neutral primitives and Palmer et al.’s more detailed but textually-anchored information activities by introducing “themes” of scholarly practice. These “themes” originate from a workshop that “...brought together scholars, IT professionals, and librarians from around the world to chart a direction for cyberinfrastructure development in the humanities”  [Project Bamboo 2010, 1].
17
Project Bamboo’s framework proposes additional format-neutral activities such as modeling, visualizing, and synthesizing which enrich the contributions of Unsworth and Palmer. Table 1 shows the relationship between the Bamboo scholarly themes, the Unsworth primitives, and Palmer et al.’s scholarly information activities [Project Bamboo 2010].
Bamboo theme of scholarly practice Unsworth primitive OCLC scholarly information activity
Gathering / Foraging Discovery Searching (direct searching, chaining, browsing, probing, accessing)
Synthesizing / Filtering Comparing
Sampling 		Collecting (gathering, organizing)
Contextualizing Referring Searching (chaining, browsing, probing)
Collecting (organizing)
Cross-cutting (monitoring)
Conceptualizing,
Refining and Critiquing 		Illustrating
Representing
Comparing 		Reading (scanning, assessing, rereading)
Cross-cutting (notetaking, translating)
Writing (assembling)
Collaborating (consulting)
Documenting methods Representing Writing (disseminating)
Cross-cutting (translating)
Managing data Discovering
Referring
Representing 		Searching (accessing)
Collecting (organizing)
Collaborating (coordinating, consulting)
Annotating /
documenting 		Annotating Writing (assembling)
Cross-cutting (notetaking)
Modeling / visualizing Illustrating
Representing 		Cross-cutting (translating)
Writing (assembling)
Overlapping teaching and research Representing Collaborating (coordinating)
Cross-cutting (translating)
Sharing / dissemination / publishing Representing Writing (disseminating)
Funding Suggested parenthetically No analogue
Collaborating Common thread throughout scholarly primitives, not listed separately Writing (co-authoring)
Collaborating (coordinating, networking, consulting)
Citation, credit, peer-review Referring Reading (assessing)
Writing (dissemination)
Collaborating (consulting)
Table 1. 
The relationship between the Project Bamboo scholarly themes, Unsworth’s primitives, and Palmer et al.’s scholarly information activities ([Project Bamboo 2010, 2–3], reproduced with permission)
18
For us, the conceptual framework of the scholarly primitives functions as a type of abstract checklist for building new digital tools. The primitives call attention to the types of activities that scholars need to do. Accordingly, we need to plan and design environments with functionalities that will facilitate the primitives.
19
To take Unsworth’s primitive of “discovery” as an example, we will ensure that the SpokenWeb sound archive facilitates scholars’ discovery of the content within our archive, and that it promotes discovery of connections and relationships between individual items such as poems, particular recordings, transcriptions, and so forth. This could take the form of search functionality, perhaps in the form of a simple search box, but could also include more sophisticated browsing tools which allow a scholar to explore the recordings without having a fixed starting point (i.e. a particular author, topic, or terminology) in mind. For instance, our browsing tools should facilitate serendipity.
20
Unsworth’s primitives are a good starting point for this type of checklist, but by their very nature they are abstract and general. To improve our design checklist, we look at Palmer et al.’s scholarly information activities and the Bamboo project’s themes of scholarly practice. Both of these frameworks are more specific than Unsworth’s list, and are grounded in empirical research that shows, among other things, that the types of activities that scholars engage in will vary as a result of their disciplinary approach. From a design perspective, then, questions about the intended audience begin to emerge. The SpokenWeb project has to take into account the discipline-specific behavior of literary scholars.
21
Palmer et al.’s Scholarly Information Practices in the Online Environment summarizes scholarly information activities across disciplines. Figure 2 provides a summary of the scholarly primitives associated with the humanities and sciences, as well as those involved in interdisciplinary ventures.
Venn diagram showing three intersecting domains: Humanities, Sciences, and Interdisciplinary.
Figure 2. 
Scholarly primitives associated with disciplinary approach ([Palmer et al. 2009, 35], reproduced with permission
22
Not surprisingly, scholars from different disciplines tend to engage in certain activities more than others. For example, browsing and rereading are important for humanities scholars, as we see in Figure 2, whereas direct searching and data sharing are more important for scientists. Our project pays close attention to Palmer et al.’s conclusion that “...humanities scholars and other researchers deeply engaged in interpreting source material rely heavily on browsing, collecting, re-reading and notetaking”  [Palmer et al. 2009, 35]. Accordingly, we are designing the SpokenWeb with features that facilitate these core activities.
23
In addition to questions of audience we want to pay close attention to the related and important question of format. As previously discussed, Palmer et al. are somewhat biased towards text-based information. Palmer et al. do list the specific formats that are typically used by a given discipline (e.g. in the “Source materials by discipline reported in the RLG reports” table on page 5), but they do not talk about the relationship between scholarly activities and non-text formats. Non-textual formats are increasingly used in scholarship, and they deserve careful attention from designers of tools. Scholarly activities change depending on the information format being used. In our view, activities that need to be supported for the use of audio formats are listening, re-listening, and manipulating the playback of recordings.
24
As noted, we have by necessity extended the scholarly information activity of “reading” as a placeholder for a slightly broader spectrum of intake or “consumptive” activities extending beyond the world of text into audio and visual realms. Our project supplements the list of core humanities primitives that we will prioritize in the site’s design (i.e., browsing, collecting, re-reading and notetaking) with the additional scholarly activities of listening and viewing, and their corresponding primitives (e.g., re-listening, re-viewing, et cetera).
25
Unsworth’s primitives thus serve as a useful starting point or checklist for those who are designing digital tools. Palmer et al.’s scholarly information activities and Project Bamboo’s themes of scholarly practice provide empirically-grounded considerations for that checklist and raise important questions of audience and format.

Reading Audio and Annotating Sound

26
We were unable to find empirical research that provided a detailed account of how scholars work with audio recordings.
27
Clara Chu’s descriptive model of the research process of literary critics shows the role of information sources at each stage, though audio recordings are not specifically addressed. As she notes, “the body of literature dealing with the information needs and uses of literary scholars is small”  [Chu 1999, 249]. Her study identifies the following phases of literary critical work:
  • Idea Generation
  • Preparation
  • Elaboration
  • Analysis and Writing
  • Dissemination
  • Further Writing and Dissemination
28
Chu notes that literary criticism is not a “clearly defined step-by-step sequential process.” In this model, information use, and the reading and re-reading of the text being studied occurs extensively in both the Preparation and Analysis and Writing stages. Naturally, we then ask how literary criticism might occur in a digital audio environment. Does the use of sounded poems yield any variation to the model of literary criticism suggested by Chu? Indeed, she recommends that a study is needed of the “way information technologies and the availability of electronic texts may be affecting literary critics’ work, communication, and information seeking”  [Chu 1999, 270].

Reading Audio

29
We wonder the extent to which the practice of careful reading and re-reading of text is replicated when a literary recording, as opposed to a printed text, is the object of study. To facilitate the practice Charles Bernstein has termed “close listening,” we are building SpokenWeb to explore how scholarly reading practices might inform scholarly listening practices. We assume that critics will need to listen repeatedly to particular passages in a web-based audio environment. Thus the chosen media player should facilitate the user’s ability to listen, slow down, pause, loop and repeat particular passages of the recording. Further, there may be a feedback loop between the listening tools or playback device and the process of close listening. Close listening could be a much richer experience in a well-designed web-based sound archive, with features that facilitate sophisticated manipulation of playback, than might be possible with playback on an analog reel-to-reel machine, for example. New tools, with the right features, could facilitate new ways of close listening.
30
Given the absence of a body of literature which explains how scholars approach a sounded as opposed to a printed poem, we provisionally suggest that the attentiveness to the text, signaled by careful and repeated reading of a poem, will have strong carryover to the way a scholar would listen critically to a performed poem. To facilitate the reading of poems with both the eyes and the ears, the SpokenWeb has integrated transcripts of the audio recordings into the web interface. It is possible for scholars to listen to and/or read the content of the poetry readings. Since much learning we do is multimodal in nature, we recognize that other modalities inform listening [Murray and Wiercinski 2012]. We believe having the option to read words as one listens to poems could benefit the researcher by enhancing comprehension and analysis of the recording.
31
The ways in which scholars read is varied. In her work interviewing humanities scholars about their practices, Oya Rieger noted:

Reading, which is a critical process in research, is a multiply nuanced concept, with each type serving a unique purpose. The scholars I interviewed referred to multiple types of reading such as deep reading, close reading, skimming, or eyeballing. They emphasized that re-reading was a significant part of interpretive work and involved periodic interactions with selected texts.  [Rieger 2010, 87]

32
Chu suggests that reading is a process that occurs repeatedly throughout the different phases of a literary critic’s work [Chu 1999]. Humanities scholars in particular tend to read and re-read texts, and as Palmer, Teffau and Pirmann suggest, “Re-reading is one of the primary reasons that scholars build personal collections. For humanities scholars, rereading a work is a significant part of interpretation and analysis”  [Palmer et al. 2009, 21]. Basic audio players featured in most web-based collections of literary recordings are not conducive to repeated listening of poems, as they do not allow the listener certain basic controls over the recording. In the SpokenWeb interface, we strive to allow the listener more control over the recording so that the reading/listening process can be conducted as much as possible on the scholar’s terms. For this reason, we have adopted Soundcloud’s media player, designed for electronic musicians and DJs, because it allows for more user control of the recording. Many libraries and archives are limited in digitization resources and could become focused on the digitization process itself and the attendant storage or metadata creation costs to the extent that the usability of the digitized content understandably becomes a low priority. For this reason, we hope to contribute an adaptable and user-friendly platform for providing access to digitized or born-digital audio recordings.
33
Perhaps the work habits of music scholars can shed light on how a group of researchers works with recordings and textual sources simultaneously, since a composer or music historian might be listening to a performance and studying a score at the same time in the way a literary critic might sit and listen to a recording and consult a print book or a transcription at the same time.
34
According to Brown, the most commonly used information resources in music research are print, audio, and video sources. She notes that “[t]he relatively high ranking of recordings shows the great importance of listening in the research process”  [Brown 2002, 82]. While the study did not provide details about the scholarly listening process, one participant in the study remarked that “one of the things that is so basic to this kind of work is that you are constantly working back and forth from the example [a music recording] to the text [the criticism being written], back to the example, back to the text”  [Brown 2002, 86]. She also notes “this constant going back and forth is a real feature of writing about music”  [Brown 2002, 86]. We suspect that this is how criticism of sounded poems might be performed as well: the critic is constantly listening and re-playing parts of the recording as she writes. Therefore, it is essential to make this kind of switching between activities (listening, reading, and writing) as natural and seamless as possible in a web-based environment.
35
It is often noted how Web browsers and online research materials tend to lead to a more shallow kind of reading, or at least change the nature of the reading experience [Nicholas and Clark 2012]] [Cull 2011] [Carr 2010]. These cultural critics tend to mourn the gradual reduction of sustained reading practices that largely characterized reading in the print world. We thus wonder how the practice of literary criticism will evolve with audiotexts and the increasing use of online resources in general. Not only are scholars engaging with an audio performance of a poem, but they are also doing so through a web-based tool. What does this mean for the critical practice of a literary scholar? This type of literary study will bring with it the need to explore how listening online compares to reading in print; the new habits that would emerge from such a practice; and a consideration of what it means to work with poetry in auditory instead of visual and typographic terms. Finally, what are the implications of our interactions with sound being mediated through a visual interface on a computer screen?

Annotating Sound

36
Having established the primacy of reading for humanities scholars, and the emerging importance of reading and listening for scholars of sounded poetry, we can now address a related scholarly primitive, annotation, and the related information practice of notetaking. Brockman et al. (as cited in [Palmer et al. 2009] and Palmer & Neumann [Palmer and Neumann 2002] establish reading and writing as fundamental activities for humanities scholars. Scholarly notetaking is an activity that enhances reading and forms a bridge towards a later finished work of written criticism. As a scholarly practice, notetaking is critical to both the reading and writing phases of literary work.
37
Hillesund’s [Hillesund 2010] study of a group of humanities and social science researchers sheds lights on scholarly work processes in its efforts to describe reading behavior all the while acknowledging that “reading is a most familiar activity, solidly packed and sedimented. It is one of those deep and complex phenomena that are so close to the mundane that their basic traits are hard to discover and talk about.” What becomes clear is how physical and material the act of reading is, how tied to the body, space and objects. According to Hillesund, annotation improves comprehension of text, slows the pace of reading, and helps scholars to record and remember points. The “annotation habit is probably a way of processing information, giving it time to fit into schemas in long-term memory and provide time and space for reflection and discovery of inferences.” Given the “kinaesthesia, and motor control (dexterity) as well as tactile and visual perception” that characterize scholarly reading and writing, we can see how important and challenging it is to develop ways of delivering digital content that work in keeping with deeply embedded physical and mental processes associated with reading and writing. Thus, reading and writing are deeply intertwined in the scholarly process, and notetaking is a central behavior to both. Most digital tools have a long way to go in facilitating this most critical activity in scholarship. Again, we note that tools for text annotation and manipulation have received much more development and critical attention in the Digital Humanities, while tools for annotation of audio formats instead take their cues from web-based entertainment sites, e-learning environments and online music delivery and discovery tools.
38
The sheer volume of valuable but fragile analogue audio material in libraries and archives suggests that many institutions will face the challenge of not only offering digitized recordings, but delivering them in such a way as to make them useful to scholars.
39
Annotating a digital source such as a recording may come less naturally to a researcher, however. Bradley and Vetch [Bradley and Vetch 2007] and Hillesund [Hillesund 2010] have observed that digital tools are not as likely to support annotation. This is something to strive for in interface design since notes represent the beginnings of written criticism. As Audenaert and Furuta suggest, having note taking capabilities in digital tools would greatly support scholarship:

Notetaking, however, is not merely the semi-formal representation of facts. Instead it is an integrated part of the iterative writing process. Scholars think of their notes specifically in terms of how that information will be represented in the published form of their work and organize them accordingly. To be successful, support for notetaking should be designed with this in mind and should provide a clear path to transition from information notes to a final publishable manuscript.  [Audenaert and Furuta 2010, 290]

40
They also suggest the importance of being able to export notes from a given system into a writing software such as Microsoft Word [Audenaert and Furuta 2010, 289]. Therefore, if skillfully deployed, annotation features in a digital environment would support both scholarly reading and writing, offering a more natural and seamless feel to a possibly unfamiliar digital workspace. Moreover, there is no need to assume that annotations will take the form of written or typed comments. In a web-based sound archive, the researchers may wish to record a spoken comment or annotation that responds to a particular passage. Similarly, an annotation might take the form of an image or a video. Our early steps in providing an opportunity for users to annotate recordings in SpokenWeb is through the adoption of SoundCloud’s commenting feature which allows users to make comments at any point in a recording. A future phase in development would establish a means for exporting the time-stamped comments into the scholar’s preferred writing environment.

Design

41
In recent years, considerable work has addressed how a humanities scholar’s “workbench” or digital research environment might look and function [Toms and O’Brien 2008] [Toms and Flora 2005] [Palmer and Neumann 2002] [Rieger 2010] [Audenaert and Furuta 2010] [Project Bamboo 2010]. Similarly, work has been undertaken to determine user requirements for the humanities [Bowman et al. 2007].
42
We wish to direct attention specifically to how digitized or born digital audio recordings might best be optimized in an online work environment. Despite a lack of research on the information behavior of scholars who work with audio formats, we establish a checklist of features and system requirements for a web-based tool for analyzing literary recordings. Drawing upon an analysis of humanities scholarship, we suggest that the following features (see Figure 3) would serve the various activities, primitives and overall scholarly behavior central to the humanities scholar’s work.
Figure 3. 
From Primitives to Design Features. This table maps some of Unsworth’s scholarly primitives to Palmer at al.’s scholarly information activities, then proposes equivalent activities specific to a sound archive. We suggest features that could enhance and support these audio-based activities and provide examples when possible.
43
Below is a list of web sites and audio tools listed in Figure 3:
44
As Palmer and Neumann have noted,

interface design and human-computer interaction has, for the most part, ignored the needs of the humanities scholar. When the interface is considered, it is limited to discussions of items such as the Boolean logic problem and the vocabulary mismatch between system provision and user understanding ... Despite the profound impact of technology on this scholarly community, little is known about how computers have affected humanities scholars’ work flow, unless it is to say that scholars adopt technologies when they augment established research practices  [Palmer and Neumann 2002, 104–5]

. By exploring how researchers use a particular type of primary source, we hope to contribute to this format-based gap in user behavior information.
45
Palmer et al.’s study suggests that “collecting, reading, writing and collaborating, and especially the cross-cutting primitives, are much more sparsely supported online and often only as a byproduct of existing systems rather than as a deliberately designed feature”  [Palmer et al. 2009, 42]. Deliberately designed features are precisely what we want to bring to web-based audio interfaces.
46
Part of the design phase of the project will examine which activities scholars carry over into the digital realm. We are intrigued by the possibility of developing a tool that can offer features to support existing habits, but also possibly enable new activities to help scholars. As Rimmer et al. note, “the media used (paper, books, shelves, etc. or screens, keyboards, mice) afford different kinds of interactions”  [Rimmer et al. 2008, 1389]. Since humanities scholars often function with habits based in both the print and electronic world, we hope to facilitate a smooth overlap between these worlds, and develop an interface that allows for seamless integration with normal scholarly activities.
47
Our evidence-based design methodology is the means by which we can build a tool humanities scholars will use and exploit as readily as their traditional textual sources. Rieger notes that “multimodal environments require distinct modes of engagement from writers and readers”  [Rieger 2010, 112]. While there is a long history of literary critics working with printed texts, the history and methods of scholars working with audio recordings remain underdocumented or unknown. As such, it is much more difficult to anticipate user behavior within such environments. We do not yet know how scholars use web-based sound recordings in their work. We do not know what interesting habits could emerge. Thus we have begun the design of SpokenWeb with the goal of delivering a flexible and modular user experience. The use of web-based audio files for literary research will be inherently multimodal since the recordings will be embedded in a graphical user interface and the listener will have the option of consulting transcripts of the recording or focusing solely on listening.
48
The LAIRAH project analyzed 21 digital humanities projects in order to elucidate the factors that led to their use and uptake in scholarly communities. Noting that only two projects carried out formal user testing in the early stages of project design, the researchers observe:

User consultation was relatively rarely undertaken, despite the fact that it helps projects to design effective resources, and to avoid developing in ways that users may find over complicated or confusing. However, user testing, like disseminating information, is a skill that most humanities scholars have not acquired. It is therefore important that digital projects should be willing to work with those who already have expertise in this area, for example, researchers from Human Computer Interaction, Library and Information Studies, or practitioner librarians.  [Warwick et al. 2008, 93]

49
Indeed, user awareness, engagement and testing was one of the LAIRAH’s six concluding recommendations for good practice in the construction of digital humanities projects [Warwick et al. 2008, 394]. We hope that in both observation and user testing we can glean information that could be useful to other digital projects intended for scholars who make extensive use of audio recordings. As Palmer and Cragin note “directly engaging domain scholars as collaborators or partners in research design and interpretation of results is important for reducing the chain of inference required to determine implications for the design and development of technologies for specific research communities”  [Warwick et al. 2008, 198].
50
As much as we believe digital humanities projects should be more oriented around user experience and informed by well-conceived user testing, we simultaneously see the merit of the viewpoint offered by Gay and Hembrooke in their book Activity-Centered Design: An Ecological Approach to Designing Smart Tools and Usable Systems:

User-centered methods also fail to identify future uses, needs, and problems that users and developers might not independently envision. This is especially important for nascent technologies, which people will inevitably view in the relatively constrictive terms of old technologies (such as using a digital hand-held machine to replace the old portable audiotape guide system in a museum).  [Gay and Hembrooke 2004, 19]

51
We want to ensure that our sound archive performs in a way that reflects scholarly practice, but since web-based audio is relatively new, we look forward to anticipating and designing around new or unexpected behaviors. To this end, we believe that the Unsworth primitives, Palmer et al.’s scholarly activities, and the Bamboo themes can function as a checklist of the types of scholarly activities that we need to design for; they are empirically based and remind us to design for known needs and requirements. But the primitives and themes can also be helpful in generating new features that take advantage of the opportunities afforded by new technologies. As stated above, user centered methods can be limiting in that they are inherently conservative, backwards-looking design methods. The Unsworth primitives can be especially helpful in this regard. They are abstract in nature and are not tied to any particular discipline or format and can therefore help us to think creatively to envision new features that take full advantage of the potential afforded by new technologies while minimizing the impact of the “constrictive terms of old technologies”  [Gay and Hembrooke 2004, 19]. The Unsworth primitives can, in a sense, function as a type of thought recipe whereby a designer is reminded of a general scholarly activity (e.g. discovery) that will need to be designed for, but with nothing further: the designer is forced to improvise and experiment with different ways of how to realize this primitive in a new digital substratum.
52
Ideally, then, the interface design will provide the more familiar and comfortable features that facilitate the types of activities that scholars know that they need to do, but will also be generative and experimental in taking advantage of the opportunities provided by new technology and will develop new features that scholars will find advantageous, and will come to need, but of which they are currently unaware.

An Evidence-based Design Methodology

  • Become familiar with Unsworth primitives, Palmer et al’s scholarly information activities, and the Bamboo themes of scholarly practice.
  • Consult available research (e.g. [Palmer et al. 2009]) to help identify all of the primitives and practices that are the most relevant to the project’s target user community. (For example, our project involves literary scholars, so we have chosen reading and annotation as key activities to support.)
  • Identify the information sources and format types specific to your project.
  • Using information gleaned from steps 1-3, adopt, adapt, or create features that would facilitate relevant primitives and practices. [1]
  • Conduct user testing to evaluate the proposed features. Redesign or modify as needed.

Conclusion

53
We have explored the applicability of various models and conceptions of scholarly behavior, particularly of humanities scholars, to interface design for a digital spoken word archive. Scholarly primitives [Unsworth 2000], scholarly information practices [Palmer et al. 2009] and the Bamboo themes of scholarly practice (n.d.) have all informed our sense of which scholarly behaviors need to be supported and facilitated in a sound archive.
54
The question of format looms large in our project, since audio formats present basic challenges of accessibility, usability and annotatability. Since our design project involves delivering and providing a workspace for analyzing a challenging format, we have chosen to focus attention on two scholarly habits - note taking and deep/close reading - that are especially relevant for literary critics. We have suggested ways in which a web-based tool can better facilitate these core activities so intrinsic to the research and writing process of the literary critic.
55
Surprisingly little information is available that reports on how scholars use audio recordings. This is striking given the historical and current importance of audio recordings in fields such as music, law, anthropology, oral history and communication studies, not to mention the ubiquity of sound recordings on the web today. Most studies of scholarly behavior analyze how scholars use texts, and thus the models and frameworks to describe scholarly activities are biased towards text and print-based information resources. We have tried to imagine how scholars use audio recordings based on well-established, even empirical, observations of scholarly behaviors. But since not much is known specifically about how researchers use web-based audio recordings, it has been necessary to draw upon evidence-based design principles developed by scholars of e-learning and multimedia learning. Thus, in order to begin to articulate design principles for web-based spoken word archives, our project is assembling evidence and principles from cognitive studies and web design and is investigating how they could apply to a humanities research tool.
56
The potential of deeper scholarly engagement with born digital or digitized sound recordings is promising. We see a fortuitous convergence of scholarly and technical factors, all of which suggest that web-based research using audio recordings will be made easier. We can benefit from useful models of scholarly behavior, the growing availability of archival media on the web, the deepening understanding of multimodal learning, the growth of performance studies, the ever-expanding array of functionalities in web-based applications and the potential for a new mode of literary analysis. It is our hope that we will contribute to an emerging and evidence-based list of design criteria that is based on scholarly behavior. Ideally, we can contribute to the formulation of functional requirements for scholarly software for humanities scholars who use audio recordings. Web-based tools need to fit in with established behaviors and practices in literary criticism, but we should also actively facilitate the emergence of sophisticated and responsive tools which can contribute to new or enhanced practices in literary scholarship. The creation of new knowledge in the humanities depends not only on better understanding the role of sound in the work of humanities scholars, but also incorporating this knowledge into the design of sound archives.

Notes

[1] The mapping from primitives and practices to interface feature may be relatively straightforward. For example, in our project designed for humanities scholars, the scholarly practice of notetaking maps directly into a feature that facilitates audio annotation. Alternatively, the primitives and practices can function more abstractly, generating thinking and questions that could lead to experimental features. For example, Unsworth’s discovery primitive challenges designers to create new features that facilitate such a rich primitive in the unique context of their project.

Works Cited

Audenaert and Furuta 2010 Audenaert, N. and Furuta, R. "What Humanists Want: How Scholars Use Source Materials", Proceedings of the 10th Annual ACM/IEEE Joint Conference on Digital Libraries, University of Queensland, Gold Coast, Queensland, Australia, June 2010. http://dl.acm.org/citation.cfm?doid=1816123.1816166
Bernstein 1998 Bernstein, C. Close Listening: Poetry and the Performed Word. Oxford University Press, New York (1998).
Bernstein 2009 Bernstein, C. “Making Audio Visible: The Lessons of Visual Language for the Textualization of Sound”, Textual Practice, 23.6 (2009): 959-973.
Bowman et al. 2007 Bowman, A., Crowther, C., Kirkham, R. and Pybus, J. User Requirements Gathering for the Humanities: AHRC Investigators’ Report, 2007. http://www.ahessc.ac.uk/files/active/0/URH-report.pdf
Bradley and Vetch 2007 Bradley, J. & Vetch, P. “Supporting Annotation as a Scholarly Tool — Experiences From the Online Chopin Variorum Edition”, Literary and Linguistic Computing, 22.2 (2007): 225-241.
Brown 2002 Brown, C.D. “Straddling the Humanities and Social Sciences: The Research Process of Music Scholars”, Library & Information Science Research, 24.1 (2002): 73-94.
Carr 2010 Carr, N. The Shallows: What the Internet is Doing to Our Brains. 1st ed. W.W. Norton, New York (2010).
Chu 1999 Chu, C.M. “Literary Critics at Work and Their Information Needs: a Research-Phases Model”, Library & Information Science Research, 21.2 (1999): 247-273.
Ciccoricco 2012 Ciccoricco, D. “The Materialities of Close Reading: 1942, 1959, 2009”, Digital Humanities Quarterly, 6.1 (2012).
Cull 2011 Cull, B.W. “Reading Revolutions: Online Digital Text and Implications for Reading in Academe”, First Monday, 16.6 (2011).
Gay and Hembrooke 2004 Gay, G. and Hembrooke, H. Activity-Centered Design: an Ecological Approach to Designing Smart Tools and Usable Systems. MIT Press, Cambridge, Massachusetts (2004).
Hillesund 2010 Hillesund, T. “Digital reading spaces: How expert readers handle books, the Web and electronic paper”, First Monday, 15.4 (2010).
Murray and Wiercinski 2012 Murray, A. and Wiercinski, J. “Looking at Archival Sound: Enhancing the Listening Experience in a Spoken Word Archive”, First Monday, 17.4 (2012).
Nicholas and Clark 2012] Nicholas, D. and Clark, D. “ ‘Reading’ in the Digital Environment”, Learned Publishing, 25.2 (2012): 93-98.
Palmer and Cragin 2008 Palmer, C.L. and Cragin, M.H. “Scholarship and Disciplinary Practices”, Annual Review of Information Science and Technology, 42.1 (2008): 163-212.
Palmer and Neumann 2002 Palmer, C.L. and Neumann, L.J. “The Information Work of Interdisciplinary Humanities Scholars: Exploration and Translation”, The Library Quarterly, 72.1 (2002): 85-117.
Palmer et al. 2009 Palmer, C.L., Teffeau, L.C. and Pirmann, C.M. Scholarly Information Practices in the Online Environment: Themes from the Literature and Implications for Library Service Development (Report Commissioned by OCLC Research), 2009. www.oclc.org/programs/publications/reports/2009-02.pdf
Project Bamboo 2010 Project Bamboo. Scholarly Practices Report, 2010. http://www.projectbamboo.org/wp-content/uploads/Project-Bamboo-Scholarly-Practices-Report.pdf
Rieger 2010 Rieger, O.Y. Humanities Scholarship in the Digital Age: The Role and Influence of Information and Communication Technologies. PhD dissertation. Cornell University, Ithaca (2010). http://ecommons.library.cornell.edu/handle/1813/17217
Rimmer et al. 2008 Rimmer, J., Warwick, C., Blandford, A., Gow, J. and Buchanan, G. “An Examination of the Physical and the Digital Qualities of Humanities Research”, Information Processing & Management, 44.3 (2008): 1374-1392.
Toms and Flora 2005 Toms, E.G. and Flora, N. “From Physical To Digital Humanities Library: Designing the Humanities Scholar’s Workbench”. In: Siemens, R. and Moorman, D. Mind Technologies: Humanities Computing and the Canadian Academic Community. University of Calgary Press, Calgary (2005).
Toms and O’Brien 2008 Toms, E.G. and Heather L. O'Brien. “Understanding the Information and Communication Technology Needs of the E-Humanist”, Journal of Documentation, 64.1 (2008): 102-130.
Unsworth 2000 Unsworth, J. “Scholarly Primitives: What Methods Do Humanities Researchers Have in Common, and How Might Our Tools Reflect This?”, Symposium on Humanities Computing: Formal Methods, Experimental Practice, King’s College, London, May 2000. http://people.lis.illinois.edu/~unsworth//Kings.5-00/primitives.html
Warwick et al. 2008 Warwick, C., Galina, I., Terras, M., Huntington, P. and Pappa, N. “The Master Builders: LAIRAH Research on Good Practice in the Construction of Digital Humanities Projects”, Literary & Linguistic Computing, 23.1 (2008):383-396.
2014 8.2  |  XML |  PDF |  Print
Creative Commons License
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.