Current Issue
Preview Issue
Previous Issues
Preview Issue
Previous Issues
- 2020: 14.3
- 2020: 14.2
- 2020: 14.1
- 2019: 13.4
- 2019: 13.3
- 2019: 13.2
- 2019: 13.1
- 2018: 12.4
- 2018: 12.3
- 2018: 12.2
- 2018: 12.1
- 2017: 11.4
- 2017: 11.3
- 2017: 11.2
- 2017: 11.1
- 2016: 10.4
- 2016: 10.3
- 2016: 10.2
- 2016: 10.1
- 2015: 9.4
- 2015: 9.3
- 2015: 9.2
- 2015: 9.1
- 2014: 8.4
- 2014: 8.3
- 2014: 8.2
- 2014: 8.1
- 2013: 7.3
- 2013: 7.2
- 2013: 7.1
- 2012: 6.3
- 2012: 6.2
- 2012: 6.1
- 2011: 5.3
- 2011: 5.2
- 2011: 5.1
- 2010: 4.2
- 2010: 4.1
- 2009: 3.4
- 2009: 3.3
- 2009: 3.2
- 2009: 3.1
- 2008: 2.1
- 2007: 1.2
- 2007: 1.1
ISSN 1938-4122
Announcements
DHQ: Digital Humanities Quarterly
2021 15.1
AudioVisual Data in DH
Editors: Taylor Arnold, Jasmijn van Gorp, Stefania Scagliola, and Lauren Tilton
Front Matter
Introduction: Special Issue on AudioVisual Data in
DH
Taylor Arnold, University of Richmond; Stefania Scagliola, Université du Luxembourg; Lauren Tilton, University of Richmond; Jasmijn Van Gorp, Utrecht University
Abstract
[en]
Our special issue explores audio and visual (AV) data as form, method, and practice in
the digital humanities. Spurred by recent advances in computing alongside disciplinary
expansions of what counts as evidence, audio and visual ways of knowing are enjoying a
more prominent place in the field. Whether the creation, analysis, and sharing of
audiovisual data or audiovisual ways of communicating scholarly knowledge, scholars are
building compelling avenues of inquiry that are changing how we know, what we know, and
why we know in the digital humanities (DH). These epistemological shifts not only
challenge existing methodological and theoretical pathways within the field of audiovisual
studies, but most importantly defy existing knowledge hierarchies within the entire field
of DH.
Founding the Special Interest Group Audio-Visual in
Digital Humanities: An Interview with Franciska de Jong, Martijn Kleppe, and Max Kemman
Stefania Scagliola, Université du Luxembourg
Abstract
[en]
An interview with Professor Franciska de Jong (Director at CLARIN ERIC), Dr. Martijn
Kleppe (Head of Research at the KB, National Library of the Netherlands), and Dr. Max
Kemman (Researcher/Consultant at Dialogic) on the founding of the ADHO Audiovisual in
Digital Humanities (AVinDH) Special Interest Group. They are interviewed by Stefania
Scagliola (Centre for Contemporary and Digital History), who co-founded the group and is a
co-editor of this special issue.
Section 1: Annotation of AV Material as Method and Theory
Exploring Film Language with a Digital Analysis Tool:
the Case of Kinolab
Allison Cooper, Bowdoin College; Fernando Nascimento, Bowdoin College; David Francis, Bowdoin College
Abstract
[en]
This article presents a case study of Kinolab, a digital platform for the analysis of
narrative film language. It describes the need for a scholarly database of clips focusing
on film language for cinema and media studies faculty and students, highlighting recent
technological and legal advances that have created a favorable environment for this kind
of digital humanities work. Discussion of the project is situated within the broader
context of contemporary developments in moving image annotation and a discussion of the
unique challenges posed by computationally-driven moving image analysis. The article also
argues for a universally accepted data model for film language to facilitate the academic
crowdsourcing of film clips and the sharing of research and resources across the Semantic
Web.
Audiovisualities out of Annotation: Three Case Studies
in Teaching Digital Annotation with Mediate
Joel Burges, University of Rochester; Solvegia Armoskaite, University of Rochester; Tiamat Fox, University of Rochester; Darren Mueller, University of Rochester; Joshua Romphf, University of Rochester; Emily Sherwood, University of Rochester; Madeline Ullrich, University of Rochester
Abstract
[en]
This article describes Mediate: An Annotation Tool for Audiovisual Media,
developed at the University of Rochester, and emphasizes the platform as a source for the
understanding of film, television, poetry, pop songs, live performance, music, and
advertising as shown in three cases studies from film and media studies, music history,
and linguistics. In each case collaboration amongst students was not only key, but also
enabled by Mediate, which allows students to work in groups to generate large
amounts of data about audiovisual media. Further, the process of data generation produces
quantitative and qualitative observation of the mediated interplay of sight and sound. A
major outcome of these classes for the faculty teaching them has been the concept of
audiovisualities: the physically and culturally interpenetrating modes of
audiovisual experience and audiovisual inscription where hearing and seeing remediate one
another for all of us as sensory and social subjects. Throughout the article, we chart how
audiovisualities have emerged for students and ourselves out of digital annotation in
Mediate.
The Media Ecology Project: Collaborative DH Synergies to
Produce New Research in Visual Culture History
Mark Williams, Dartmouth College; John Bell, Dartmouth College
Abstract
[en]
This essay details the development and current NEH-funded research goals of The Media
Ecology Project (MEP), directed by Prof. Mark Williams and designed by Dr. John Bell at
Dartmouth. The virtuous cycle of access, research, and preservation that MEP realizes is
built upon a foundation of technological advance (software development) plus large-scale
partnership networks with scholars, students, and institutions of historical memory such
as moving image archives. The development of our Onomy vocabulary tool and NEH-funded
Semantic Annotation Tool (SAT) are detailed, including their application in two
advancement grants from the NEH regarding 1) early cinema history, and 2) television
newsfilm that covered the civil rights movement in the U.S.
MEP is fundamentally 1) a sustainability project that 2) develops literacies of moving
image and visual culture history, and 3) functions as a collaborative incubator that
fosters new research questions and methods ranging from traditional Arts and Humanities
close-textual analysis to computational distant reading. New research questions in
relation to these workflows will literally transform the value of media archives and
support the development of interdisciplinary research and pedagogy/curricular goals (e.g.,
media literacy) regarding the study of visual culture history and its legacies in the 21st
century.
Audiated Annotation from the Middle Ages to the Open
Web
Tanya E. Clement, University of Texas at Austin; Liz Fischer, University of Texas at Austin
Abstract
[en]
Current theories about the significance of annotations in literary studies are based
primarily on assumptions developed in print culture about verbal texts. In these textual
theories, the text is typically present, authorized, and centralized as the ideal text for
an ideal reader, and to annotate is to add authorized comments in a sociotechnical system
that includes publication, dissemination, and reception. To audiate is to imagine a song
that's not playing. In music learning theory, audiation is based on the concept that the
musician learns to play music by developing their own musical aptitude, her individual
interpretation of a musical score based on her particular experience of the music. This
short article introduces audiation as an alternate theoretical framing for articulating
the significance of personal literary annotations. Comparing commentary on psalms in the
Middle Ages to IIIF (International Image Interoperability Framework) web annotations, we
use the concept of audiation to situate annotations within literary study in terms of a
more capacious understanding of the individual's interpretation of text and of the reading
experience as part of an unauthorized, distributed, and decentralized system. By bringing
together theories and technologies of annotation with sound, we offer the concept of
audiated annotations as a means to re-evaluate modes of access, discovery, and analysis of
cultural objects in digital sound studies.
Section 2: Analyzing (Meta)Data
Healing the Gap: Digital Humanities Methods for the
Virtual Reunification of Split Media and Paper Collections
Stephanie Sapienza, Maryland Institute for Technology in the Humanities; Eric Hoyt, University of Wisconsin-Madison; Matt St. John, University of Wisconsin-Madison; Ed Summers, Maryland Institute for Technology in the Humanities; JJ Bersch, University of Wisconsin-Madison
Abstract
[en]
This paper introduces and unpacks several challenges faced by stewards who work with
audiovisual resources, departing from the premise that audiovisual resources are
undervalued and underutilized as primary source materials for scholarship and therefore
receive less attention in the sphere of digital humanities. It will then present original
research from the Maryland Institute for Technology in the Humanities (MITH), in
conjunction with the University of Wisconsin-Madison and the Wisconsin Historical Society,
on a project entitled Unlocking the Airwaves: Revitalizing an Early Public Radio
Collection. As a case study, Unlocking the Airwaves successfully meets these challenges by
employing strategies such as virtual reunification, linked data, minimal computing, and
synced transcripts, to provide integrated access to the collections of the National
Association of Educational Broadcasters (NAEB), which are currently split between the
University of Maryland (audio files) and the Wisconsin Historical Society (paper
collections). The project demonstrates innovative approaches towards increasing the
discoverability of audiovisual collections in ways that allow for better contextual
description, and offers a flexible framework for connecting audiovisual collections to
related archival collections.
PodcastRE Analytics: Using RSS to Study the Cultures and
Norms of Podcasting
Eric Hoyt, University of Wisconsin-Madison; J.J. Bersch, University of Wisconsin-Madison; Susan Noh, University of Wisconsin-Madison; Samuel Hansen, University of Michigan and University of Wisconsin-Madison; Jacob Mertens, University of Wisconsin-Madison; Jeremy Wade Morris, University of Wisconsin-Madison
Abstract
[en]
Over the past decade, podcasting has grown into one of the most important media forms in
the world. This article argues that podcasting’s unique technical affordances —
particularly RSS feeds and user-entered metadata — open up productive methods for
exploring the cultural practices and meanings of the medium. We share three different
methods for studying RSS feeds and podcast metadata: 1) visualizing how topics and
keywords trend over time; 2) visualizing networks of commonly associated keywords entered
by podcasters; and 3) analyzing norms and common practices for the duration of podcasts
(as a time-based media format, podcasting is unusual in that it is not bound by the
programming schedules and technical limitations that provide strict parameters for most
audiovisual forms). The methods and preliminary results reveal how metadata can function
as a surrogate for studying large collections of time-based media objects. Yet our study
also shows that, when it comes to born digital media, the metadata are never fully
separate from the objects they describe. We argue that future work in AV in DH needs to
delineate between methods best suited for digitized media collections compared to those
most appropriate for born digital media collections.
Transdisciplinary Analysis of a Corpus of French
Newsreels: The ANTRACT Project
Jean Carrive, Institut National de l'Audiovisuel; Abdelkrim Beloued, Institut National de l'Audiovisuel; Pascale Goetschel, Centre d'Histoire Sociale des Mondes Contemporains; Serge Heiden, ENS Lyon; Antoine Laurent, Laboratoire d'Informatique de l'Université du Mans; Pasquale Lisena, EURECOM; Franck Mazuet, Centre d'Histoire Sociale des Mondes Contemporains; Sylvain Meignier, Laboratoire d'Informatique de l'Université du Mans; Bénédicte Pincemin, ENS Lyon; Géraldine Poels, Institut National de l'Audiovisuel; Raphaël Troncy, EURECOM
Abstract
[en]
The ANTRACT project is a cross-disciplinary apparatus dedicated to the analysis of the
French newsreel company Les Actualités Françaises (1945-1969)
and its film productions. Founded during the liberation of France, this state-owned
company filmed more than 20,000 news reports shown in French cinemas and throughout the
world over its 24 years of activity. The project brings together research organizations
with a dual historical and technological perspective. ANTRACT's goal is to study the
production process, the film content, the way historical events are represented and the
audience reception of Les Actualités Françaises newsreels
using innovative AI-based data processing tools developed by partners specialized in
image, audio, and text analysis. This article focuses on the data processing apparatus and
tools of the project. Automatic content analysis is used to select data, to segment video
units and typescript images, and to align them with their archival description. Automatic
speech recognition provides a textual representation and natural language processing can
extract named entities from the voice-over recording; automatic visual analysis is applied
to detect and recognize faces of well-known characters in videos. These multifaceted data
can then be queried and explored with the TXM text-mining platform. The results of these
automatic analysis processes are feeding the Okapi platform, a client-server software that
integrates documentation, information retrieval, and hypermedia capabilities within a
single environment based on the Semantic Web standards. The complete corpus of Les Actualités Françaises, enriched with data and metadata, will
be made available to the scientific community by the end of the project.
Topological properties of music collaboration networks:
The case of Jazz and Hip Hop
Lukas Gienapp, Leipzig University; Clara Kruckenberg, Leipzig University; Manuel Burghardt, Leipzig University
Abstract
[en]
Studying collaboration in music is a prominent area of research in fields such as
cultural studies, history, and musicology. For scholars interested in studying
collaboration, network analysis has proven to be a viable methodological approach. Yet, a
challenge is that heterogeneous data makes it difficult to study collaboration networks
across music genres, which means that there are almost only studies on individual genres.
To solve this problem, we propose a generalizable approach to studying the topological
properties of music collaboration networks within and between genres that relies on data
from the freely available Discogs database. To illustrate the approach, we provide a
comparison of the genres Jazz and Hip Hop.
Section 3: Creative and Liberatory Ways to Remix AV Data
Afrofuturist Intellectual Mixtapes: A Classroom Case
Study
Tyechia L. Thompson, Virginia Tech; Dashiel Carrera, Virginia Tech
Abstract
[en]
This article is a classroom case study of the Intellectual Mixtape Project, an
AudioVisual digital humanities module. The intellectual mixtape uses jazz and hip hop as
a framework to create an audio compilation and “conversation” that samples
literary-audio texts (such as SunRa speeches, Octavia Butler interviews, Tracy K. Smith’s
poetry readings, etc.). Each track of the intellectual mixtape has three audios: 1) the
literary-audio texts from the syllabus, 2) the students’ voice in their own words, and 3)
an audio of the students’ choice. As a companion to each track, students write 500 words
of liner notes that must include the title of their track and their curation and mixing
decisions.Students then publish their entire intellectual mixtape (three or more tracks)
with original or “remixed” cover art on an online platform.
The first part of the study will discuss the structure of the intellectual
mixtape assignments. In these assignments, students are provided with literary-audio
texts, required to complete and submit audio homework assignments, and taught the basics
of audio editing. This method of teaching and analyzing literature shifts the practice
of literary analysis from top down approaches that privileges the authority of the text
and instead encourages the student to “converse” with the text to
create new knowledge. This methods also reflects the artistic practice of Afrofuturist
artists and theorist who improvise, remix, and sample to create their work.
The second part of the study will discuss a performance and midterm adaptation of
the Intellectual Mixtape Project entitled Sound of Space: An Interactive
Afrofuturist Experience. “Sound of Space” was an immersive
performance with four-sensory stations that featured Afrofuturist themes. The midterm
adaptation was showcased in the Cube, a four-story high, state-of-the-art multimedia black
box theater at Virginia Tech. In preparation for the performance, students merged sound
engineering, 360 degree-video-projection, improvisational performance, and light design.
“Sound of Space” introduced students and audiences’ to an
immersive Afrofuturist-audio experience and pushed the boundaries of literary
analysis.
The third part of the study will address challenges with the Intellectual Mixtape
Project. Challenges include finding relevant literary-audio texts and dealing with the
many limitations imposed by U.S. copyright law. Some ways to address the
challenges imposed by U.S. copyright law might be to 1) reclassify sampling audio as a
form of quotation, 2) use databases of copyright-free music, 3) find culturally
significant works from lesser-known artists who will license their tracks, and/or 4) pay
royalties.
Annotating our Environs with the Sound and Sight of
Numbers: The DataScapes Project
John Bonnett, Brock University; Joe Bolton, Business Insight 3; William Ralph, Brock University; Amy Legault, Billyard Insurance Group; Erin MacAfee, University of Ottawa; Michael Winter, Brock University; Chris Jaques, Badal.io; Mark Anderson, Independent Consultant
Abstract
[en]
The DataScapes Project is an exploration of how Augmented Reality objects
can be used as constituents for Landscape Architecture. Using Stephen Ramsay’s
Screwmeneutics and Harold Innis' Oral Tradition as our theoretical points of departure,
the project integrated the products of Data Art – the visualisation and sonification of
data – as the constituents for our two works: The Five Senses and Emergence.
The Five Senses was the product of protein data, while
Emergence was generated using text from the King James version of the Holy
Bible. In this exploratory treatment, we present the methods used to generate and display
our two pieces. We further present anecdotal, qualitative evidence of viewer feedback, and
use that as a basis to consider the ethics, challenges and opportunities that a future AR
Landscape Architecture will present for scholars in the Digital Humanities.
What Does A Photograph Sound Like? Digital Image
Sonification As Synesthetic AudioVisual Digital Humanities
Michael J. Kramer, SUNY Brockport
Abstract
[en]
Computers have the capacity to transpose the pixels, shapes, and other features of visual
material into sound. This act of data correlation between the visual and the audial
produces a new artifact, a sonic composition created from the visual source. The new
artifact, however, correlates precisely to data in the original, thus allowing for fresh
ways of perceiving its form, content, and context. Seeming to distort the visual object
into an aural one paradoxically allows an observer to observe the visual evidence anew,
with more accuracy. A kind of generative, synesthetic criticism becomes possible by
cutting across typical boundaries between the visual and the audio, the optic and the
aural. Listening to as well as looking at visual artifacts by way of digital
transpositions of data enables better close readings, more compelling interpretations, and
deeper contextual understandings. Building on my earlier scholarship into image glitching,
remixing, and sonification, this essay investigates a photograph of Joan Baez performing
at the Greek Amphitheater in Berkeley, California, during the early 1960s. The image comes
from my project on the Berkeley Folk Music Festival and the history of the folk music
revival on the West Coast of the United States. Here, the use of digital image
sonification becomes particularly intriguing. While we cannot magically recover the music
being made in the photograph, we can more closely attend to the ghosts of sound within the
silent snapshot. Digital image sonification does not recover the music itself, but it does
help to amplify issues of gender, power, embodiment, spectacle, performance, hierarchy,
and performance in my perceptions of Baez making music in the photograph. Using the ear as
well as the eye to scan the image for its multiple levels of meaning leads to unsuspected
perceptions, which then support more revealing analysis. In digital image sonification, a
cyborgian dance of data, signal, image, sound, history, and human perception emerges,
activating visual materials for renewed scrutiny. In doing so, this mode of AudioVisual DH
activates the scholarly imagination in promising new ways.
From close listening to distant listening: Developing
tools for Speech-Music discrimination of Danish music radio
Iben Have, Aarhus University; Kenneth Enevoldsen, Aarhus University
Abstract
[en]
Digitization has changed flow music radio. Competition from music streaming services like
Spotify and iTunes has to a large extend outperformed traditional playlist radio, and the
global dissemination of software generated playlists in public service radio stations in
the 1990s has superseded the passionate music radio host. But digitization has also
changed the way we can do research in radio. In Denmark digitization of almost all radio
programming back to 1989, have made it possible to actually listen to the archive to
investigate how radio content has changed historically. This article investigates the
research question: How has the distribution of music and talk on the Danish
Broadcasting Corporation’s radio channel P3 developed 1989-2019 by comparing a
qualitative case study with a new large-scale study. Methodologically this shift from a
close listening to a few programs to large-scale distant listening to more than 65,000
hours of radio enables us to discuss and critically compare the methods, results,
strengths and shortcomings of the two analysis. Previous studies have demonstrated that
Convolutional Neural Networks (CNNs) trained for image recognition of spectograms of the
audio outperforms alternative approaches, such as Support Vector Machines (SVMs). The
large-scale study presented shows that the CNN-based approach generalizes well, even
without fine-tuning, to speech and music classification in Danish radio, with an overall
accuracy of 98%.
Hearing Change in the Chocolate City: Computational
Methods for Listening to Gentrification
Alison Martin, Dartmouth College
Abstract
[en]
In this article, I outline a method of combining ethnography and computational soundscape
analysis in order to listen to processes of gentrification in Washington, DC. I utilize
Kaleidoscope Pro, a software suite built to visualize and cluster bioacoustic recordings
(typically birds and bats) to cluster points of tension in the soundscape of Shaw, a
rapidly gentrifying neighborhood in DC. Clustering and visualizing these sounds (which
include car horns, sirens, public transportation, and music) makes audible the sonic
markers of gentrification in Shaw. Furthermore, listening to gentrification is a call to
engage with the sonic right to the city, histories of legislating sound, and sonorities of
memory and nostalgia. This work contributes to the burgeoning black digital humanities
canon by thinking through how computational methods can help us to hear black life.
Although the digital humanities have turned to embrace the sonic in recent years, there is
still much to be done in considering how to embrace the aural in DH work. This project
invites us to listen closely to a changing neighborhood, and emphasizes sound as a valid
mode of knowledge production, questioning how a sonic rendering of gentrifying space
through the digital might move us toward more equitable soundscapes.
Section 4: Reconfiguring Computational Methods for AV
Advances in Digital Music Iconography: Benchmarking the
detection of musical instruments in unrestricted, non-photorealistic images from the
artistic domain
Matthia Sabatelli, Montefiore Institute; Nikolay Banar, University of Antwerp; Marie Cocriamont, Royal Museums of Art and History, Brussels; Eva Coudyzer, Royal Institute for Cultural Heritage; Karine Lasaracina, Royal Museums of Fine Arts of Belgium, Brussels; Walter Daelemans, University of Antwerp; Pierre Geurts, University of Liège; Mike Kestemont, University of Antwerp
Abstract
[en]
In this paper, we present MINERVA, the first benchmark dataset for the detection of
musical instruments in non-photorealistic, unrestricted image collections from the realm
of the visual arts. This effort is situated against the scholarly background of music
iconography, an interdisciplinary field at the intersection of musicology and art history.
We benchmark a number of state-of-the-art systems for image classification and object
detection. Our results demonstrate the feasibility of the task but also highlight the
significant challenges which this artistic material poses to computer vision. We evaluate
the system to an out-of-sample collection and offer an interpretive discussion of the
false positives detected. The error analysis yields a number of unexpected insights into
the contextual cues that trigger the detector. The iconography surrounding children and
musical instruments, for instance, shares some core properties, such as an intimacy in
body language.
Music Theory, the Missing Link Between Music-Related Big
Data and Artificial Intelligence
Jeffrey A. T. Lupker, The University of Western Ontario; William J. Turkel, The University of Western Ontario
Abstract
[en]
This paper examines musical artificial intelligence (AI) algorithms that can not only
learn from big data, but learn in ways that would be familiar to a musician or music
theorist. This paper aims to find more effective links between music-related big data and
artificial intelligence algorithms by incorporating principles with a strong grounding in
music theory. We show that it is possible to increase the accuracy of two common
algorithms (mode prediction and key prediction) by using music-theory based techniques
during the data preparation process. We offer methods to alter often-used Krumhansl
Kessler profiles , and the manner in which they are employed
during preprocessing, to aid the connection of musical big data and mode or key predicting
algorithms.
Comparative K-Pop Choreography Analysis through
Deep-Learning Pose Estimation across a Large Video Corpus
Peter Broadwell, Stanford University; Timothy R. Tangherlini, University of Calfornia, Berkeley
Abstract
[en]
The recent advent of deep learning-based pose detection methods that can reliably detect
human body/limb positions from video frames, together with the online availability of
massive digital video corpora, gives digital humanities researchers the ability to conduct
"distant viewing" analyses of movement and particularly full-body choreography at much
larger scales than previously feasible. These developments make possible innovative,
revelatory digital cultural analytics work across many sources, from historical footage to
contemporary images. They are also ideally suited to provide novel insight to the study of
K-pop choreography. As a specifically non-textual modality, K-pop dance performances,
particularly those of corporate and government-sponsored "idol" groups, are a key
component of K-pop’s core mission of projecting "soft power" into the international
sphere. A related consequence of this strategy is the ready availability in online video
repositories of many K-pop music videos, starting from the milieu's origins in the 1990s,
including an ever-growing collection of official "dance practice" videos and
fan-contributed dance cover videos and supercuts from live performances. These latter
videos are a direct consequence of the online propagation of the "Korean wave" by
generations of tech-savvy fans on social media platforms.
In this paper, we describe the considerations and choices made in the process of applying
deep learning-based posed detection to a large corpus of K-pop music videos, and present
the analytical methods we developed while focusing on a smaller subset of dance practice
videos. A guiding principle for these efforts was to adopt techniques for characterizing,
categorizing and comparing poses within and between videos, and for analyzing various
qualities of motion as time-series data, that would be applicable to many kinds of
movement choreography, rather than specific to K-pop dance. We conclude with case studies
demonstrating how our methods contribute to the development of a typography of K-pop poses
and sequences of poses ("moves") that can facilitate a data-driven study of the
constitutive interdependence of K-pop and other cultural genres. We also show how this
work advances methods for "distant" analyses of dance performances and larger corpora,
considering such criteria as repetitiveness and degree of synchronization, as well as more
idiosyncratic measures such as the "tightness" of a group performance.
Moving Cinematic History: Filmic Analysis through
Performative Research
Jenny Oyallon-Koloski, University of Illinois at Urbana–Champaign; Dora Valkanova, University of Illinois at Urbana–Champaign; Michael J. Junokas, University of Illinois at Urbana–Champaign; Kayt MacMaster, University of Illinois at Urbana–Champaign; Sarah Marks Mininsohn, University of Illinois at Urbana–Champaign
Abstract
[en]
In this paper, we argue for the value of motion capture-driven research that moves
audiovisual analysis in a performative direction to integrate the dancer/researcher into
the cinematic space. Like the work of videographic practitioners who communicate their
findings through the audiovisual medium, rather than the written medium, this work seeks
to engage with what Catherine Grant and Brad Haseman have called performative research by
applying a practice-led approach to moving image analysis. Through a physical and virtual
embodiment of film and dance form, we seek to better understand the formal implications of
dance’s integration into cinematic space and the material conditions that affected
filmmakers’ narrative and stylistic choices. The Movement Visualization Tool (mv tool) is
a virtual research environment that generates live feedback of multiple agents’ movement.
Accessible motion capture technology renders an abstracted skeleton of the moving agents,
providing information about movement pathways through space using color-based and
historical traceform filters. The tool can also replicate a mobile frame aesthetic,
allowing for a constructed mover and a virtually constructed camera to engage in
performative dialogue. We use the mv tool and videographic methods to recreate and
disseminate two cases: movement scales from Laban/Bartenieff Movement Studies and dance
sequences from narrative cinema. Rather than working from existing audiovisual content, we
posit that the act of recreating the movement phrases leads to a deeper understanding of
the choreography and, in the case of the filmic examples, of the formal practices that led
to their creation.
Towards a User-Friendly Tool for Automated Sign
Annotation: Identification and Annotation of Time Slots, Number of Hands, and Handshape
Manolis Fragkiadakis, Leiden University; Victoria Nyst, Leiden University; Peter van der Putten, Leiden University
Abstract
[en]
The annotation process of sign language corpora in terms of glosses, is a highly
labor-intensive task, but a condition for a reliable quantitative analysis. During the
annotation process the researcher typically defines the precise time slot in which a sign
occurs and then enters the appropriate gloss for the sign. The aim of this project is to
develop a set of tools to assist the annotation of the signs and their formal features in
a video irrespectively of its content and quality. Recent advances in the field of deep
learning have led to the development of accurate and fast pose estimation frameworks. In
this study, such a framework (namely OpenPose) has been used to develop three different
methods and tools to facilitate the annotation process. The first tool estimates the span
of a sign sequence and creates empty slots in an annotation file. The second tool detects
whether a sign is one- or two-handed. The last tool recognizes the different handshapes
presented in a video sample. All tools can be easily re-trained to fit the needs of the
researcher.
Section 5: Forms of AV Scholarship
Books Aren't Dead: Resurrecting Audio Technology and
Feminist Digital Humanities Approaches to Publication and Authorship
Emily Edwards, Bowling Green State University; Robin Hershkowitz, Bowling Green State University
Abstract
[en]
This article explores how the podcast medium as a form of audio technology has
facilitated the reimagining of academic publication and feminist praxis. In this case
study, we situate the podcast Books Aren’t Dead (BAD), an
affiliate of the Fembot Collective, within a broader context of digital humanities
scholarship and the field's potential to utilize audio technology to realize feminist
approaches. BAD, as a podcast, serves as an open-access
medium that brings authors and reviewers together in a collaborative context. Audiobook
reviews allow for a conversation between author and interviewer, whereby the author can
place the work in a broader scholastic and contemporary context for listeners as well as
actively engage with constructive critique and questions. The result is a dynamic
scholarly communication rather than the static textual product of a book review. We
discuss the unique role of audio technology within the knowledge production process from a
performance studies and archival point of view. Additionally, in the spirit of the project
BAD, we also provide an addendum to our textual discussion
by including a podcast where we discuss these themes as co-producers, graduate students,
and young academics, exploring how audio technology can break down barriers to publication
and authorship.
Another Type of Human Narrative: Visualizing Movement
Histories Through Motion Capture Data and Virtual Reality
Eugenia S. Kim, The Hong Kong Academy for Performing Arts
Abstract
[en]
In this article I propose that motion capture (mocap) and virtual reality (VR) technology
can be used to record and visualize movement histories as a supplement to oral histories
or for when a memory is based in a embodied experience. One specific example would be the
presentation of illness narratives. To illustrate this situation, I examine the concept of
illness narratives, particularly those created by dance artists, and use my movement
history, Lithium Hindsight 360, as a case study. This analysis comes from the
perspective of a hybrid movement artist, VR creator, archivist and digital humanist, with
first-hand experience of the challenges encountered when creating a movement history. The
challenges are presented within the context of mocap recording, data curation, digital
preservation and sustainability issues. I end this article by providing some basic
practical strategies and recommendations for researchers who are new to documenting
movement histories.
Deformin' in the Rain: How (and Why) to Break a Classic
Film
Jason Mittell, Middlebury College
Abstract
[en]
Digital source materials such as films can be transformed in ways that suggest an
innovative path for digital humanities research: computationally manipulating sounds and
images to create new audiovisual artifacts whose insights might be revealed through their
aesthetic power and transformative strangeness. Following upon the strain of digital
humanities practice that Mark Sample terms the “deformed
humanities,” this essay subjects a single film to a series of deformations: the
classic musical Singin' in the Rain. Accompanying more than
twenty original audiovisual deformations in still image, GIF, and video formats, the essay
considers both what each new version reveals about the film (and cinema more broadly) and
how we might engage with the emergent derivative aesthetic object created by algorithmic
practice as a product of the deformed humanities.
Reviews
Book Review: Digital Sound Studies (2018)
Tracey El Hajj, University of Victoria
Abstract
[en]
The edited volume Digital Sound Studies brings together
various voices addressing the potential of digital approaches to sound, practically and
theoretically . Contributors explore methodologies, platforms,
and initiatives that demonstrate interdisciplinary and inclusive work that centers sound
and listening while demonstrating how such work can advance humanities scholarship. The
contributions provide a balanced critique of DH as a norm and culture alongside detailing
digital sound studies' contributions to DH, the humanities, and the public. The volume is
an excellent resource for those interested in digital sound studies.
Articles
From the Presupposition of Doom to the Manifestation
of Code: Using Emulated Citation in the Study of Games and Cultural Software
Eric Kaltman, Department of Computer Science, California State University Channel Islands; Joseph Osborn, Department of Computer Science, Pomona College; Noah Wardrip-Fruin, Department of Computational Media, University of California, Santa Cruz
Abstract
[en]
For the field of game history to mature, and for game studies more broadly to
function in a scholarly manner in the coming decades, one necessity will be
improvement of game citation practices. Current practices have some obvious problems,
such as a lack of standardization even within the same journal or book series. But a
more pressing problem is disguised by the field’s youth: Common citation practices
depend on the play experiences and cultural knowledge of a generation of game studies
scholars and readers who are largely old enough to have lived through the eras they
are discussing. More sustainable and precise alternatives cannot fall back on the
tools available for fixed media — such as the direct quotations and page numbers used
for books or the screenshots (of images that appear to all viewers) and timecode used
for video. Instead, this essay imagines an alternative approach, working in the
digital humanities traditions of speculative collections and tool-based
argumentation. In the speculative future we present, there are scholarly collections
of software, as well as tools available for citing software states and integrating
these citations into scholarly arguments. A working prototype of such a tool is
presented, together with examples of scholarly use and the results of an evaluation
of the concept with game scholars.
Fostering Community Engagement through Datathon
Events: The Archives Unleashed Experience
Samantha Fritz, Department of History, University of Waterloo; Ian Milligan, Department of History, University of Waterloo; Nick Ruest, Digital Scholarship Infrastructure Department, York Univeristy; Jimmy Lin, David R. Cheriton School of Computer Science, University of Waterloo
Abstract
[en]
This article explores the impact that a series of Archives Unleashed datathon events
have had on community engagement both within the web archiving field, and more
specifically, on the professional practices of attendees. We present results from
surveyed datathon participants, in addition to related evidence from our events, to
discuss how our participants saw the datathons as dramatically impacting both their
professional practices as well as the broader web archiving community. Drawing on and
adapting two leading community engagement models, we combine them to introduce a new
understanding of how to build and engage users in an open-source digital humanities
project. Our model illustrates both the activities undertaken by our project as well
as the related impact they have on the field. The model can be broadly applied to
other digital humanities projects seeking to engage their communities.
Leonardo, Morelli, and the Computational
Mirror
Alison Langmead, University of Pittsburgh; Christopher J. Nygren, University of Pittsburgh; Paul Rodriguez, San Diego Supercomputer Center; Alan Craig, Indepenent Consultant and Researcher
Abstract
[en]
By bringing forward and interpreting the results from a collaborative research project that used contemporary computing techniques to investigate Giovanni Morelli’s nineteenth-century method for making stylistic attributions of old master paintings, this article examines how humanists make claims to knowledge and how this process may or may not be modellable or mechanizable within the context of classical, deterministic, digital computation. It begins with an explanation of the rationale behind choosing the Morellian practice of attribution, continues with a survey of another effort at computationally implementing Morelli’s method, and then presents our own computational techniques and results. The article concludes with what we have come to understand about the roles of responsibility, trust, and expertise in the social practice of art attribution, and the dangers in assuming that such human entailments are native to digital computers.
Reviews
Networks, Maps, and Time: Visualizing Historical
Networks Using Palladio
Melanie Conroy, The University of Memphis
Abstract
[en]
Many tools can produce maps, graphs, and charts that may differ in seemingly minor
ways. Data visualization tools are one type of “middleware” that can be all but forgotten when one is presented with final
products such as papers and presentations containing visualizations . Since the output of various software packages is
sometimes similar, it is easy to forget the assumptions that went into the diagram,
the dataset, and the software when looking at the final product — or even while using
the tool if one becomes sufficiently accustomed to the interface. In this review, I
revisit the visualization suite Palladio – which Miriam Posner has called a “Swiss Army knife for humanities data”
– and the many projects that have made use of Palladio’s
core features in the years since its launch. I examine the strengths and limitations
of Palladio, as a network and map-making tool for exploring data and for rapidly
prototyping diagrams, designed with an iterative process in mind. I contrast this
iterative mentality with the analytic sensibility of tools like Gephi and Cytoscape,
and review the primary features of Palladio with one primary case study (my own
visualizations of the French Enlightenment network) and examples of how the features
have been used in other digital humanities projects. Palladio is very useful for
qualitative studies of data that include geospatial and chronological dimensions,
especially when the data are tagged with different types of qualitative metadata, but
it also tends to impose a historical geographical view on the data by foregrounding
geospatial relationships, time, and other historical considerations.
A Review of Twitter and Tear
Gas
Nanditha Narayanamoorthy, York University
Abstract
[en]
Zeynep Tufekci’s book Twitter and Tear Gas (Yale University Press; 2017) speaks to
high-profile, anti-authoritarian networked protests. She engages with street protests
and online movements to bring new perspectives and dialogues on the need for
reconfiguration of digitally networked online spaces, and the trajectories of these
social movements online. Her work contributes to scholarship in digital activism, and
digital humanities in the context of networked movements.
A Review of Intergenerational
Connections in Digital Families
Sucharita Sarkar, D.T.S.S. College of Commerce, Mumbai, India
Abstract
[en]
This review synthesizes Sakari Taipale’s book Intergenerational
Connections in Digital Families (Springer International Publishing, 2019),
partially from an auto-ethnographic perspective. Borrowing from the book’s structure,
the review is divided into three parts. The first section examines the definitions of
digital families and the role of everyday communication technologies in connecting
such families. The second section critiques the interconnected roles of family
members and generations in maintaining digital connections, especially through
Taipale’s revival of the notion of the “warm expert.” The final section assesses the
book’s conclusions in the context of changing social policies. It also looks at the
possibilities for future research in the domain of digital family studies (and, by
extension, in digital humanities) that can germinate from Taipale’s concise
study.
Author Biographies
URL: http://www.digitalhumanities.org/dhq/vol/15/1/index.html
Last updated:
Comments: dhqinfo@digitalhumanities.org
Published by: The Alliance of Digital Humanities Organizations and The Association for Computers and the Humanities
Affiliated with: Digital Scholarship in the Humanities
DHQ has been made possible in part by the National Endowment for the Humanities.
Copyright 2005 -
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License
Last updated:
Comments: dhqinfo@digitalhumanities.org
Published by: The Alliance of Digital Humanities Organizations and The Association for Computers and the Humanities
Affiliated with: Digital Scholarship in the Humanities
DHQ has been made possible in part by the National Endowment for the Humanities.
Copyright 2005 -
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License
