Abstract
How can digital traces of the performing arts be interpreted? What methodologies can
be proposed to “make them talk”? The specificity of these traces
leads to specific methodological responses, especially as the aim here is to
interpret the traces in both directions of understanding and replayability. It is as
much a question of advancing a reflection on the conditions for the recovery of work,
as it is of analyzing an artistic approach based on its digital traces. Starting from
these epistemological issues, I present the development of two softwares, Rekall and
MemoRekall. These tools are part of a larger trend which I call “theatre analytics”. Theatre analytics is based on data from the traces of
the performing arts. Unlike the general approach of “big data in the social
sciences and humanities,” where we seek to determine general, average, profile
categories or repeatable (or even predictable) patterns, our primary concern remains
to capture the singular, the detail, the difference, the anomaly, in a constant round
trip between the micro and the macro dimensions. The ambition of theatre analytics is
to offer a different way of looking at big data, more oriented towards complexity
than the quantity of data.
Digital technology has flooded the theatre. On stage, it is omnipresent: image
projections, real-time transformations of the actors' voices, interactions between the
performers and the stage enhanced by all kinds of devices. Behind the scenes, there is
no longer a single sound or light control room that is not digital, with the exception
of shows in the dark, in daylight or flickering candlelight or even the rare
performances without a sound broadcasting system. However, it is not just about the
shows. Creative processes are also impacted, as are other aspects of theatrical
activity: distribution of works, criticism, public reception, communication, financing.
If one wants to study the performing arts today, it is impossible not to take into
account companies' websites, blog and online newspaper articles, recordings broadcast on
Youtube or Vimeo, social networks, email exchanges, budget tables, ticketing software.
Today, directors' notebooks also take the form of Word files, image-based blogs on
Tumblr or Instagram, posts on Facebook, Evernote notebooks, Pinterest tables, task lists
in Trello, exchanges in Slack. The traces of the performing arts have become digital
traces. This phenomenon has been driven by two major trends: the emergence of the
born-digital heritage and theatre archives digitization.
The digital traces of the performing arts irremediably change not only the nature of the
sources on which we rely to analyze works and write the history of performing arts but
also how these studies are conducted and their results. The transformation of the trace
into data-based information, making data a new type of source, is fraught with
consequences for the memory of the ephemeral theatre, consequences that are not yet
fully measured by the various actors, be they cultural institutions, librarians or
researchers. Historiography in a digital context is not a revival of quantitative
history in the strict sense of the term. It calls first and foremost for practices, and
it is also in this practice that the narrative is housed. It is because it is a question
of practice, and not just of exposing traces, but instead of exposing them to
manipulation that a paradigm shift is taking place, inviting us to “make history
differently.” Because the traces are transformed, they invite a renewal of the
critique of sources that we thought we knew and make research questions evolve and lead
to other epistemologies. The “digital turn” proposes a paradigm shift in how we
build the memory of performing arts by renewing source criticism, methodological
thinking, and narratives, moving beyond traditional methods of close reading to
computational assisted reading and contributing to the general reflection on digital
history.
How can performing arts digital traces be interpreted? What methodologies can be
advanced to “make them talk?” The specificity of these traces leads to specific
methodological responses, especially since the aim here is to interpret the traces in
both directions of understanding and replayability. This is as much a question of
developing a reflection on the conditions for the recovery of creative work as it is of
analyzing an artistic approach based on its digital traces. Traditional methods, in
particular close reading, i.e., a reading, a precise analysis of each of the traces,
have to be combined with new approaches.
This growing awareness lead me to the creation of a software prototype, Rekall, and a
web application, MemoRekall, which I am still developing today. Both environments are
free and open source. Rekall and MemoRekall provide new opportunities of interpreting
digital documents for performing arts research. They offer different and unexpected
points of view on a dataset, either from a distant or a close reading approach, either
by extracting metadata or by linking documents. Furthermore, the softwares take into
account the open nature of the digital archive, which reconfigures itself as new
documents are added, new links are created, and allow collaborative research.
From an epistemological point of view, the creation of these tools is anchored in the
emergence of a new field of research that can be called “theatre analytics.”
Theatre analytics are based on data from the traces of the performing arts. In line with
“culture analytics,” the term refers to the calculation and emphasizes the
analysis of works, which is the foundation for the preservation and historiography of
the performing arts. In both cases it is a matter of following in detail a process to
show why and how a work is constituted by observing its various components and of
borrowing from mathematics to study, preserve and write a history of the performing
arts. Theatre analytics does not challenge existing methodologies but allows them to be
supplemented by taking into account what is generally discarded, namely, digital traces,
considered too complex, too numerous, too fragile, with dubious authenticity. The aim is
not to promote an approach that would be purely quantitative or statistical but on the
contrary to see how qualitative analyses can also be conducted in a digital context.
Unlike the general approach of “big data in the social sciences and humanities,”
where we seek to determine general, average, profile categories or repeatable (or even
predictable) patterns, our primary concern remains to seize the singular, the detail,
the difference, the anomaly in a constant round trip between the micro and the macro
dimensions, and looking for mesoscales. In the tracking of the singular at the core of
big data, the deviation from the mean is made visible as such and no longer as an error
against a standard. Nor is it a question of considering that singularity would be
calculable, as some research on emotions or creativity seems to state by reducing
digital traces to mathematical artifacts. On the contrary, we argue for an approach
where the singular, the ideosyncraric
via computational methods,
be the focus of a qualitative interpretation. Data visualisation is ultimately
qualitative and must give rise to an accurate and informed reading. As Johanna Drucker,
who calls for a “visual epistemology,” points out “Visualizations are always interpretations — data does not have an
inherent visual form that merely gives rise to a graphic expression”
[
Drucker 2014, 7]. In this light, the ambition of theatre analytics
is to offer a different way of looking at big data that is more oriented towards
complexity, rather than the quantity of data, with a diversity of computer approaches
depending on the nature of the data and the research questions asked. Alongside other
methodologies less specific to the performing arts but which can also beRekall and
MemoRekall afforts such an approach. Part of the text is missing : Alongside other
methodologies less specific to the performing arts but which can also be convened (NLP,
network analysis, machine learning, algorithm creation, ...) Rekall and MemoRekall
afforts such an approach.
This paper will explicate the development context of Rekall and MemoRekall and provide
lessons learnt from this experiment. The detailed technical functions of the two
softwares can be found elsewhere [
Bardiot et al. 2014]
[
Bardiot 2015]
[
Bardiot 2017]. The question can be asked how a performing arts researcher
comes to develop new software applications? To explain this, first, I will set out the
specificity of the problems generated by the digital traces of the performing arts.
Second, main objectives of the project and the choices that result from it are
explained. Third, the different stages of development and in particular the
collaborations with artistic teams, programmers and designers are laid out.
The digital traces of the performing arts
Compared to non-digital traces, what do digital traces change for the preservation,
analysis of works and the performing arts history? Let us take the tip of the
“iceberg” of technological obsolescence and big data. A creative process,
from its initial ideas to the première, lasts an average of four years -
the same length as that of a technological cycle. In other words, at the moment of
the première, it is necessary to update the management software and
computer systems accompanying the tour, to save and sort the hundreds, sometimes
thousands, of documents related to the work in a new version. As soon as the work is
created, the artistic and technical team is confronted with problems of technological
obsolescence and a quantity of information that may have a decisive impact on the
production and its distribution of the work and can even, in the medium term, make it
impossible to tour.
The preservation of performances is not the only phenomenon concerned by these
issues. The creative processes are also affected. To take the example of the
director's notebooks, in 20 years, who will remember applications such as Trello,
Slack, Evernote or Tumblr? Will we be able to read the digital traces that have been
stored and have access to the data they contain while guaranteeing their
authenticity? Not only are performances ephemeral, which is inherent to the
performing arts, but their digital traces seem even more evanescent. They are also
increasingly numerous, overwhelming us with a continuous flow of data, in an
unprecedented disproportion. These questions are not all new, but their amplification
gives them a new acuity. How can we write the history of the performing arts from not
only fragile and variable traces, but also too numerous at for researcher or
archivist?
It is undeniable that, today, digital technology involves radical, and even
irreversible, changes in our relationship to memory, history, and what is left as a
legacy. Given this, one can but note that memory and the digital seem to oppose each
other. This need not be so. Reconciling them is essential. Digital traces cannot be
ignored in building and safeguarding the memory of the performing arts. These issues
are complex and urgent. If nothing is done, the technological obsolescence of digital
technology will overtake all other considerations, and an entire part of our culture
will vanish.
To take a concrete example, under the title Re: Walden,
are regrouped different avatars of the same creative work, developed by the French
director Jean-François Peyret from 2006 to 2014. This involved two installations
(2010 and 2013 at Le Fresnoy, France), performances in the installation, four scenic
versions (including one for the Avignon 2013 festival, France), a concert at EMPAC
(USA) and an extension in the virtual environment Second
Life. Between 2012 and 2013, I collected several hard disks from the
artistic team. These were those of Thierry Coduys (electroacoustic and computer
devices), Agnès de Cayeux (virtual world) and Julie Valéro (dramaturgy). Although
this collection is mostly incomplete (ideally, access to all the digital data of each
member of the artistic team would have been required), the data collected were 10,360
files for 20.5 GigaBytes.
The files collected include the following elements: scores
[1],
photographs and video recordings of rehearsals and performances, sound recordings for
voice synthesis, sound files as well as computer programs (Max/MSP and Pure Data
patches), images, management documentation (contacts, schedules...), technical
sheets, room plans, communication documents (playbills, press releases, reviews
published in newspapers or blogs), archived e-mails, screenshots of
Second Life, texts for each of the actors or texts from the
automatic translator developed for the project, meeting minutes, notes of intent from
the artistic team.
This example is symptomatic of digital traces of the performing arts, even if it
concerns a complex creation that takes place over several years. They are
heterogeneous, numerous but incomplete and fragile. The digital traces of the
performing arts are heterogeneous, both in terms of the type of file (image, sound,
text, video, computer program) and their content. All is happily mixed without any
distinction and with the elements of the show such as aspects about the creative
process or publicity compaigns, and the tour. These three aspects make it possible to
define three main categories of digital traces that correspond to the main
chronological stages in the life of a performance: the creative process, the work,
audience reception. We can add a fourth one, not represented here: the administrative
digital traces of cultural institutions and companies, i.e., everything related to
the production of shows.
There are many digital traces, so much so that the term “big data” is used. We
have counted several thousand files for Re: Walden. The
situation is nothing new, and we can only observe the inflation of digital traces.
However, despite their impressive volume, the traces are far from being exhaustive.
Besides, some tasks left only minimal traces, for example there were only a few
photos and patches in the Pure Data software for the residency at the École Régionale
d'Acteurs de Cannes. Finally, digital traces do not take into account a large part of
the exchanges, actions, and reflections taking place on and off the set, if they had
not been systematically recorded or were in the form of reports and notes. Part of
the digital activity itself rarely leads to the creation of preserved traces, like
the one that takes place on electronic networks, such as documents shared on Google
drive, conversations on Skype or applications such as Slate or Trello, images made
available on Tumblr.
Digital traces are fragile. The duration of access and readability of a digital file
might be five years. So that not only are we faced with ephemeral works, but also
with traces increasingly fragile. For a large part of them, in a few years,
especially those from proprietary software (Word for example), we will probably no
longer be able to open them and therefore read them. The loss of traces seems to be
an unavoidable phenomenon. In fact, many artists have already lost not only traces
but also creative works.
In this overview, we understand that digital traces exacerbate both the properties of
“analogical” traces, while at the same time differentiating themselves from
them. As for the first observation, linked to heterogeneity, number or fragility, it
is ultimately “merely” a question of accentuating phenomena already well
identified in the analogical world. The great diversity of the nature of documents in
the performing arts collections; the inflation of archives is a recurring problem
identified, notably by Krzystof Pomian [
Pomian 1992], where paper ends
up being reduced to dust, inks are erased, magnetic strips become electronic snow.
Despite the appearance of continuity, in fact digital technology fundamentally
changes the ontology of the trace. To use Bruno Bachimont's expression, we have gone
from a “graphic reason”
[2]
[
Goody 1977] to a “computational reason”
[
Bachimont 2008]. As they become numerical, traces are coded and
presented as numbers, which allows for computational calculation. This makes the
question of reading traces and their authenticity complex (for they must be
“decoded”) while opening up to being manipulated. In other words, digital
traces are “analyzable”, in the triple sense of
discretization (the decomposition of a thing into its elements), computer calculation
and data visualization. Analytical geometry, attributed to Descartes that allows a
graphic representation of algebra, is the prelude to contemporary data
visualizations. The digital traces of the performing arts thus presents several
paradoxes: technological obsolescence versus full archiving, authenticity versus
modification, and repetition versus ephemerality.
At the crossroads
In 2006, as a performing arts researcher, I conducted a case study on Belgian
choreographer Michèle Noiret on technological obsolescence issues about the
performing arts at the margins, given that the works are inherently ephemeral. The
case study was part of the DOCAM research program (2005-2010) led by the Daniel
Langlois Foundation in Montreal on the documentation and preservation of media arts.
The programme includes numerous partners, including museums. It builds on the
pioneering work of the Langlois Foundation and John Ippolito on the variable media
approach [
Depocas et al. 2003]. Jon Ippolito will then publish with Richard
Rinehart
Re-Collection
[
Rinehart and Ippolito 2014]. In museums, the very concrete question of
technological obsolescence of works was raised. Pip Laurenson proposed new
conservation paradigms in questioning the central notions of state and authenticity
[
Laurenson 2006]. Faced with these questions, the researcher's
natural reflex is to identify existing solutions. Best practice guidelines and case
studies were published as part of international projects (Interpares, DOCAM, Caspar)
[
Bardiot 2012]; researchers worked on specific ontologies [
Rinehart 2007]; some artists - particularly choreographers - in
collaboration with scientists, proposed solutions for a work or set of works [
Bleeker 2017]. However, no existing solution seemed able to solve the
concrete problems identified in the performing arts field. They can be summarized as
follows:
- How can technical teams be enabled to take quick notes during rehearsals and to
aggregate all technical documents? The different control rooms are separate, each
one (lighting designer, general manager, video director, sound engineer) has its
documentation system in digital or paper format, more or less structured with each
participant defines his or her working method, his or her guide to “best
practices.”
- How can a show a few months or a few years later be restaged while respecting
as carefully as possible the initial artistic intentions? Due to built-in digital
obsolescence, technologies must be continuously adapted during the show tour. In
addition, when the creative team is not the same as the touring team, rigorous
documentation is required.
- How can we compensate for the memory loss caused by digital technology? It
becomes difficult to trace the history of the creative processes, especially since
the documents they generate are themselves ephemeral, if nothing else, in their
intended use.
- What status and place should be given to video recording? A widespread
practice, with varying results regarding the quality of video recording
implementation, it systematically calls for comment. Video recordings per se are not self-sufficient. Most often produced by the
company itself (for promotional purposes), they are nevertheless one of the most
significant traces of the art work.
- How can one move through the hundreds, sometimes thousands of documents to
reconstruct the creation process? How can data visualization contribute to the
analysis of a creative work's genetic analysis? By comparing different processes,
is it possible to identify forms, patterns?
- The digital traces of the works are very numerous and the reading of each trace
is almost impossible. The question becomes rather: what information, what
knowledge about the works is it possible to extract from all the traces for
preservation purposes? What data should be used to reconstruct a creative process?
In order to respond, at least partially, to these challenges, I published a first
paper [
Bardiot 2009] proposing a conceptual model of a computer
solution articulating documents around the video recording of a performance. In
contrast to the practice of creating a specific interface for each performance, I
designed a software that can be applied to as many creative works as possible. As my
concerns increasingly met those of cultural institutions confronted with the
obsolescence of technologies - and the costs that this entails in addition to the
difficulty of presenting works to the public - I decided to develop the software by
myself. At the end of 2012, I raised the first funds that enabled me to hire a team
of developers. Five years after the first draft, the situation had changed. The
issues of big data and digital traces mentioned above could no longer be ignored. It
was necessary to go beyond the articulation of a documentary corpus around a video
recording. In the preparatory meetings that followed, the guiding principles of the
project, first entitled
Éclats (in reference to the
fragmentation and multiplicity of documents) and then
Rekall (in reference to the memory company in K. Dick's
We Can Remember It for You Wholesale), were clarified. From
the outset, Open Source was a self-evident option. Indeed, some funders such as the
French Ministry of Culture imposed the Open Source option. Beyond that, Open Source
is a prerequisite for the sustainability of the solutions developed. It would have
been antinomic and counterproductive to proceed otherwise in a project dedicated to
the preservation of art works.
Rekall is an environment prototype to document and analyze creative processes, and to
simplify the preservation of performances. It uses the metadata present in all
documents of a performance in order to extract crucial data (author, date, place,
keyword) then used by data visualization tools to indicate behaviors. In this way,
Rekall provides an overview of the creation process and identifies the most important
documents (which will then have to undergo specific preservation measures). The
software can also be used during rehearsals (for example, to annotate documents,
review the history of a plot sheet), or once the creation is completed (to understand
its process, distinguish documents related to the work from those related to its
production). Rekall is cross platform. It is developed with the Qt development
framework. For each version of a document, Rekall extracts all available metadata
using the open source tool exiftool. The interface is in HTML5 and CSS3.
Two primary objectives guided Rekall's design: to help artists (the first curators of
their works) to document their creations in order to ensure their retrieval and
overcome the obsolescence of digital technologies; to help researchers study the
genetics of performances in a context of born-digital heritage and Big Data. These
two objectives may seem very distant at first sight. However, it is the same
materials (digital traces of the works) that are collected, analyzed and visualized
based on different modalities and time scales. In this way, the collection can take
place for the artists during the creative process itself, as the work stages
progress. A living archive emerges in perpetual movement. The researcher intervenes
once the process is completed, once the documents have somehow stabilized, frozen to
become the traces of what has happened. Rekall must, therefore, be able to be used
from the onset of the first ideas of a show, during the creative process and then
after its creation, with a dual perspective of preservation of the work and
historiography.
Very quickly, it became clear to us that we had to develop a multimodal environment
that included both digitized documents and natively digital documents. The digital
traces of the performing arts include texts, images, sounds, videos, computer
programs. A common practice of big data is to separate documents by file type and
work in the form of “silos” (e.g. text, image). Such processing prevents us from
obtaining an overview of the traces collected and from examining many phenomena such
as the evolution of an idea through different documents, from image to text, from
text to table, from table to video recording. Besides, it is essential to navigate
between the micro and macro dimensions, between close reading and distant reading,
between diachronic and synchronic representations. In other words, being able to
represent a process without erasing its complexity; to switch from data to file and
back again. Often, distant reading and data visualization involve a break between the
source file and the data from which it is extracted. Maintaining this link is
fundamental for several reasons. The data are always reductive fragments and it is
important to be able to recontextualize them in order to interpret them better. The
transition from a trace to data is a reductive and interpretive operation. Reduction
also means making a precipitate of traces, which makes it possible to amplify and
exhale a phenomenon that would otherwise have remained invisible. The transformation
affords new insights into the tracks and allows us to detect clues that would
otherwise have gone unnoticed. The use of data visualization is essential in the case
of performance preservation to capture the main steps of the creative process and to
identify the main files that will then need to be actively preserved, often on a
case-by-case basis. In a very concrete way, in a corpus of 10,000 documents, the
process of visualization must make it possible to identify the 50 documents that the
researcher must read in detail to analyze the work or that the artistic team must
migrate or emulate to continue touring.
Among the traces of the performing arts, the video recording is a particular type of
document. It has long been - and is still often - considered as a panacea regarding
documentation of the ephemeral performing arts. Provoking many debates since the
1960s, often considered as a “betrayal” of the original work [
Melzer 1995a]
[
Melzer 1995b], video recordings have now become commonplace. A video
recording is both an essential and a fragmented document. In its essential form, it
records bodies in movement and thus transforms performances into documents. In its
fragmented form, it gives only one point of view, presenting only a part of what
takes place on stage, generating temporal collusion through editing, adapting stage
lighting to the lighting conditions of the cameras. A commentary, that is to say, an
annotation must necessarily accompany these modifications. Two types of strategies
are developed in order to overcome the shortcomings or biases of video recording:
inter-documentary (connecting the video-capture to a larger documentary corpus) and
intra-documentary (annotating the video-capture). These two approaches can be
usefully combined. They simultaneously play different roles with the document,
between the deepening and connecting roles.
In a first phase, Rekall was developed not only as a document management tool in a
multimodal environment, but also as a video annotation software, with three
complementary strategies: the creation of links between multiple documents offering
the possibility of commenting and illuminating the documents between them; the choice
of video recording as a backbone from which all other documents would be organized;
and the possibility of adding textual comments. Subsequently, these two aspects were
separated. The video annotation part about of Rekall has now become MemoRekall.
MemoRekall offers a reading of a major document: video recordings. The software is a
user-friendly web application that explains a video by annotating it and linking it
to external documents or web pages. The arrangement of documents, links and
annotations creates a new document, called a “capsule”, that can
be embedded into a web page. The capsules organization is developed from and around
video recordings in a hypertextual logic. MemoRekall can be used to create a
multimedia educational booklet, run a digital spectator school, edit a scientific web
documentary or simplify the distribution of works. MemoRekall is developed in HTML5 /
CSS3 with a backend using PHP/MySQL technologies. The video player is developed in
HTML5, allowing it to be integrated into the main web browsers (Firefox, Safari,
Google Chrome). The personal user account was developed with the Symfony
framework.
Rekall and MemoRekall are complementary platforms aimed at preserving and analysing
the digital traces of the performing arts. They allow to organize and visualize in a
multimodal environment all the traces of a creative processe (Rekall) or only a
selection of the traces (MemoRekall), paying special attention to the video recording
of the works (MemoRekall). With Rekall, it’s possible to retrace the historic of a
file (but not the versioning, which can be done with complementary tools). In terms
of research, Rekall and MemoRekall are platforms suited to genetic analysis, close
and distant reading of a work, documentation and preservation of a performance.
Developping Rekall and MemoRekall : collaborate, test, adapt
In 2012, Guillaume Marais (ergonomics and interface design), Guillaume Jacquemin (IT
development) and Thierry Coduys (consultant, designer of electronic devices for the
performing arts) joined the project. They already had a first common experience of
software development with IanniX, a real-time sequencer for digital art inspired by
UPIC (a former project by Iannis Xenakis). Throughout his artistic career, which led
him to IRCAM, Thierry Coduys has developed electroacoustic and computer devices for
numerous directors, composers (including Luciano Berio, Ivan Fedele, and Pascal
Dusapin) and choreographers. He was regularly confronted with the question of
touring, restaging and transmitting works with particularly complex technological
caracteristics. Guillaume Jacquemin and Guillaume Marais created the Buzzing Light
agency in 2009. Guillaume Jacquemin is an embedded systems engineer and is trained
through multiple experiences in interactive multimedia and digital art. Guillaume
Marais is an ergonomist and interaction designer and is trained in information
systems engineering. In addition to their complementary skills in programming and
ergonomics, Guillaume Marais and Guillaume Jacquemin were used to contributing to
artistic projects and had a keen appetite for contemporary music. It was an essential
factor: to develop Rekall, it was crucial to establish a dialogue around the key
issues of the software. We used an AGILE approach and progressed incrementally based
on user feedback.
I am often asked why I have not chosen instead to work with a computer science
research laboratory. There are several answers to this question. One is that I needed
to be in a very operational, fluid and fast environment. Furthermore, it did not
appear that the project raised computer research issues. In 2012, in France, a
computer development project led by a theatre studies researcher was not always well
accepted by the academic community. Finally, on one hand, the funding obtained came
from funds for innovation in the field of culture (Pictanovo) and from cultural
institutions.
[3] This gave great flexibility in hiring service providers. On the
other hand, the lack of research funding and the administrative management of the
project by a theatre (the Phénix scène nationale Valenciennes) and then, in 2015, by
a publishing house (Subjectile) did not allow the hiring of a Ph.D. student or a
postdoc. Today, with the development of the digital humanities and the resulting
cultural change, the software development framework would probably be different. So
indeed, the contribution of Thierry Coduys, Guillaume Jacquemin and Guillaume Marais
were crucial. The results we obtained would have been very different with another
team that wouldn’t have had any experience with artistic creative processes. For
example, creating software that can be used by artists has several consequences. The
main point is that it must be as easy to use as possible and still meet the rigorous
requirements of researchers. This involved a reflection on interface design and
ergonomics and two essential decisions. The first is that Rekall had to operate
discreetly in the background on the assumption that artists have very little time to
archive their work even if they are aware of its importance. The artistic team works
on its usual tools while Rekall records in the background all the manipulations that
are done as well as the users' working environment (the version of their OS, the open
applications, and their version) in order to collect all fundamental information from
a preservation perspective. It is essential to keep the most exact trace possible of
the technological components and of the aesthetic and historical dimensions. At the
same time, it is important to describe the effects of these same components, in line
with the variable media approach in order to make them evolve and update [
Depocas et al. 2003]. The artistic team only opens the software when it needs
to have an overview, annotate certain documents, or go back over the history of the
files. The second decision is the lack of a specialized ontology. Past experiences
show that these ontologies are difficult to apply by non-specialists and, to
underline the point, they do not respond to all the situations encountered. The tool
has to be as labile as possible and able to accommodate all types of artistic
approaches. The only scheme used is the simplified Dublin Core scheme to describe the
files (this scheme is expected to be implemented soon). In the same spirit, even if
we are open to the possibility of importing thesauri to index the files, we initially
favored folksonomy.
Once the team and the first funding were gathered, the keywords of our working method
became collaboration, testing, and adaptation. During a first step, it took the form
of face-to-face working meetings on a monthly basis, and then less frequently. The
first exchanges were in the form of drawings and graphs. At the heart of our
discussions were the issues mentioned above and to which we tried to provide concrete
solutions in the form of usage scenarios. In other words, how were we to reconcile
theory and practice? A state of the art study was carefully examined, in particular,
the work initiated by William Forsythe and the research carried out at IRCAM, were
discussed at length. Once the issues and objectives were clearly defined, a first
development phase began, leading to a beta version at the end of 2013. There were
many iterations, Guillaume Marais and Guillaume Jacquemin working together on the
development and interface, Thierry Coduys and myself giving feedback. If I brought a
researcher's point of view, Thierry Coduys brought the artistic team’s point of
view.
We also associated two artistic teams to the project, in the theatre (Jean-François
Peyret) and dance (Mylène Benoit) domains, in residence respectively at Le Fresnoy
and Le Phénix scène nationale Valenciennes (France). There both were the initial
partners of the project. On the one hand, we needed “real” data to work on a
tangible case. This was made possible by the Re: Walden
project mentioned above. On the other hand, I wanted to be involved in a creative
process from the beginning to understand better how Rekall could fit in and what its
contributions could be. Mylène Benoit opened the doors of the rehearsals of Notre Danse, a choreography created in 2014. Many informal
interviews with these artists and their teams have allowed us to clarify many aspects
of Rekall. Finally, workshops, as well as public presentations (study days,
conferences, talks), allowed us to confront different audiences (researchers,
artists, technicians) and to fine-tune the project. A new use, which we had not
considered, emerged: the possibility of using Rekall as a writing tool. This confirms
the flexibility of the platform. Nonetheless, to consider all the implications of
this new use would be a very important shift. At this stage of the project, we
decided not to address it directly. We took the idea into account for MemoRekall and
mentioned this possibility during workshops.
Several specialists were invited to share their feedback on Rekall during
brainstorming sessions with the entire team. In particular, we met with: Alain
Bonardi (musicologist and composer, Paris 8), Pierre Couprie (musicologist, Paris 4),
Émeline Brulé (design researcher, ENSAD), Cécile Obligi (BnF) and Sébastien Peyrard
(metadata standards, BnF), Frédéric Bevilacqua (IRCAM), Lev Manovich (CUNY), Annick
Bureaud (Leonardo/Olats), Scott Delahunta (Motion Bank), amongst others. Listening to
users so as not to develop a solution only for my personal use and to respond to as
wide a range of uses as possible remains a constant concern, even today.
By working with others and listening, the development team were led to adapt the
initial project. Rekall has known several versions - and we hope so, will know more.
The first phase, from the end of 2012 to November 2013, was based on the case study
of Jean-François Peyret's Re: Walden project. The Rekall
team has been collecting documents, as well as those related to the different stages
of the creative process since 2006. This made it possible to develop the
data-visualization part of Rekall. Indeed, it quickly became apparent that the
software had to offer two main uses: data visualization and a timeline. The data
visualization interface presents the entire corpus in a multimodal environment and
allows the user to play with different views from the extraction of metadata from the
files. The timeline interface allows organizing a selection of documents on a time
axis. One or more video recordings are used as the primary document to define the
time axis. It is then possible to play several video streams in parallel, which
allows for example to compare several rehearsals or a front and a zenithal view.
Timeline mode uses only a small selection of documents. It is principally designed
for document annotation purposes, to complement information that is not perceptible
and intelligible through data visualization.
With Thierry Coduys, we tested an alpha version from November 2013. He focused on
artistic uses, I studied related research issues. Another development phase on the
timeline began in February 2014. In May 2014, a workshop at Le Fresnoy brought
together developers, technicians, and artists to discuss the development. Many points
were validated, notably on metadata processing and the general interface. Specific
functionalities related to the documentation of the creation process were specified,
some interface details were revised, and some functionalities simplified. This work
allowed the beta version to be released online on June 13, 2014 (see Figure 1).
Following various tests, we decided to change the development methods and integrate
an interface in HTML5 and CSS3 in order to anticipate the uses and development of an
online version. This new development took place in the fall of 2014. The code was
completely rewritten, and a V0 was published in November 2014. The user feedback and
beta test on the V0 led us to refine and simplify the interface and the various
functionalities, with a clear separation between data visualization and timeline. A
V1 of the prototype was available at the end of 2015 (see Figure 2).
The initial version of Rekall was aimed at a professional audience. Nevertheless, a
demand for the general public and schoolchildren quickly appeared. The school
partners of the Phénix scène nationale Valenciennes wanted to have access to the
content created with Rekall on a website available to students. They also wanted
students to be able to document the works themselves collaboratively and directly,
either from documents provided by the theatres and artistic teams hosted or from
documents created by the students themselves. At the same time, a team of young
writers contacted us. They wanted to renew theater criticism by having a
collaborative writing tool that could be superimposed on a video recording available
and broadcasted on the Internet.
This is why we started to develop MemoRekall at the end of 2014, with funding from
the French Ministry of Culture and other cultural institutions that joined the
project. All the work done on the timeline part of Rekall was used and simplified to
the utmost. While Rekall remains for the moment a standalone software, MemoRekall is
designed from the beginning as a web application accessible from a browser (see
Figure 3). In order to validate the software design, various workshops and training
sessions were organized in September 2015 with middle school and high school
students, higher education students, teachers and cultural professionals (more than
100 people participated in this phase of the project). Edwige Perrot, a theater
researcher and a teacher, joined the team to create capsules and lead the workshops
for school pupils. MemoRekall was made available to the public at the end of
September 2015. The web application has allowed us to solve codec problems that were
not solved with Rekall. We decided to release a Version 2 of the prototype in 2017
without the timeline interface. Rekall now focused on the data visualization part and
processed a more significant number of documents simultaneously (about 6,000). This
first scaling up must be continued as it is not rare to gather more than 10 000
documents for one work as we have seen with Re: Walden.
Also in 2017, Estelle Senay, network engineer, joined the team. A personal user
account to MemoRekall was added and a new website was deployed at the end of the
year. In 2018 the usage scenarios were fine tuned and training provided for
MemoRekall. If the web application is taken as an operational tool, Rekall is still
at the advanced prototype stage. A search for funding is underway to scale up the
system. Many features remain to be developed. For example, the performing arts
involve collaborative teams, it would therefore be useful to have different people
working together on a same Rekall project. At this stage, Rekall allows a
collaborative collection of the traces but lends itself to an individual analysis of
these traces. This necessitates that collaborative tools be developed further for
Rekall.
Conclusion: Outlook
The preservation of the performing arts in the digital age does not call for a
univocal solution but for a complementarity of strategies: diachronic documentation
of works based on documentation that is as rigorous as it is precise, preserving a
historical and material anchorage; inter and intra-documentary annotation of video
recording, this trace that is now omnipresent; circulation between different reading
distances. The complex nature of performing arts makes it irreducible to a single
approach; any attempt at documentation is characterized by its incompleteness. What
these different strategies have in common is the redocumentarisation of traces. Each
time a reader consults, links or extracts data from digital traces, it re-documents
them. Re-playing, “re-enacting” a performance is necessarily today
to go through a performative act carried out within the digital traces. This
performativity is synonymous with instrumentation. We have thus proposed two
complementary tools, Rekall and MemoRekall, which could be used in wider fields such
as time based media art preservation, digital art history or electronic
publishing.
The reception of Rekall and MemoRekall is enthusiastic. As soon as it appeared in
2015, theatres and classes seized MemoRekall for their activities. Today, more than 1
500 capsules have been produced. Since its first version in 2014, Rekall has been
downloaded several thousand times. The whole project has given rise to numerous
articles, talks, and workshops, in France and abroad. There are many development
opportunities, and new functionalities must be implemented to get as close as
possible to an answer to the questions raised by the digital traces of the performing
arts. Moreover, if the project was initially designed for the performing arts, its
scope may cover other artistic practices.
The development of this project, based in a specific field and in the French context,
leads to several reflections. On the one hand, Rekall changes the genetic studies of
the performing arts. It is now possible to take into account all the digital traces
of a creation process in a multimodal environment. Retreiving metadata with Rekall
allows distant reading and at the same time while a keeping a link to the original
file. It also allows close reading of the most important documents identified through
distant reading. For researchers of the performing arts this provides a new way to
circulate between micro and macro dimensions of a corpora. On the other hand,
MemoRekall provides for researchers a new publishing tool, articulating
video-recording, documents (archives or documents created by the researcher) and
annotations, and allows co-editing. The connection and complementarity between Rekall
and MemoRekall is a project in its own right, but certain “bricks” have already
been laid (common development core, compatible interfaces). Indeed, we are seeking to
reconcile two approaches often considered to be opposed in digital history and the
treatment of digital traces. The first one shows the sources and superimposes
different strata to make them explicit. We find here the question of annotation and a
critical work specific to the humanities. This approach is anchored in a practice of
editorialization. The second approach extracts data sources and proposes to analyse
them by applying different computational methods. This approach is based on the école
des annales and the quantitative approach.
The question of the sustainability of digital technology is not only a technological
issue. It is also an institutional and cultural issue. For Rekall and MemoRekall,
this would mean finding a framework that allows for the maintenance, development, and
hosting of open source and free software solutions. It is not the role of cultural
institutions that financed the project to put this responsibility on a small team
made up, by in large, of self-employed workers in, invarariably, precarious economic
situations. MemoRekall is now hosted by Huma-num (a French “Very Large Research
Infrastructure” - “Très Grande Infrastructure de Recherche”, TGIR) that
provides long term preservation of the capsules created with the web-application.
Clarisse Bardiot, Université Polytechnique Hauts-de-France
Works Cited
Bachimont 2008 Bachimont, B., 2008. “Formal Signs and Numerical Computation: Between Intuitionism and
Formalism. Critique of Computational Reason”, in: Schramm, H., Schwarte,
L., Lazardzig, J. (Eds.), Instruments in Art and Science: On the
Architectonics of Cultural Boundaries in the 17th Century. Walter de
Gruyter, Berlin, New York, 362-382.
Bardiot 2009 Bardiot, C., 2009. “Noter, annoter. Comment documenter les arts de la scène à composante
technologique” - “Noting/Annotating. Documenting the
technology-based performing arts”. Art Press 2
30–36.
Bardiot 2012 Bardiot, C., 2012. “Eclats : un projet de logiciel pour annoter des captations vidéo”, in:
Martinez Thomas, M., Proust, S. (Eds.), Hispania : La Notation
Du Travail Théâtral : Du Manuscrit Au Numérique. Presented at the La
notation du travail théâtral: du manuscrit au numérique, Lansman, Toulouse, France,
131–148.
Bardiot 2015 Bardiot, C., 2015. “Video recording and documentation of scenic arts: from the annotation to the
visualization of metadata, the example of the Rekall software”. Acoustic Space : Data Drift. Archiving Media and Data Art in the
21st Century 14, 159–168.
Bardiot 2017 Bardiot, C., 2017. “Organiser et conserver la mémoire de l’éphémère : les capsules de
MemoRekall”. Culture et Musées,
159–174.
Bardiot et al. 2014 Bardiot, C., Marais, G.,
Jacquemin, G., Coduys, T., 2014. “Rekall : un environnement
open-source pour documenter, analyser les processus de création et simplifier la
reprise des œuvres scéniques”, in: Captation,
Transformation, Sonification. Presented at the Les Journées d’Informatique
Musicale, AFIM, Talence, France, pp. 119–129.
Bleeker 2017 Bleeker, M. (Ed.), 2017. Transmission in Motion: The Technologizing of Dance.
Routledge, London ; New York.
Depocas et al. 2003 Depocas, A., Ippolito, J., Jones,
C. (Eds.), 2003. Permanence Through Change: The Variable Media
Approach = La Permanence par le changement :
l’approche des médias variables. Guggenheim Museum Publications ;
Fondation Daniel Langlois pour l’art, la science et la technologie, New York;
Montréal.
Drucker 2014 Drucker, J., 2014. Graphesis: Visual Forms of Knowledge Production, MetaLABprojects. Harvard
University Press, Cambridge (MA).
Goody 1977 Goody, J., 1977. The
Domestication of the Savage Mind. Cambridge University Press,
Cambridge.
Melzer 1995a Melzer, A., 1995. “‘Best Betrayal’: The Documentation of Performance on Video and Film, Part
1”. New Theatre Quarterly 11, 42,
147–157.
Melzer 1995b Melzer, A., 1995. “‘Best Betrayal’: The Documentation of Performance on Video and Film, Part
2”. New Theatre Quarterly 11, 43,
259–76.
Pomian 1992 Pomian, K., 1992. “Les
archives. Du trésor des chartes au Caran”, in: Nora, P. (Ed.), Les Lieux de Mémoire. T. III, Vol. 3. Gallimard, Paris,
163-233.
Rinehart and Ippolito 2014 Rinehart, R., Ippolito,
J., 2014. Re-Collection: Art, New Media, and Social Memory,
Leonardo. The MIT Press, Cambridge (MA).