DHQ: Digital Humanities Quarterly
2020
Volume 14 Number 3
2020 14.3  |  XMLPDFPrint

Theatre analytics: developing software for theatre research

Clarisse Bardiot  <clarisse_bardiot_at_me_dot_com>, Université Polytechnique Hauts-de-France

Abstract

How can digital traces of the performing arts be interpreted? What methodologies can be proposed to “make them talk”? The specificity of these traces leads to specific methodological responses, especially as the aim here is to interpret the traces in both directions of understanding and replayability. It is as much a question of advancing a reflection on the conditions for the recovery of work, as it is of analyzing an artistic approach based on its digital traces. Starting from these epistemological issues, I present the development of two softwares, Rekall and MemoRekall. These tools are part of a larger trend which I call “theatre analytics”. Theatre analytics is based on data from the traces of the performing arts. Unlike the general approach of “big data in the social sciences and humanities,” where we seek to determine general, average, profile categories or repeatable (or even predictable) patterns, our primary concern remains to capture the singular, the detail, the difference, the anomaly, in a constant round trip between the micro and the macro dimensions. The ambition of theatre analytics is to offer a different way of looking at big data, more oriented towards complexity than the quantity of data.

Digital technology has flooded the theatre. On stage, it is omnipresent: image projections, real-time transformations of the actors' voices, interactions between the performers and the stage enhanced by all kinds of devices. Behind the scenes, there is no longer a single sound or light control room that is not digital, with the exception of shows in the dark, in daylight or flickering candlelight or even the rare performances without a sound broadcasting system. However, it is not just about the shows. Creative processes are also impacted, as are other aspects of theatrical activity: distribution of works, criticism, public reception, communication, financing. If one wants to study the performing arts today, it is impossible not to take into account companies' websites, blog and online newspaper articles, recordings broadcast on Youtube or Vimeo, social networks, email exchanges, budget tables, ticketing software. Today, directors' notebooks also take the form of Word files, image-based blogs on Tumblr or Instagram, posts on Facebook, Evernote notebooks, Pinterest tables, task lists in Trello, exchanges in Slack. The traces of the performing arts have become digital traces. This phenomenon has been driven by two major trends: the emergence of the born-digital heritage and theatre archives digitization.
The digital traces of the performing arts irremediably change not only the nature of the sources on which we rely to analyze works and write the history of performing arts but also how these studies are conducted and their results. The transformation of the trace into data-based information, making data a new type of source, is fraught with consequences for the memory of the ephemeral theatre, consequences that are not yet fully measured by the various actors, be they cultural institutions, librarians or researchers. Historiography in a digital context is not a revival of quantitative history in the strict sense of the term. It calls first and foremost for practices, and it is also in this practice that the narrative is housed. It is because it is a question of practice, and not just of exposing traces, but instead of exposing them to manipulation that a paradigm shift is taking place, inviting us to “make history differently.” Because the traces are transformed, they invite a renewal of the critique of sources that we thought we knew and make research questions evolve and lead to other epistemologies. The “digital turn” proposes a paradigm shift in how we build the memory of performing arts by renewing source criticism, methodological thinking, and narratives, moving beyond traditional methods of close reading to computational assisted reading and contributing to the general reflection on digital history.
How can performing arts digital traces be interpreted? What methodologies can be advanced to “make them talk?” The specificity of these traces leads to specific methodological responses, especially since the aim here is to interpret the traces in both directions of understanding and replayability. This is as much a question of developing a reflection on the conditions for the recovery of creative work as it is of analyzing an artistic approach based on its digital traces. Traditional methods, in particular close reading, i.e., a reading, a precise analysis of each of the traces, have to be combined with new approaches.
This growing awareness lead me to the creation of a software prototype, Rekall, and a web application, MemoRekall, which I am still developing today. Both environments are free and open source. Rekall and MemoRekall provide new opportunities of interpreting digital documents for performing arts research. They offer different and unexpected points of view on a dataset, either from a distant or a close reading approach, either by extracting metadata or by linking documents. Furthermore, the softwares take into account the open nature of the digital archive, which reconfigures itself as new documents are added, new links are created, and allow collaborative research.
From an epistemological point of view, the creation of these tools is anchored in the emergence of a new field of research that can be called “theatre analytics.” Theatre analytics are based on data from the traces of the performing arts. In line with “culture analytics,” the term refers to the calculation and emphasizes the analysis of works, which is the foundation for the preservation and historiography of the performing arts. In both cases it is a matter of following in detail a process to show why and how a work is constituted by observing its various components and of borrowing from mathematics to study, preserve and write a history of the performing arts. Theatre analytics does not challenge existing methodologies but allows them to be supplemented by taking into account what is generally discarded, namely, digital traces, considered too complex, too numerous, too fragile, with dubious authenticity. The aim is not to promote an approach that would be purely quantitative or statistical but on the contrary to see how qualitative analyses can also be conducted in a digital context. Unlike the general approach of “big data in the social sciences and humanities,” where we seek to determine general, average, profile categories or repeatable (or even predictable) patterns, our primary concern remains to seize the singular, the detail, the difference, the anomaly in a constant round trip between the micro and the macro dimensions, and looking for mesoscales. In the tracking of the singular at the core of big data, the deviation from the mean is made visible as such and no longer as an error against a standard. Nor is it a question of considering that singularity would be calculable, as some research on emotions or creativity seems to state by reducing digital traces to mathematical artifacts. On the contrary, we argue for an approach where the singular, the ideosyncraric via computational methods, be the focus of a qualitative interpretation. Data visualisation is ultimately qualitative and must give rise to an accurate and informed reading. As Johanna Drucker, who calls for a “visual epistemology,” points out “Visualizations are always interpretations — data does not have an inherent visual form that merely gives rise to a graphic expression” [Drucker 2014, 7]. In this light, the ambition of theatre analytics is to offer a different way of looking at big data that is more oriented towards complexity, rather than the quantity of data, with a diversity of computer approaches depending on the nature of the data and the research questions asked. Alongside other methodologies less specific to the performing arts but which can also beRekall and MemoRekall afforts such an approach. Part of the text is missing : Alongside other methodologies less specific to the performing arts but which can also be convened (NLP, network analysis, machine learning, algorithm creation, ...) Rekall and MemoRekall afforts such an approach.
This paper will explicate the development context of Rekall and MemoRekall and provide lessons learnt from this experiment. The detailed technical functions of the two softwares can be found elsewhere [Bardiot et al. 2014] [Bardiot 2015] [Bardiot 2017]. The question can be asked how a performing arts researcher comes to develop new software applications? To explain this, first, I will set out the specificity of the problems generated by the digital traces of the performing arts. Second, main objectives of the project and the choices that result from it are explained. Third, the different stages of development and in particular the collaborations with artistic teams, programmers and designers are laid out.

The digital traces of the performing arts

Compared to non-digital traces, what do digital traces change for the preservation, analysis of works and the performing arts history? Let us take the tip of the “iceberg” of technological obsolescence and big data. A creative process, from its initial ideas to the première, lasts an average of four years - the same length as that of a technological cycle. In other words, at the moment of the première, it is necessary to update the management software and computer systems accompanying the tour, to save and sort the hundreds, sometimes thousands, of documents related to the work in a new version. As soon as the work is created, the artistic and technical team is confronted with problems of technological obsolescence and a quantity of information that may have a decisive impact on the production and its distribution of the work and can even, in the medium term, make it impossible to tour.
The preservation of performances is not the only phenomenon concerned by these issues. The creative processes are also affected. To take the example of the director's notebooks, in 20 years, who will remember applications such as Trello, Slack, Evernote or Tumblr? Will we be able to read the digital traces that have been stored and have access to the data they contain while guaranteeing their authenticity? Not only are performances ephemeral, which is inherent to the performing arts, but their digital traces seem even more evanescent. They are also increasingly numerous, overwhelming us with a continuous flow of data, in an unprecedented disproportion. These questions are not all new, but their amplification gives them a new acuity. How can we write the history of the performing arts from not only fragile and variable traces, but also too numerous at for researcher or archivist?
It is undeniable that, today, digital technology involves radical, and even irreversible, changes in our relationship to memory, history, and what is left as a legacy. Given this, one can but note that memory and the digital seem to oppose each other. This need not be so. Reconciling them is essential. Digital traces cannot be ignored in building and safeguarding the memory of the performing arts. These issues are complex and urgent. If nothing is done, the technological obsolescence of digital technology will overtake all other considerations, and an entire part of our culture will vanish.
To take a concrete example, under the title Re: Walden, are regrouped different avatars of the same creative work, developed by the French director Jean-François Peyret from 2006 to 2014. This involved two installations (2010 and 2013 at Le Fresnoy, France), performances in the installation, four scenic versions (including one for the Avignon 2013 festival, France), a concert at EMPAC (USA) and an extension in the virtual environment Second Life. Between 2012 and 2013, I collected several hard disks from the artistic team. These were those of Thierry Coduys (electroacoustic and computer devices), Agnès de Cayeux (virtual world) and Julie Valéro (dramaturgy). Although this collection is mostly incomplete (ideally, access to all the digital data of each member of the artistic team would have been required), the data collected were 10,360 files for 20.5 GigaBytes.
The files collected include the following elements: scores[1], photographs and video recordings of rehearsals and performances, sound recordings for voice synthesis, sound files as well as computer programs (Max/MSP and Pure Data patches), images, management documentation (contacts, schedules...), technical sheets, room plans, communication documents (playbills, press releases, reviews published in newspapers or blogs), archived e-mails, screenshots of Second Life, texts for each of the actors or texts from the automatic translator developed for the project, meeting minutes, notes of intent from the artistic team.
This example is symptomatic of digital traces of the performing arts, even if it concerns a complex creation that takes place over several years. They are heterogeneous, numerous but incomplete and fragile. The digital traces of the performing arts are heterogeneous, both in terms of the type of file (image, sound, text, video, computer program) and their content. All is happily mixed without any distinction and with the elements of the show such as aspects about the creative process or publicity compaigns, and the tour. These three aspects make it possible to define three main categories of digital traces that correspond to the main chronological stages in the life of a performance: the creative process, the work, audience reception. We can add a fourth one, not represented here: the administrative digital traces of cultural institutions and companies, i.e., everything related to the production of shows.
There are many digital traces, so much so that the term “big data” is used. We have counted several thousand files for Re: Walden. The situation is nothing new, and we can only observe the inflation of digital traces. However, despite their impressive volume, the traces are far from being exhaustive. Besides, some tasks left only minimal traces, for example there were only a few photos and patches in the Pure Data software for the residency at the École Régionale d'Acteurs de Cannes. Finally, digital traces do not take into account a large part of the exchanges, actions, and reflections taking place on and off the set, if they had not been systematically recorded or were in the form of reports and notes. Part of the digital activity itself rarely leads to the creation of preserved traces, like the one that takes place on electronic networks, such as documents shared on Google drive, conversations on Skype or applications such as Slate or Trello, images made available on Tumblr.
Digital traces are fragile. The duration of access and readability of a digital file might be five years. So that not only are we faced with ephemeral works, but also with traces increasingly fragile. For a large part of them, in a few years, especially those from proprietary software (Word for example), we will probably no longer be able to open them and therefore read them. The loss of traces seems to be an unavoidable phenomenon. In fact, many artists have already lost not only traces but also creative works.
In this overview, we understand that digital traces exacerbate both the properties of “analogical” traces, while at the same time differentiating themselves from them. As for the first observation, linked to heterogeneity, number or fragility, it is ultimately “merely” a question of accentuating phenomena already well identified in the analogical world. The great diversity of the nature of documents in the performing arts collections; the inflation of archives is a recurring problem identified, notably by Krzystof Pomian [Pomian 1992], where paper ends up being reduced to dust, inks are erased, magnetic strips become electronic snow. Despite the appearance of continuity, in fact digital technology fundamentally changes the ontology of the trace. To use Bruno Bachimont's expression, we have gone from a “graphic reason”[2] [Goody 1977] to a “computational reason” [Bachimont 2008]. As they become numerical, traces are coded and presented as numbers, which allows for computational calculation. This makes the question of reading traces and their authenticity complex (for they must be “decoded”) while opening up to being manipulated. In other words, digital traces are “analyzable”, in the triple sense of discretization (the decomposition of a thing into its elements), computer calculation and data visualization. Analytical geometry, attributed to Descartes that allows a graphic representation of algebra, is the prelude to contemporary data visualizations. The digital traces of the performing arts thus presents several paradoxes: technological obsolescence versus full archiving, authenticity versus modification, and repetition versus ephemerality.

At the crossroads

In 2006, as a performing arts researcher, I conducted a case study on Belgian choreographer Michèle Noiret on technological obsolescence issues about the performing arts at the margins, given that the works are inherently ephemeral. The case study was part of the DOCAM research program (2005-2010) led by the Daniel Langlois Foundation in Montreal on the documentation and preservation of media arts. The programme includes numerous partners, including museums. It builds on the pioneering work of the Langlois Foundation and John Ippolito on the variable media approach [Depocas et al. 2003]. Jon Ippolito will then publish with Richard Rinehart Re-Collection [Rinehart and Ippolito 2014]. In museums, the very concrete question of technological obsolescence of works was raised. Pip Laurenson proposed new conservation paradigms in questioning the central notions of state and authenticity [Laurenson 2006]. Faced with these questions, the researcher's natural reflex is to identify existing solutions. Best practice guidelines and case studies were published as part of international projects (Interpares, DOCAM, Caspar) [Bardiot 2012]; researchers worked on specific ontologies [Rinehart 2007]; some artists - particularly choreographers - in collaboration with scientists, proposed solutions for a work or set of works [Bleeker 2017]. However, no existing solution seemed able to solve the concrete problems identified in the performing arts field. They can be summarized as follows:
  • How can technical teams be enabled to take quick notes during rehearsals and to aggregate all technical documents? The different control rooms are separate, each one (lighting designer, general manager, video director, sound engineer) has its documentation system in digital or paper format, more or less structured with each participant defines his or her working method, his or her guide to “best practices.”
  • How can a show a few months or a few years later be restaged while respecting as carefully as possible the initial artistic intentions? Due to built-in digital obsolescence, technologies must be continuously adapted during the show tour. In addition, when the creative team is not the same as the touring team, rigorous documentation is required.
  • How can we compensate for the memory loss caused by digital technology? It becomes difficult to trace the history of the creative processes, especially since the documents they generate are themselves ephemeral, if nothing else, in their intended use.
  • What status and place should be given to video recording? A widespread practice, with varying results regarding the quality of video recording implementation, it systematically calls for comment. Video recordings per se are not self-sufficient. Most often produced by the company itself (for promotional purposes), they are nevertheless one of the most significant traces of the art work.
  • How can one move through the hundreds, sometimes thousands of documents to reconstruct the creation process? How can data visualization contribute to the analysis of a creative work's genetic analysis? By comparing different processes, is it possible to identify forms, patterns?
  • The digital traces of the works are very numerous and the reading of each trace is almost impossible. The question becomes rather: what information, what knowledge about the works is it possible to extract from all the traces for preservation purposes? What data should be used to reconstruct a creative process?
In order to respond, at least partially, to these challenges, I published a first paper [Bardiot 2009] proposing a conceptual model of a computer solution articulating documents around the video recording of a performance. In contrast to the practice of creating a specific interface for each performance, I designed a software that can be applied to as many creative works as possible. As my concerns increasingly met those of cultural institutions confronted with the obsolescence of technologies - and the costs that this entails in addition to the difficulty of presenting works to the public - I decided to develop the software by myself. At the end of 2012, I raised the first funds that enabled me to hire a team of developers. Five years after the first draft, the situation had changed. The issues of big data and digital traces mentioned above could no longer be ignored. It was necessary to go beyond the articulation of a documentary corpus around a video recording. In the preparatory meetings that followed, the guiding principles of the project, first entitled Éclats (in reference to the fragmentation and multiplicity of documents) and then Rekall (in reference to the memory company in K. Dick's We Can Remember It for You Wholesale), were clarified. From the outset, Open Source was a self-evident option. Indeed, some funders such as the French Ministry of Culture imposed the Open Source option. Beyond that, Open Source is a prerequisite for the sustainability of the solutions developed. It would have been antinomic and counterproductive to proceed otherwise in a project dedicated to the preservation of art works.
Rekall is an environment prototype to document and analyze creative processes, and to simplify the preservation of performances. It uses the metadata present in all documents of a performance in order to extract crucial data (author, date, place, keyword) then used by data visualization tools to indicate behaviors. In this way, Rekall provides an overview of the creation process and identifies the most important documents (which will then have to undergo specific preservation measures). The software can also be used during rehearsals (for example, to annotate documents, review the history of a plot sheet), or once the creation is completed (to understand its process, distinguish documents related to the work from those related to its production). Rekall is cross platform. It is developed with the Qt development framework. For each version of a document, Rekall extracts all available metadata using the open source tool exiftool. The interface is in HTML5 and CSS3.
Two primary objectives guided Rekall's design: to help artists (the first curators of their works) to document their creations in order to ensure their retrieval and overcome the obsolescence of digital technologies; to help researchers study the genetics of performances in a context of born-digital heritage and Big Data. These two objectives may seem very distant at first sight. However, it is the same materials (digital traces of the works) that are collected, analyzed and visualized based on different modalities and time scales. In this way, the collection can take place for the artists during the creative process itself, as the work stages progress. A living archive emerges in perpetual movement. The researcher intervenes once the process is completed, once the documents have somehow stabilized, frozen to become the traces of what has happened. Rekall must, therefore, be able to be used from the onset of the first ideas of a show, during the creative process and then after its creation, with a dual perspective of preservation of the work and historiography.
Very quickly, it became clear to us that we had to develop a multimodal environment that included both digitized documents and natively digital documents. The digital traces of the performing arts include texts, images, sounds, videos, computer programs. A common practice of big data is to separate documents by file type and work in the form of “silos” (e.g. text, image). Such processing prevents us from obtaining an overview of the traces collected and from examining many phenomena such as the evolution of an idea through different documents, from image to text, from text to table, from table to video recording. Besides, it is essential to navigate between the micro and macro dimensions, between close reading and distant reading, between diachronic and synchronic representations. In other words, being able to represent a process without erasing its complexity; to switch from data to file and back again. Often, distant reading and data visualization involve a break between the source file and the data from which it is extracted. Maintaining this link is fundamental for several reasons. The data are always reductive fragments and it is important to be able to recontextualize them in order to interpret them better. The transition from a trace to data is a reductive and interpretive operation. Reduction also means making a precipitate of traces, which makes it possible to amplify and exhale a phenomenon that would otherwise have remained invisible. The transformation affords new insights into the tracks and allows us to detect clues that would otherwise have gone unnoticed. The use of data visualization is essential in the case of performance preservation to capture the main steps of the creative process and to identify the main files that will then need to be actively preserved, often on a case-by-case basis. In a very concrete way, in a corpus of 10,000 documents, the process of visualization must make it possible to identify the 50 documents that the researcher must read in detail to analyze the work or that the artistic team must migrate or emulate to continue touring.
Among the traces of the performing arts, the video recording is a particular type of document. It has long been - and is still often - considered as a panacea regarding documentation of the ephemeral performing arts. Provoking many debates since the 1960s, often considered as a “betrayal” of the original work [Melzer 1995a] [Melzer 1995b], video recordings have now become commonplace. A video recording is both an essential and a fragmented document. In its essential form, it records bodies in movement and thus transforms performances into documents. In its fragmented form, it gives only one point of view, presenting only a part of what takes place on stage, generating temporal collusion through editing, adapting stage lighting to the lighting conditions of the cameras. A commentary, that is to say, an annotation must necessarily accompany these modifications. Two types of strategies are developed in order to overcome the shortcomings or biases of video recording: inter-documentary (connecting the video-capture to a larger documentary corpus) and intra-documentary (annotating the video-capture). These two approaches can be usefully combined. They simultaneously play different roles with the document, between the deepening and connecting roles.
In a first phase, Rekall was developed not only as a document management tool in a multimodal environment, but also as a video annotation software, with three complementary strategies: the creation of links between multiple documents offering the possibility of commenting and illuminating the documents between them; the choice of video recording as a backbone from which all other documents would be organized; and the possibility of adding textual comments. Subsequently, these two aspects were separated. The video annotation part about of Rekall has now become MemoRekall.
MemoRekall offers a reading of a major document: video recordings. The software is a user-friendly web application that explains a video by annotating it and linking it to external documents or web pages. The arrangement of documents, links and annotations creates a new document, called a “capsule”, that can be embedded into a web page. The capsules organization is developed from and around video recordings in a hypertextual logic. MemoRekall can be used to create a multimedia educational booklet, run a digital spectator school, edit a scientific web documentary or simplify the distribution of works. MemoRekall is developed in HTML5 / CSS3 with a backend using PHP/MySQL technologies. The video player is developed in HTML5, allowing it to be integrated into the main web browsers (Firefox, Safari, Google Chrome). The personal user account was developed with the Symfony framework.
Rekall and MemoRekall are complementary platforms aimed at preserving and analysing the digital traces of the performing arts. They allow to organize and visualize in a multimodal environment all the traces of a creative processe (Rekall) or only a selection of the traces (MemoRekall), paying special attention to the video recording of the works (MemoRekall). With Rekall, it’s possible to retrace the historic of a file (but not the versioning, which can be done with complementary tools). In terms of research, Rekall and MemoRekall are platforms suited to genetic analysis, close and distant reading of a work, documentation and preservation of a performance.

Developping Rekall and MemoRekall : collaborate, test, adapt

In 2012, Guillaume Marais (ergonomics and interface design), Guillaume Jacquemin (IT development) and Thierry Coduys (consultant, designer of electronic devices for the performing arts) joined the project. They already had a first common experience of software development with IanniX, a real-time sequencer for digital art inspired by UPIC (a former project by Iannis Xenakis). Throughout his artistic career, which led him to IRCAM, Thierry Coduys has developed electroacoustic and computer devices for numerous directors, composers (including Luciano Berio, Ivan Fedele, and Pascal Dusapin) and choreographers. He was regularly confronted with the question of touring, restaging and transmitting works with particularly complex technological caracteristics. Guillaume Jacquemin and Guillaume Marais created the Buzzing Light agency in 2009. Guillaume Jacquemin is an embedded systems engineer and is trained through multiple experiences in interactive multimedia and digital art. Guillaume Marais is an ergonomist and interaction designer and is trained in information systems engineering. In addition to their complementary skills in programming and ergonomics, Guillaume Marais and Guillaume Jacquemin were used to contributing to artistic projects and had a keen appetite for contemporary music. It was an essential factor: to develop Rekall, it was crucial to establish a dialogue around the key issues of the software. We used an AGILE approach and progressed incrementally based on user feedback.
I am often asked why I have not chosen instead to work with a computer science research laboratory. There are several answers to this question. One is that I needed to be in a very operational, fluid and fast environment. Furthermore, it did not appear that the project raised computer research issues. In 2012, in France, a computer development project led by a theatre studies researcher was not always well accepted by the academic community. Finally, on one hand, the funding obtained came from funds for innovation in the field of culture (Pictanovo) and from cultural institutions.[3] This gave great flexibility in hiring service providers. On the other hand, the lack of research funding and the administrative management of the project by a theatre (the Phénix scène nationale Valenciennes) and then, in 2015, by a publishing house (Subjectile) did not allow the hiring of a Ph.D. student or a postdoc. Today, with the development of the digital humanities and the resulting cultural change, the software development framework would probably be different. So indeed, the contribution of Thierry Coduys, Guillaume Jacquemin and Guillaume Marais were crucial. The results we obtained would have been very different with another team that wouldn’t have had any experience with artistic creative processes. For example, creating software that can be used by artists has several consequences. The main point is that it must be as easy to use as possible and still meet the rigorous requirements of researchers. This involved a reflection on interface design and ergonomics and two essential decisions. The first is that Rekall had to operate discreetly in the background on the assumption that artists have very little time to archive their work even if they are aware of its importance. The artistic team works on its usual tools while Rekall records in the background all the manipulations that are done as well as the users' working environment (the version of their OS, the open applications, and their version) in order to collect all fundamental information from a preservation perspective. It is essential to keep the most exact trace possible of the technological components and of the aesthetic and historical dimensions. At the same time, it is important to describe the effects of these same components, in line with the variable media approach in order to make them evolve and update [Depocas et al. 2003]. The artistic team only opens the software when it needs to have an overview, annotate certain documents, or go back over the history of the files. The second decision is the lack of a specialized ontology. Past experiences show that these ontologies are difficult to apply by non-specialists and, to underline the point, they do not respond to all the situations encountered. The tool has to be as labile as possible and able to accommodate all types of artistic approaches. The only scheme used is the simplified Dublin Core scheme to describe the files (this scheme is expected to be implemented soon). In the same spirit, even if we are open to the possibility of importing thesauri to index the files, we initially favored folksonomy.
Once the team and the first funding were gathered, the keywords of our working method became collaboration, testing, and adaptation. During a first step, it took the form of face-to-face working meetings on a monthly basis, and then less frequently. The first exchanges were in the form of drawings and graphs. At the heart of our discussions were the issues mentioned above and to which we tried to provide concrete solutions in the form of usage scenarios. In other words, how were we to reconcile theory and practice? A state of the art study was carefully examined, in particular, the work initiated by William Forsythe and the research carried out at IRCAM, were discussed at length. Once the issues and objectives were clearly defined, a first development phase began, leading to a beta version at the end of 2013. There were many iterations, Guillaume Marais and Guillaume Jacquemin working together on the development and interface, Thierry Coduys and myself giving feedback. If I brought a researcher's point of view, Thierry Coduys brought the artistic team’s point of view.
We also associated two artistic teams to the project, in the theatre (Jean-François Peyret) and dance (Mylène Benoit) domains, in residence respectively at Le Fresnoy and Le Phénix scène nationale Valenciennes (France). There both were the initial partners of the project. On the one hand, we needed “real” data to work on a tangible case. This was made possible by the Re: Walden project mentioned above. On the other hand, I wanted to be involved in a creative process from the beginning to understand better how Rekall could fit in and what its contributions could be. Mylène Benoit opened the doors of the rehearsals of Notre Danse, a choreography created in 2014. Many informal interviews with these artists and their teams have allowed us to clarify many aspects of Rekall. Finally, workshops, as well as public presentations (study days, conferences, talks), allowed us to confront different audiences (researchers, artists, technicians) and to fine-tune the project. A new use, which we had not considered, emerged: the possibility of using Rekall as a writing tool. This confirms the flexibility of the platform. Nonetheless, to consider all the implications of this new use would be a very important shift. At this stage of the project, we decided not to address it directly. We took the idea into account for MemoRekall and mentioned this possibility during workshops.
Several specialists were invited to share their feedback on Rekall during brainstorming sessions with the entire team. In particular, we met with: Alain Bonardi (musicologist and composer, Paris 8), Pierre Couprie (musicologist, Paris 4), Émeline Brulé (design researcher, ENSAD), Cécile Obligi (BnF) and Sébastien Peyrard (metadata standards, BnF), Frédéric Bevilacqua (IRCAM), Lev Manovich (CUNY), Annick Bureaud (Leonardo/Olats), Scott Delahunta (Motion Bank), amongst others. Listening to users so as not to develop a solution only for my personal use and to respond to as wide a range of uses as possible remains a constant concern, even today.
By working with others and listening, the development team were led to adapt the initial project. Rekall has known several versions - and we hope so, will know more. The first phase, from the end of 2012 to November 2013, was based on the case study of Jean-François Peyret's Re: Walden project. The Rekall team has been collecting documents, as well as those related to the different stages of the creative process since 2006. This made it possible to develop the data-visualization part of Rekall. Indeed, it quickly became apparent that the software had to offer two main uses: data visualization and a timeline. The data visualization interface presents the entire corpus in a multimodal environment and allows the user to play with different views from the extraction of metadata from the files. The timeline interface allows organizing a selection of documents on a time axis. One or more video recordings are used as the primary document to define the time axis. It is then possible to play several video streams in parallel, which allows for example to compare several rehearsals or a front and a zenithal view. Timeline mode uses only a small selection of documents. It is principally designed for document annotation purposes, to complement information that is not perceptible and intelligible through data visualization.
With Thierry Coduys, we tested an alpha version from November 2013. He focused on artistic uses, I studied related research issues. Another development phase on the timeline began in February 2014. In May 2014, a workshop at Le Fresnoy brought together developers, technicians, and artists to discuss the development. Many points were validated, notably on metadata processing and the general interface. Specific functionalities related to the documentation of the creation process were specified, some interface details were revised, and some functionalities simplified. This work allowed the beta version to be released online on June 13, 2014 (see Figure 1).
A screenshot from the Rekall interface, showing a split screen with various
                  data visualizations.
Figure 1. 
Rekall “timeline” interface with split screen. Beta version, 2014.
Following various tests, we decided to change the development methods and integrate an interface in HTML5 and CSS3 in order to anticipate the uses and development of an online version. This new development took place in the fall of 2014. The code was completely rewritten, and a V0 was published in November 2014. The user feedback and beta test on the V0 led us to refine and simplify the interface and the various functionalities, with a clear separation between data visualization and timeline. A V1 of the prototype was available at the end of 2015 (see Figure 2).
A screenshot from the Rekall interface showing various files.
Figure 2. 
Rekall interface. V1 prototype, 2015.
The initial version of Rekall was aimed at a professional audience. Nevertheless, a demand for the general public and schoolchildren quickly appeared. The school partners of the Phénix scène nationale Valenciennes wanted to have access to the content created with Rekall on a website available to students. They also wanted students to be able to document the works themselves collaboratively and directly, either from documents provided by the theatres and artistic teams hosted or from documents created by the students themselves. At the same time, a team of young writers contacted us. They wanted to renew theater criticism by having a collaborative writing tool that could be superimposed on a video recording available and broadcasted on the Internet.
This is why we started to develop MemoRekall at the end of 2014, with funding from the French Ministry of Culture and other cultural institutions that joined the project. All the work done on the timeline part of Rekall was used and simplified to the utmost. While Rekall remains for the moment a standalone software, MemoRekall is designed from the beginning as a web application accessible from a browser (see Figure 3). In order to validate the software design, various workshops and training sessions were organized in September 2015 with middle school and high school students, higher education students, teachers and cultural professionals (more than 100 people participated in this phase of the project). Edwige Perrot, a theater researcher and a teacher, joined the team to create capsules and lead the workshops for school pupils. MemoRekall was made available to the public at the end of September 2015. The web application has allowed us to solve codec problems that were not solved with Rekall. We decided to release a Version 2 of the prototype in 2017 without the timeline interface. Rekall now focused on the data visualization part and processed a more significant number of documents simultaneously (about 6,000). This first scaling up must be continued as it is not rare to gather more than 10 000 documents for one work as we have seen with Re: Walden. Also in 2017, Estelle Senay, network engineer, joined the team. A personal user account to MemoRekall was added and a new website was deployed at the end of the year. In 2018 the usage scenarios were fine tuned and training provided for MemoRekall. If the web application is taken as an operational tool, Rekall is still at the advanced prototype stage. A search for funding is underway to scale up the system. Many features remain to be developed. For example, the performing arts involve collaborative teams, it would therefore be useful to have different people working together on a same Rekall project. At this stage, Rekall allows a collaborative collection of the traces but lends itself to an individual analysis of these traces. This necessitates that collaborative tools be developed further for Rekall.
A screenshot of the MemoRekall interface showing the application being
                  used.
Figure 3. 
MemoRekall interface, 2015.

Conclusion: Outlook

The preservation of the performing arts in the digital age does not call for a univocal solution but for a complementarity of strategies: diachronic documentation of works based on documentation that is as rigorous as it is precise, preserving a historical and material anchorage; inter and intra-documentary annotation of video recording, this trace that is now omnipresent; circulation between different reading distances. The complex nature of performing arts makes it irreducible to a single approach; any attempt at documentation is characterized by its incompleteness. What these different strategies have in common is the redocumentarisation of traces. Each time a reader consults, links or extracts data from digital traces, it re-documents them. Re-playing, “re-enacting” a performance is necessarily today to go through a performative act carried out within the digital traces. This performativity is synonymous with instrumentation. We have thus proposed two complementary tools, Rekall and MemoRekall, which could be used in wider fields such as time based media art preservation, digital art history or electronic publishing.
The reception of Rekall and MemoRekall is enthusiastic. As soon as it appeared in 2015, theatres and classes seized MemoRekall for their activities. Today, more than 1 500 capsules have been produced. Since its first version in 2014, Rekall has been downloaded several thousand times. The whole project has given rise to numerous articles, talks, and workshops, in France and abroad. There are many development opportunities, and new functionalities must be implemented to get as close as possible to an answer to the questions raised by the digital traces of the performing arts. Moreover, if the project was initially designed for the performing arts, its scope may cover other artistic practices.
The development of this project, based in a specific field and in the French context, leads to several reflections. On the one hand, Rekall changes the genetic studies of the performing arts. It is now possible to take into account all the digital traces of a creation process in a multimodal environment. Retreiving metadata with Rekall allows distant reading and at the same time while a keeping a link to the original file. It also allows close reading of the most important documents identified through distant reading. For researchers of the performing arts this provides a new way to circulate between micro and macro dimensions of a corpora. On the other hand, MemoRekall provides for researchers a new publishing tool, articulating video-recording, documents (archives or documents created by the researcher) and annotations, and allows co-editing. The connection and complementarity between Rekall and MemoRekall is a project in its own right, but certain “bricks” have already been laid (common development core, compatible interfaces). Indeed, we are seeking to reconcile two approaches often considered to be opposed in digital history and the treatment of digital traces. The first one shows the sources and superimposes different strata to make them explicit. We find here the question of annotation and a critical work specific to the humanities. This approach is anchored in a practice of editorialization. The second approach extracts data sources and proposes to analyse them by applying different computational methods. This approach is based on the école des annales and the quantitative approach.
The question of the sustainability of digital technology is not only a technological issue. It is also an institutional and cultural issue. For Rekall and MemoRekall, this would mean finding a framework that allows for the maintenance, development, and hosting of open source and free software solutions. It is not the role of cultural institutions that financed the project to put this responsibility on a small team made up, by in large, of self-employed workers in, invarariably, precarious economic situations. MemoRekall is now hosted by Huma-num (a French “Very Large Research Infrastructure” - “Très Grande Infrastructure de Recherche”, TGIR) that provides long term preservation of the capsules created with the web-application.
Clarisse Bardiot, Université Polytechnique Hauts-de-France

Notes

[1] In Jean-François Peyret's vocabulary, scores are a montage of texts that serve as a basis for rehearsals and then become the central theme of the performance.
[2] In French, the translation of the title of Goody’s book is La raison graphique : la domestication de la pensée sauvage.
[3] In 2014 the French Ministry of Culture, MA national scene of Montbéliard, Le Fresnoy, Le phénix national scene of Valenciennes then in 2015 the National Centre for Circus Arts, the International Institute of Puppetry and the Rhône Alpes region, in collaboration with the Maison de la Danse, the Hexagone national scene of Meylan, the Lux national scene of Valence, the Théâtre Nouvelle Génération and the École Nationale Supérieure d'Arts et Techniques du Théâtre.

Works Cited

Bachimont 2008 Bachimont, B., 2008. “Formal Signs and Numerical Computation: Between Intuitionism and Formalism. Critique of Computational Reason”, in: Schramm, H., Schwarte, L., Lazardzig, J. (Eds.), Instruments in Art and Science: On the Architectonics of Cultural Boundaries in the 17th Century. Walter de Gruyter, Berlin, New York, 362-382.
Bardiot 2009 Bardiot, C., 2009. “Noter, annoter. Comment documenter les arts de la scène à composante technologique” - “Noting/Annotating. Documenting the technology-based performing arts”. Art Press 2 30–36.
Bardiot 2012 Bardiot, C., 2012. “Eclats : un projet de logiciel pour annoter des captations vidéo”, in: Martinez Thomas, M., Proust, S. (Eds.), Hispania : La Notation Du Travail Théâtral : Du Manuscrit Au Numérique. Presented at the La notation du travail théâtral: du manuscrit au numérique, Lansman, Toulouse, France, 131–148.
Bardiot 2015 Bardiot, C., 2015. “Video recording and documentation of scenic arts: from the annotation to the visualization of metadata, the example of the Rekall software”. Acoustic Space : Data Drift. Archiving Media and Data Art in the 21st Century 14, 159–168.
Bardiot 2017 Bardiot, C., 2017. “Organiser et conserver la mémoire de l’éphémère : les capsules de MemoRekall”. Culture et Musées, 159–174.
Bardiot et al. 2014 Bardiot, C., Marais, G., Jacquemin, G., Coduys, T., 2014. “Rekall : un environnement open-source pour documenter, analyser les processus de création et simplifier la reprise des œuvres scéniques”, in: Captation, Transformation, Sonification. Presented at the Les Journées d’Informatique Musicale, AFIM, Talence, France, pp. 119–129.
Bleeker 2017 Bleeker, M. (Ed.), 2017. Transmission in Motion: The Technologizing of Dance. Routledge, London ; New York.
Depocas et al. 2003 Depocas, A., Ippolito, J., Jones, C. (Eds.), 2003. Permanence Through Change: The Variable Media Approach = La Permanence par le changement : l’approche des médias variables. Guggenheim Museum Publications ; Fondation Daniel Langlois pour l’art, la science et la technologie, New York; Montréal.
Drucker 2014 Drucker, J., 2014. Graphesis: Visual Forms of Knowledge Production, MetaLABprojects. Harvard University Press, Cambridge (MA).
Goody 1977 Goody, J., 1977. The Domestication of the Savage Mind. Cambridge University Press, Cambridge.
Laurenson 2006 Laurenson, P., 2006. “Authenticity, Change and Loss in the Conservation of Time-Based Media Installations” Tate Papers. https://www.tate.org.uk/research/publications/tate-papers/06/authenticity-change-and-loss-conservation-of-time-based-media-installations.
Melzer 1995a Melzer, A., 1995. “‘Best Betrayal’: The Documentation of Performance on Video and Film, Part 1”. New Theatre Quarterly 11, 42, 147–157.
Melzer 1995b Melzer, A., 1995. “‘Best Betrayal’: The Documentation of Performance on Video and Film, Part 2”. New Theatre Quarterly 11, 43, 259–76.
Pomian 1992 Pomian, K., 1992. “Les archives. Du trésor des chartes au Caran”, in: Nora, P. (Ed.), Les Lieux de Mémoire. T. III, Vol. 3. Gallimard, Paris, 163-233.
Rinehart 2007 Rinehart, R., 2007. “The Media Art Notation System: Documenting and Preserving Digital/Media Art”. Leonardo 40, 181–187. https://doi.org/10.1162/leon.2007.40.2.181
Rinehart and Ippolito 2014 Rinehart, R., Ippolito, J., 2014. Re-Collection: Art, New Media, and Social Memory, Leonardo. The MIT Press, Cambridge (MA).
2020 14.3  |  XMLPDFPrint