Film and digital humanities scholars have created theoretical and historical models that expand our understanding of how bodies can manifest on screen [Brannigan, 2011] [Bordwell, 2005] [Genné, 2018] [Keating, 2019] [McLean, 2008], while others have built digital tools that enable productive methods of study through the application of computerized methods [Arnold et al., 2019] [Acland and Hoyt, 2016] [Flueckiger and Halter, 2018]. Substantial research outside of a cinematic context has explored the value of quantifying aspects of LBMS’s movement frameworks [Bernardet et al., 2019] [LaViers and Egerstedt, 2012] [Tsachor and Shafir, 2019] [Woolford, 2017] and of using computerized methods to study figure movement [Assaf et al., 2019] [Alemi et al., 2014] [Cuan et al., 2019] [Slaney et al., 2018]. However, this work rarely includes its movement data as part of the published output, reducing the reader’s ability to directly engage with the object of study’s dynamic nature or to evaluate the data generated by a performative research paradigm.

Our research method perceives the object of study as one in motion as well as in stasis, engaging with what Henri Bergson has called an "intuitive" knowledge of movement [Bergson, 2007] to better understand choreographic patterns and the myriad choices filmmakers make in staging the body for the camera. Carol-Lynne Moore sees close ties between Bergson’s categorizations of intellectual and intuitive perception and Rudolf Laban’s theories of Space from LBMS, which describes where the body moves in its environment. Whereas intellectual knowledge perceives movement as snapshots in time, an intuitive understanding perceives movement as a "flowing continuity fluctuating endlessly" [Moore, 2009, 84]. This approach to perception has phenomenological implications as well. Our ability to study past traceforms of movement through the mv tool’s historical traceform filter evokes the sort of "thickness" that Maurice Merleau-Ponty theorizes in his evocation of the accumulation of perceptual content [Merleau-Ponty, 2002, 321]. Embodying the movement is a key part of our analytical process; the videographic content presented in this article serves both as a record of the movement recreations and as a component of the research output.

It can further be argued that intuitive understanding is also an implicit aspect of videographic criticism, which allows practitioners to "enter into" their object of study [Bergson, 2007, 130], in this case an audiovisual or filmic form. As Grant elaborates, in videographic criticism the researcher engages with the logic of the medium on its own terms in a way that encourages experimentation and play [Grant, 2016]. Jason Mittell similarly points to the transformative value of entering moving images through nonlinear editing, arguing, "Even if you don’t make something new with the sounds and images imported into the editing platform, you can still discover something new by exploring a film via this new interface" [Mittell, 2019, 226]. Rather than working from existing audiovisual content, we posit that the act of recreating the movement phrases leads to a deeper understanding of movement patterns and, in the case of the filmic examples, of the formal practices that led to their creation.

Haseman positions this kind of practice-led approach as central to a performative paradigm, but he does not discuss the potential of considering choices and constraints as a part of such an approach. All creative practices are affected by constraints or obstacles of various kinds at both the macro and micro level of production: a director makes storytelling and aesthetic choices based on the budget available to them; a cinematographer chooses how to place and move the camera based on available technology; a choreographer can only work in a particular dance style (jazz, for example) if the surrounding cultural conditions have encouraged professional dancers to also train in that form. Filmmakers are constantly using craft practices, existing models and experience, and trial and error to guide their decision-making. As a result, stylistic history can benefit from what David Bordwell calls a problem-solution model of understanding the choices filmmakers make by envisioning stylistic history as a "network of problems and solutions" [Bordwell, 1997, 149].

A practice-led approach, through the physical and intellectual act of recreation, encourages the analyst to engage with a similar level of material specificity. Indeed, as Bordwell argues, adopting a problem-solution model to understanding film history has recreation of practical details as an inherent aspect of its approach; the task "is one of reconstruction. On the basis of surviving films and other documents, the historian reconstructs a choice situation" [Bordwell, 1997, 156]. Through a performative approach, the act is one of recreation rather than reconstruction. By directly engaging with the process, we end up with a tangible creation, informed by careful study of the original, that was subject to its own series of problems and solutions. The video and motion-capture recreations of LBMS scales and dance numbers published here, in turn, communicate research findings that complement our written analysis of those same movement examples. For example, in "No Strings (Reprise)" from Top Hat there is a movement where Fred Astaire shifts his weight into a chair and ostensibly falls asleep. In practicing the sequence for recreation with our dancers, we discovered that using a chair for the ending led to a more constrained and cautious movement quality, due to the light chair creating an unstable situation as the movers shifted weight onto it. Choreographing a graceful final posture for Astaire’s number resulted in a series of choices, perhaps an adjustment to the choreography, a change of furniture so that a lower armchair would permit Astaire to better counterbalance into it, or the addition of the carpet which mitigates most of the instability. We similarly needed to find solutions to the problem, but in our case, different furniture or securing the chair to the floor was not an option (i.e. the obvious solution of nailing the chair in place would certainly have infuriated the building owner and prevented our ability to continue working in the studio). Since our greatest priority was maintaining the spinning choreography, the feeling of lightness, and the increasing drowsiness of Astaire’s character, we chose to have the dancers shift into a final resting pose on the floor instead, which allowed us to more effectively maintain the movement qualities and the long line of the body seen in the final pose.

Methodologically, by conducting research in an audiovisual medium, our work will lead us to make choices and confront obstacles in parallel to the films we were recreating.[1] Catherine Grant touches on this issue in referencing concerns from the art-scholar Barbara Bolt, articulating that "following Haseman, the problem for the 'performative’ (or creative) academic researcher can lie in recognising and mapping the effects or 'transformations’ that have occurred in their practice-research" [Grant, 2016, 14]. In our work, we choose to approach this problem as an opportunity. Incorporating an acknowledgment and understanding of constraints and choices into the performative research method leads to a more rigorous process and, as we found with our case studies, to new and valuable insights. It also helped us refine our understanding of the Kinect’s technological advantages and limitations in motion capture.

Our results were affected by the resources and technology available to us, and we made deliberate, informed decisions based on those practical constraints. In choosing to work primarily with motion-capture data over video, however, we are able to move beyond the gravitational and physical constraints of reality, and our research tool is specifically designed to modify the rendering of the captured data, allowing for experimentation with the relationship between figure and frame.

The data collection module is a motion capture application that utilizes remote sensors to extract abstract movement data from users. Movement data is captured using the Microsoft Kinect V2, which we chose for its relative robustness-to-cost ratio, its portability, and its lightweight network protocol coupling capability, principally with Open Sound Control (OSC). The Kinect uses infrared cameras to generate depth maps, which are then internally transformed into a variety of different generalized representations, including a 25-joint skeleton frame. Using custom developed software applications [Junokas et al., 2018], users can access the Kinect’s application programming interface and construct a 3-by-25 skeleton sensor array for each body captured by the Kinect (i.e. the three spatial dimensions along horizontal/x, vertical/y, and sagittal/z planes for each of the 25 joints).

To increase resolution and/or range, multiple Kinect data collection stations can be added to the system, generating independent skeleton sensor arrays using the same module articulated above but in multiple instances. These arrays are then sent to the next module for data analysis and preprocessing (Figure 1).

The skeleton sensor arrays are sent from the data collection module into the data analysis and preprocessing module, where they are prepared for the next series of modules within the system. The analysis and preprocessing is done with Max/MSP/Jitter [Puckette, 2018], which we chose due to its signal-processing based computational design, native support for real-time interaction, compatibility with the chosen protocol (i.e. OSC), and the ability to easily create object oriented visualization filters.

The multiple skeleton sensor arrays from the Kinects can be synced at a constant frame rate, combining the multiple data streams into one representative skeleton frame, transforming them to generate the highest resolution and largest sensor capture range possible from the input skeleton sensor arrays. For this research, while three Kinects were used, comparable resolutions were achieved using the data from one Kinect. This was largely due to the asynchronous frame rate of data capture across the three Kinects, leading to a non-unified temporal dataset that could not simply be made into a higher resolution (e.g. a frame missing from one Kinect would not necessarily be at the same frame as the other Kinects, making corrective interpolation or substitution substantially more difficult). Due to this, the ultimate skeleton frame was composed from a singular sensor’s data capture.

The skeleton frame is then parsed into respective body arrays for individual, positional mover analysis, taking the first and second derivative of the skeleton joint positions, generating an approximate measure of velocity and acceleration. The magnitude of these derivatives is calculated, generating an approximate measure of speed and the magnitude of acceleration. Using these physical representations, kinematic limitations can be placed on the skeleton frame, setting programmatic bounds modeled on physically possible human movement. This reduces jitter and discontinuity errors between frames, making a more realistic physical representation for the digital data stream.

From these bounded positions, kinematic measures and relative positions are measured and collected, providing an extension to the sensor’s positioning. These relative positions measure the dimensional difference of each joint from every other joint. On top of these physically limited relative positions, infinite impulse response filters (with dynamic coefficients that can be user defined) can be applied, weighting the current frame’s position by the previous frame’s position, creating a type of smoothing on the positions, which dampens jerky frame-to-frame movements, resulting in the preprocessed skeleton array, the main source of data for the rest of the system (Figure 2). In this research, the mv tool uses the preprocessed skeleton array to drive the visualization of mover and the relative camera position.

The mv tool visualization principally serves as an abstract joint skeleton representation that shows the subject’s movement in relationship to a virtual camera position. Purposefully eliminating other environmental contexts, this abstracted view focuses on what is being captured by the system, ultimately informing the direction of our research. Additionally, the mv tool allows for visual filters to be placed on the abstract skeleton, highlighting several opportunities for further analysis and understanding. For the purposes of this work, we apply three filters to provoke deeper research: dynamic camera positioning based on joint movement, historical joint traceforms, and spatial color maps.

The mv tool has the capability to dynamically change the position of the virtual camera in the digital space, operating independently or coupled with the subject it is "filming." While users can manually control the three-dimensional digital camera position in the digital space, the camera movement can be tied to track a specific joint of the subject, moving in relation to the subject figure’s movement. For example, the camera can track the pelvic joints of the subject, moving in parallel or in counterpoint to the joint, allowing the subject’s pelvis to control the position of the camera as it moves (Figure 3).

A historical joint trail filter can also be applied to the skeleton figure, leaving a dynamic history of the subject’s movement in the scene. The persistence of the traceforms in the scene can be manipulated by the user, ranging from no historical traceforms to the entire temporal record of the captured movement. This provides the user with a visual "history," giving better insight into the paths and patterns of their movement (Figure 4).

Spatial color maps can also be added to the skeleton joints, providing insight into the trajectory of a given joint’s movement from an anchored point (Figure 5). For example, the trajectory of the right wrist from the spine mid can be colored in relation to its proximity to a three dimensional unit projection from the spine mid, providing the mover with a color map relating which of twenty-seven unit projections their movement aligns closest with (i.e. the complete set of permutation that can be made using only 1 and 0 in 3D).

The ability to manipulate recorded movement data in these ways allows the analyst to draw attention to different aspects of the body’s movement in space. These adjustments create a "poeticized quantification" of data that resembles videographic deformation methods articulated by Jason Mittell, which he argues can help scholars formulate new observations about an art object’s form [Mittell, 2019]. By algorithmically changing our visualizations of human movement, we can amplify certain observable patterns, posit hypothetical alternatives (in the relationship between camera and figure, especially), or simply marvel at the expansive complexities of the human body.

Top Hat is directed by Marc Sandrich, photographed by David Abel, and choreographed collaboratively by Fred Astaire and Hermes Pan. It was the first film for which Pan served as dance director [Franceschina, 2012, 60]. Pan, not Sandrich, likely worked with Abel on the camerawork for the dance numbers and had decision-making power over the integration of the figure movement into the cinematic space. Constantine, a writer for Dance Magazine who interviewed Pan in 1945, suggests that the dance director’s job is to “focus the camera on the most important part of the choreography and swing the camera in rhythm with the dancers” [Franceschina, 2012, 60]. Patrick Keating discusses this relationship with more detail in his discussion of the “No Strings (I’m Fancy Free)” number, in which “the camerawork remains completely subordinate to Jerry’s [Astaire’s] movements. When Jerry dances behind some chairs, the camera dollies in to follow; when Jerry spins to the right, a pan keeps him in frame” [Keating, 2019, 85]. Astaire had a reputation for remaining closely involved in all the stylistic elements of his dances and for wanting the camera to be an "involved but unobtrusive spectator" [Mueller, 1985, 26], a "subservient" form of camerawork that was nonetheless the result of a complex choreographic process [Keating, 2019, 85].

This reprise occurs very soon in the plot after Jerry Mulligan’s (Astaire) "No Strings (I’m Fancy Free)," in which his enthusiastic tap dancing awakens and irritates Gale Tremont (Ginger Rogers). Immediately smitten with Gale, he performs a soft-shoe "sand-man" version of the number to help lull her back to sleep, as Edward Everett Horton’s character (Horace Hardwick) observes him. Jerry’s motivation, therefore, is to perform a soothing number that stylistically contrasts its bombastic predecessor, but his movement quality is also the result of his newly discovered feelings for Gale. In preparing the recreation, we focused on the sense of lift in Astaire’s physicality (Light Weight), particularly in the upper body, paired with a core stability evoking his ballroom training that results in him skimming the surface of the floor. MacMaster and Mininsohn also embody the deliberate weight shifting steps coming from a tap dance vocabulary that move him through both swooping lateral staging changes and circular pathways. In our recreations, MacMaster performed in tap shoes and emphasized the formal steps in her learning process given that she has tap dance training; Mininsohn, drawing on her improvisation training, performed barefoot and prioritized the movement qualities and their relationship to narrative motivations (Figure 14, Figure 15, Figure 16, and Figure 17).

The choreography for this number involves numerous turns and pivots that move the figure away from a frontal facing and an ending weight shift into a chair which, as discussed above, our dancers performed into the ground. Both proved difficult for the Kinect to render properly; the latter results in some glitching rather than a clean line of the body, and the rotations are more difficult to perceive in the abstracted space. This points to the Kinect’s design for home gaming use, which is adapted for standing (or sitting), front-facing postures without much simultaneous movement of the limbs across the midline. However, adding the historical traceforms to Mininsohn’s recorded skeleton allows us to perceive the graceful pathways of Astaire’s movement, the Free Flowing movement that skims the surface, and the choreographic emphasis on repeated pathways and continuous motion (Figure 18).

We also learn from a closer analysis of the "No Strings (Reprise)" number that one of the camera movements during the dancing is the result of Horton’s, as well as Astaire’s, movement. Jerry’s dancing in the beginning of the number moves him repeatedly through the horizontal space of the frame, but it isn’t until Horace walks rightward towards the couch as Jerry also slides right in front of him that the camera pans slightly right to follow, ensuring that Horace remains fully visible as he sits on the couch. The camera remains static until the end of the dance, even though Astaire’s leftward pivot turns in the middle of the dance briefly cause his arms to go out of frame. By adding our mobile frame filter to keep the skeleton centered in the frame, we can visualize what Astaire’s phrase would have looked like had the camera followed all of his lateral shifts; here the body element maintained in the center of the frame is the skeleton’s pelvis (Figure 19).

This goes against the logical assumption that the dancers on-screen would dictate figure-motivated camera movement, as both Constantine and Keating articulate. Yet such a choice on Pan and Abel’s part does not necessarily contradict this rationale. The lack of reframing to the left ensures that Horace remains visible on the right and reinforces that the number is serving a crucial narrative purpose, one that is guiding the filmmakers’ stylistic decisions as much as the impulse to clearly capture Astaire’s artistry on-screen. Having Jerry perform the number is not only for the sake of aesthetic excess (as a demonstration of Astaire’s physical prowess) but also to make amends for disturbing Gale. In the process of the number, Horace also watches the soothing dancing and is lulled to sleep before Jerry succumbs to sleep himself. Keeping Horace in frame during the dancing allows us to see that he is watching Jerry dance, and a later cut to a closer framing of him looking tired cues us to watch the background as Jerry continues to dance, where we can observe Horace put his head down on the couch. Even in the earlier "No Strings" number, the choreographic and cinematographic decisions are made to either keep Astaire dancing close enough to Horton so that the latter remains in frame or have Astaire’s dancing carry him far enough through lateral space that Horton does not awkwardly reappear at unexpected moments in the background. Ultimately, the camera helps to emphasize the most important function of the choreography: narrative comprehension. Recreating and manipulating the movement for the purposes of formal analysis enhances our, and hopefully the reader’s, ability to observe these choreographic and cinematic patterns.

Beau travail, loosely adapted from Herman Melville’s Billy Budd, is directed by Claire Denis and photographed by Denis’ long-time collaborator Agnès Godard. The film’s narrative portrays much choreographed movement in its focus on French Foreign Legion soldiers stationed in Djibouti, and choreographer Bernardo Montet’s creative contributions were essential to the film’s preproduction design [Mayne, 2005, 93]. Beau travail includes several dance scenes between local Djibouti women and the Legionnaires but also emphasizes performances of military and domestic exercises, blurring the line between dance, military calisthenics, and pedestrian movement. The film relies little on dialogue or explicit narration, forcing the viewer to read the implicit meaning of the figure movement; Judith Mayne describes the film as a "kind of choreographed ritual" [Mayne, 2005, 93]. The protagonist Galoup’s final solo dance occurs after the character’s presumed suicide, which allows the viewer to read the motivation of that final dance in numerous ways: as a frenetic but ultimately futile attempt at escape [Mayne, 2005, 101], as an example of the post-colonial body [Hayward, 2002], as a marker of queer displacement [Sosa, 2011]. Denis viewed the scene as an attempt at freedom; while initially planning to include the number before the suicide, she changed the plot order to "give the sense that Galoup could escape himself" but also because after filming the number she realized it was the stronger ending [Darke, 2000, 18]. Our close analysis through recreation of the film’s final dance scene is equally revelatory in its emphasis on the explicit diegetic details present and the craft mechanics of the scene.

Montet was closely involved with Beau travail’s narrative development in preproduction, but his influence on Galoup’s final dance sequence, performed by Denis Lavant, is unclear. Denis recalls the number’s process in an interview with Senses of Cinema:

But we never rehearsed the dance scene at the end of Beau Travail. I told him [Denis Lavant] it’s the dance between life and death. It was written like that in the script, and he said, 'What do you mean by "the dance between life and death"?’ So, I let him hear that great disco music [laughs], and he said, 'This is it.’ So, we didn’t need to rehearse. . . . He said, 'You don’t want us to fix some of it?’ I thought it was better to keep the energy inside, because if we started fixing some stuff then we would have made many takes. And we made one take. But he was exhausted at the end" [Hughes, 2009].

Lavant already had extensive movement training at the time of filming, with a background in circus and pantomime, so it is plausible that he created the number without Montet’s assistance. Denis’ comment about a lack of rehearsal and a single take seems suspect but indicates that an improvised approach to avoid an over-rehearsed look was key to the process. Most accounts of this number describe the dancing as frenetic, but upon closer analysis we can observe that Lavant’s movement is in fact quite graceful, with increasingly controlled and complex movements originating from the body center (the result of core stability and alternating Bound and Free Flow) and reverberating into the limbs. Like Astaire’s movement, much of Lavant’s movement patterns are driven by an upward impulse and a feeling of lift, but with a greater sensation of strength (Strong Weight) in contrast to Astaire’s lightness. Unlike the fluidity and ongoing nature of the "No Strings" number, Lavant frequently starts and stops. The spatial relationship of camera and figure also creates an off-kilter feeling. Rather than film Lavant straight-on, the camera remains to his left (likely to avoid revealing the camera in the mirrored wall). His gaze and facing frequently focuses leftward as well, as if he is intrigued by something off-screen left that the viewer cannot see. In recreating this number, MacMaster and Mininsohn focused on this sense of increasing and halting range of motion, as if in the afterlife Galoup is discovering an ability to fully express himself for the first time. They embodied the specific pathways and gestures Galoup employs in the first half of the number (including his smoking) but chose to improvise the more expansive movement of the second half (Figure 20, Figure 21, Figure 22, and Figure 23).

Denis’ film does not fall within the confines of the musical genre. But the space of Galoup’s final dance feels more liminal or beyond the limits of the established diegesis the way many musical sequences would, an impression that comes from the disparate stylistic choices of staging, choreography, cinematography, and sound. Unlike the earlier club scenes, where we hear French Creole and Turkish popular music, Galoup here performs to Corona’s Eurodance hit, "Rhythm of the Night." Previously we see Legionnaires and local Djibouti women dancing, but now Galoup is alone (with his mirror self). As the narrative progresses, the camera allows us to see more of the movement by adopting more distant framings. The early disco scene is shot at close proximity, allowing us to see the men and women dancing together in medium close-ups. The second disco scene in which the women dance as Galoup watches shows them in a medium long shot. It is only during Galoup’s final number where we see him dance in a full long shot, and even then we frequently lose his extremities as they extend beyond the edges of the frame. Adding historical traceforms allows us to perceive the increasing expansiveness of Galoup’s movement and release, especially after his leap to the ground (2:08 in Figure 24's video).

Godard’s camera in Beau travail’s final number never dollies through the space, but she frequently pans and tilts to holistically follow Lavant’s movement through the small space of the disco. A visualization of the movement with the addition of a parallel mobile camera (that reframes along the horizontal, sagittal, and vertical planes) takes this follow aesthetic to the extreme (Figure 25).

A perfectly aligned, parallel camera removes the feeling of spontaneity from the number. While Godard’s cinematography calmly and subtly reframes as Lavant moves through the space, parts of Lavant’s body occasionally move out of frame, specifically his head during sudden leaps, imbuing him with a sense of freedom and release. If Lavant were always centered, the expansiveness of his movements would potentially be diminished by the parallel movements of the camera, especially as the number progresses. In seeing a perfectly centered version of the dance, we also are reminded that Lavant is not truly alone in the number, as Godard keeps his mirrored image in frame throughout, balancing the space between Lavant and his reflection. Like "No Strings (Reprise)," Galoup’s final solo number results in a composition that reminds the viewer of the importance of both figures through deliberate choreography and mobile framing.