Abstract
The government of the United Kingdom is offering significant amounts of funding to
increase engagement between researchers and technology companies involved in the
“creative economy”, a sector worth £100b per annum. The sector has naturally
evolved into clusters involving actors as diverse as the BBC, the Victoria and Albert
Museum, and freelance game designers. It collaborates and competes with multinational
powerhouses including Google, Microsoft, and Pixar. The imbrication of academic
researchers in the sector makes it a good example of the extension of academia to
industry, raising ethical but also methodological and practical questions. This
article describes a project that embedded a research software engineer (RSE) from a
digital humanities (DH) laboratory into a small technology start-up engaged in high
risk R&D of an immersive narrative story-telling platform. The platform is
enabled by artificial intelligence and has the capacity to remember user input and
modify narrative options and character mood accordingly. The team included a
researcher specialising in narrative theory and digital production. The project
demonstrated the utility of the critical application of DH methods and the need to
develop career pipelines to produce people capable of working at the intersection of
higher education and industry.
Introduction
New entertainment technologies are creating technical, intellectual, and HR
challenges. The problems extend from issues of technology integration to enabling and
critiquing hybrid forms of expression that defy categorisation, and finding employees
with the skills needed to thrive in the industry. Products powered by artificial
intelligence, natural language processing, and immersive technologies (AR, VR,
chatbots, games, interactive story dramas, and affective systems) are technically
complex to design and build, and require diverse teams including writers, designers,
developers, artists, and project managers [
McDuff and Czerwinski 2018].
New multimedia and transmedia design and engineering paradigms need to be found to
foster innovation, and new career paths and skillsets are needed to enable creative
production. Without solutions to these problems the United Kingdom’s £100bn
investment in creative industries will stand little chance of success. To compete,
small to medium enterprises (SME) are engaging in high-risk R&D to find new ways
to build talent and manage production. They need the support of research software
engineering (RSE) and academic partners to ensure technical, creative, and
intellectual quality. Digital humanities teams have an important part to play in this
process.
These issues have become a focus of the UK government’s research and funding policy,
coupled to a wider industrial strategy designed to retain the country’s global
competitiveness. The problem domain provides a classic example of Henry Etzkowitz and
Loet Leydesdorff's “triple helix” of industry-government-university
collaboration ([
Etzkowitz and Leydesdorff 1995]; [
Etzkowitz et al. 1998]) and an opportunity to explore the potential of
applied methods in Digital Humanities (DH). This article describes one such
experiment, demonstrating theories and methods that can facilitate the application of
DH perspectives to the creative industries and (by extension) the skills and career
pathways needed to produce people capable of delivering such work.
Section I describes the project at the heart of the paper, “Applying AI to Storytelling — bringing computational research into the creative
industries” (Applying AI), a collaboration between a London technology
start-up specialising in immersive narrative storytelling (Charisma
Entertainment), and researchers and engineers at King’s College London.
Applying AI offers important insights into the challenges of producing non-linear and
affective technological systems and narrative content that aspire to world-building
and intensive human-computer interaction (HCI). Section II explains why this project
is best understood theoretically in terms of applied or “interested methods”
[
Asdal 2018], and its relationship to global trends in academic
research. Section III provides socio-technical, economic, and theoretical context for
the experiment, to position it within the United Kingdom’s funding and policy
environment and frame it with relevant scholarly literature. Section IV outlines
potential new production pipelines capable of enabling the production of immersive
products, and the need to develop people with blended creative and technical skills,
and Section V describes the challenges of knowledge transfer and pedagogy in emergent
convergent zones such as the UK’s creative economy. In taking this ambitiously
holistic approach, bringing a broad conceptual apparatus to bear on the problem of
immersive engineering in DH, we hope to open multiple avenues simultaneously and
prompt conversations about future directions in research, engineering, pedagogy, and
career development.
Section I: Applying AI to Storytelling — bringing computational research into the
creative industries
The project “Applying AI to Storytelling — bringing computational
research into the creative industries” (Applying AI) brought together two
streams of inquiry. The first was centred around embedding a DH RSE from King’s
Digital Lab (KDL) within a creative industry SME (Charisma
Entertainment), to function as a connection point between industry and
academia. The aim was to facilitate traffic in ideas and methods between the creative
industries, RSEs, and academics, and to overcome cultural and institutional barriers
that inhibit collaboration. It also sought to develop career pathways and skills for
RSEs to help them work effectively with industry through embedded co-creation,
understanding the culture and working practices of a start-up and how they align with
RSE practice. This work was led by James Smithies of KDL, with Elliott Hall as the
KDL RSE who would embed with the SME. A workshop on RSE/Industry collaboration was
held midway through the project, bringing together representatives from higher
education, the creative industries, and related fields to disseminate the project’s
work on collaboration and to generate feedback on how it could be taken forward in
the future.
The second stream of research was led by Sarah Atkinson and centred around evolving
new working principles, practices, and workflows in the creation of interactive and
immersive narratives and the development of the charisma.AI interactive media
platform. This work package was the primary reason Charisma
Entertainment approached King’s College London for assistance. The
optimisation and refinement of the machine learning algorithms that powered their
charisma.AI platform was essential in creating compelling and rounded human
characters and convincing character interactions between the AI and the audience, and
(crucially) AI-character memory. This raised complex issues relating to the creative
writing process. The work package involved an initial workshop for new media authors,
writers, and narrative designers to adapt and adopt their established working
practices for an AI environment where character memory was of profound importance.
Charisma Entertainment needed the narratives generated for
charisma.AI to be subjected to critical examination in both literary and social
science terms, identifying how conscious and unconscious bias informs the author’s
conception of the character and the player’s responses to it.
A second workshop involved audience and user testing, enabling feedback and responses
to be provided to the creative team. The initial shape of the collaboration was
designed for Hall to take on a facilitation role during production of the project’s
showcase demo. He was to use his skills as a published novelist and RSE to enhance a
collaboration between Charisma Entertainment and Parabolic Theatre, a
performance theatre company known in London for their immersive show
For King and Country. The aim was to adapt the show into a
Unity-based game that would use charisma.AI for all character interactions.
For King and Country was chosen as a demo because it would
stretch the capabilities of the charisma.AI platform. An example of a relatively new
genre of theatre, associated with a broader trend towards immersive experiences [
Atkinson and Kennedy 2018], the show has proven to be a popular experience
in London. Its narrative is compelling: Germany is winning World War Two and about to
invade the UK. The audience meets in a rump parliament, then splits off to work in
the various ministries before coming back to parliament to hear the results of their
work: has the UK been defeated, or has their work repulsed the invasion? The aim was
to create a game demo that would replace the actors and facilitators with AI
characters, and for audiences to interact as avatars in the game. The characters
would interact with the audiences through text-based conversation and (of fundamental
importance) remember and sometimes misremember what they had said. The AI characters
had been programmed so that they could also lie and remember when they were lied to.
In many ways, charisma.AI is the latest iteration of a long-standing tradition in
electronic and interactive literature, which spanned chatbots such as Eliza (1964 -
1966), early iconic text-based games known as Multi-User Dungeons (MUDS), and
hypertext fiction that once held great promise for the development of new modes of
narrative experience. Such products were related to the well-known
Twistaplot and
Choose Your Own
Adventure children’s books of the 1980s, and culturally related to the
popular
Dungeons & Dragons board game. Works like
Michael Joyce’s
Afternoon, a story (1987), built using
Eastgate Systems’ Storyspace tool, were claimed by some critics to augur a new era of
“ergodic” literature. In 1997 Espen Aarseth claimed such “hypertext literature” produced
a short circuit between signifier and signified, a suspension
of differance [accent on the e] that projects an objective level
beyond the text, a primary metaphysical structure that generates both textual
sign and our understanding of it, rather than the other way around. [Aarseth 1997]
The belief was that new technologies were radically transforming the
experience of reading from a passive to an (inter)active experience. 1990s critics
noted similarities between ergodic literature and elements of postmodern fiction such
as Barthes labyrinth, itself reaching back to Greek myth [
Aarseth 1997].
As Dave Ciccoricco has pointed out, these kinds of narrative in fact implement a
range of different “networked” narratives, from simple
branching to more complex tree-like structures [
Ciccoricco 2007]. The
genre has never quite broken through into widespread mainstream acceptance, but there
has been a steady stream of innovation over the years and recent technological
developments have prompted creative renewal. In 2005 the video game Façade
illustrated the potential of generative AI narratives. In 2018 Netflix produced the
interactive branching narrative show
Bandersnatch (to
mixed reviews) and the same year the live streamed show
Artificial was released by Twitch, winning an Emmy for Innovation in
Interactive Media.
Artificial enabled direct interaction
with the audience, including votes to fundamentally alter the plot and even remove
characters, suggesting something of the potential of the new generation of
technology. Charisma Entertainment are contributing to this same history
of creative and technological development with charisma.AI, hoping for simultaneous
breakthroughs in technological and creative development.
High quality academic critique is essential to this process. Interest in ludology
(the study of games not only as cultural artefacts but as expressions of human
identity and phenomenological experience) based on Johan Huizinga’s 1938 book
Homo Ludens
[
Huizinga 1938] ground criticism of early multimedia products in a
philosophically serious tradition and suggested that multimedia and transmedia
products resist “narrowly instrumental analysis” and are
properly conceived as both a foundational human experience and art [
Rodriguez 2006], but criticism of electronic literature waned when
narrative quality failed to attain the expected heights. The ideas that had been
generated were enough to feed new modes of critical interpretation that remain
relevant, however, including Donna Haraway’s articulation of a “cyborg manifesto”
[
Haraway 1991] and Katherine Hayles’ notion of “intermediation” that suggests an electronic text “performs actions that bind together author and program, player and computer, into
a complex system characterized by intermediating dynamics”
[
Hayles 2008]. The wider tradition can be usefully read through these
critical lenses, and beyond to articulations of “imagineering” and immersion [
Demetriou 2018], and
transhumanist [
Chu 2014] philosophies that emphasise the blending of
human and machine. There is widespread acceptance that adding aesthetic experience to
human-computer interaction heightens phenomenological entanglement.
The team at Charisma Entertainment were aware of the creative and
critical tradition their charisma.AI platform was part of but they were primarily
responding to the narrative potential inherent in emerging technologies. In the case
of charisma.AI that related to character memory, and the ability of AI characters to
“remember” and “misremember” information (such as a player’s name, but
also narrative decisions and other aspects of gameplay) relayed to them by players.
The charisma.AI system has been developed to connect with external services such as
weather reports, via Application Programming Interfaces (API), and alter the gameplay
accordingly (for example, mirroring the weather outside the player’s physical house
in the virtual environment).
The For King and Country demo included additional
challenging requirements, requiring the team to create several different kinds of
character-to-audience conversations: one-on-one between players, ministers, and other
heads of state (for example the American President); a version of a parliamentary
debate where the player was part of conversations with several AI characters, and
would need to influence them to vote in their favour; and multi-character,
free-flowing conversational exchanges which would be very different to the one-on-one
conversations from other charisma.AI projects. In one key speech, the player, acting
as Prime Minister, tries to sway the AI characters to their point of view, creating a
range of technical and creative challenges: in one scenario, for example, AI
characters do not respond to the player’s speech if they have not interacted with
them or have had negative interactions with them during the preceding narrative. It
is worth noting that before the project started, charisma.AI’s natural language
processing (NLP) could only parse short, declarative sentences, not the multiple
sequential sentences that would be required for a convincing and compelling speech.
The creative implications of this narrative complexity were commented upon by writers
attending the writer’s workshop at the start of the project. The group had a wide
range of experience writing for the television and gaming industries (working on high
profile productions including the BBC’s Sherlock series)
but were struck by the cascading complexity that accrues from the ability of
AI-enabled characters to remember and – more problematically – to strategically
misremember information. To simplify the problem, the workshop participants thought
through the implications of a character in a basic whodunnit narrative being able to
remember or misremember player information. When should the writer save a piece of
information, and when should they release it (at a dinner party with everyone
present, in a private exchange between two characters)? How would they write a
narrative that included one thread where a character faithfully remembered a piece of
information, but in other contexts duplicitously misremembered it? How could writers
control the exponential layers of complexity generated by such scenarios, and how
could a writing interface be designed to ensure writers could keep track of the
various narrative permutations? The narrative complexity that could be facilitated by
charisma.AI is significantly increased by the ability of the writers to define and
assign moods to characters, which can act as additional narrative gateways to new
scenarios: say the wrong thing or ignore an important character, and they could
choose to misremember rather than remember information conveyed at an earlier stage
in the narrative.
Revealingly, given the analysis about skills requirements that we will detail below,
Hall’s role changed significantly as the project progressed. The lead writer for
Parabolic Theatre was unable to contribute to the project as intended due to
scheduling conflicts. Hall took over primary writing duties for the For King and Country demo, as well as designing new logical
structures inside charisma.AI to adapt its workflow for use as a plugin inside Unity,
while working with the Charisma Entertainment team on the demo’s game
flow. He continued in these roles when the project’s focus moved to a new demo, an
adaptation of John Wyndham’s The Kraken Wakes. This
evolving collaboration led to several meaningful outcomes: two functional showcase
demos - For King and Country, and The Kraken Wakes - were written by Hall. The deeper collaboration on the
two demos created a test case for how a DH research software engineer (RSE) could be
embedded within a commercial development team, providing both technical and creative
input. The changing priorities and roles during the project provided an important
test of institutional flexibility and a model for effective collaboration between
higher education and the creative industries.
Section II: Pure versus Applied Research
Applying AI therefore raises interesting questions about the nature of applied DH
research. The project was initiated in conversations between Charisma
Entertainment and King’s College London but was oriented towards answering
problems for the industry partner, rather than research questions developed
independently in the university. This is, of course, the essence of “applied”
research in the formal Science and Technology Studies (STS) sense of the term:
industry partners (whether in the creative industries or other industrial sectors)
seek out research teams who can help them navigate an impasse in their R&D or
product development process, and research questions are designed with the sole
purpose of resolving that impasse. Research is “applied” to “real world”
problems, to develop new products or services for the marketplace. The fact
Charisma Entertainment operates in the creative marketplace is
incidental to this more reductive point. In this telling, applied research is opposed
to “pure” research, which originates in the laboratories and offices of
researchers free from outside influence. Pure (or “basic” or “fundamental”)
research aims to build theories or produce knowledge but often has no practical or
clinical application: it represents research in the sense implied by the Oxford
English Dictionary’s quotation of E. Grimeston tr. J. de Acosta in 1604:
Whether these two great gulphes do ioyne in any other part of the
world, which was the enterprize of Fernando Magellan a Portugall gentleman, whose
great courage and constancie in the research of this subject, and happy successe
in the finding thereof, gave the name of etenall memory to this straight (O.E.D.
Online).
Pure research is designed to fill in empty space on the map of
knowledge, not to resolve practical problems. In this definition of the term, digital
humanists do not engage in nearly as much “applied” research as they might
think. Technical effort generally aims to fill gaps in the map of knowledge, not to
build new products and services or solve problems encountered in the corporate or
government worlds.
And yet, as many readers will have surmised, the distinction established in STS
between pure and applied research is anything but straight-forward. Indeed, as Zoe
Nyssa [
Nyssa 2019b] has pointed out, STS scholars have built a
discipline out of exploring and ultimately collapsing the distinction, variously
noting the rather fine semantic distinction between knowledge sought to “fill in
blank spaces on the map” and knowledge sought to increase our ability to
manipulate or intervene in the known world: the cultural and politically mediated
[
Knorr-Cetina 1981] aspects of both modes of research, the inherently
messy “mangle”
[
Pickering 1995] of researchers, tools and methods, or the way in which
objects and humans involved in the research process can be conceived (famously, by
Bruno Latour) as mere actants [
Latour and Woolgar 1979] in a continuous
process. Simplistic binary conceptions of applied versus pure research collapse under
even limited scrutiny: in theoretical terms, and as a way of understanding the
research endeavour in all its complexity, it provides the flimsiest of foundations.
The fact the OED entry does not make a distinction between the two modes is telling.
And yet the distinction retains utility in the context of the DH work described in
this article. While it is undoubtedly necessary to collapse the pure versus applied
distinction when exploring the epistemological and methodological depths of
laboratory science and DH projects designed to advance knowledge, it remains
conceptually useful in the context of projects designed to enable commercial sector
partners. Perhaps controversially, the distinction also sheds light on the
relationship between DH, neo-liberalism, and the corporate university, highlighting a
nascent area of DH practice that is presumably even more controversial than the
production of historical databases or use of machine learning algorithms [
Allington et al. 2016]: if the use of digital tools by humanists in the
service of general humanities scholarship is controversial, their use in the service
of commercial profit must surely be doubly so.
It is important to note in this context that, rather than being merely arguably
connected to the corporate university, applied DH (in the strict sense of applied
research intended by STS scholars) reflects hard commercial relationships between
universities and the commercial sector initiated by the United States Bayh-Dole Act
(1980). Bayh-Dole has been identified as a watershed moment by historians of higher
education, changing the social and economic contract between universities and funding
agencies and allowing them to pursue intellectual ownership of an invention
themselves instead of needing to assign it to the federal government. This opened the
research environment to market conditions, incentivising universities to define and
protect their intellectual property (IP) and monetise it through the development of
commercial ventures. Other governments (notably the United Kingdom, Australia, and
New Zealand) watched with interest and, over time, increasingly oriented their
policies to commercial concerns too. In simple terms, the Act was the legal
instrument in America (and the model in many other countries) by which “highly contentious”
[
Murray 2010] forms of neoliberalism were unleashed on the university
sector. As Allington et al notoriously implied in 2016 DH, whether the community is
comfortable with it or not, it is imbricated in this wider context [
Allington et al. 2016].
Section III: The Creative Economy
The Applying AI to Storytelling project occupied the intersection of these currents,
as much as being a straight-forward DH project. Significant amounts of funding have
been directed towards Etzkowitz and Leydesdorff’s “triple
helix” in the United Kingdom, characteristic of a “new
social contract between the university and larger society”
[
Etzkowitz and Leydesdorff 1995] and based on intensive knowledge
transfer and co-production across the full spectrum of research, development, and
production. Noting progressive changes in the orientation of universities to the
private sector since the 1980s, Etzkowitz and Leydesdorff’s 1995 paper claimed that
university-industry relations had “become a laboratory for the
conscious reshaping of…knowledge infrastructure”
[
Etzkowitz and Leydesdorff 1995] in a manner that was transforming “the role of the state in academia, the role of corporations in
innovation and of the university in the economy…”
[
Etzkowitz and Leydesdorff 1995]. In the United Kingdom, research
activities (often conceived as “applied” research) that encourage collaboration
between academia, industry, and government reflect this larger international
reorientation of knowledge production towards instrumentalist concerns.
And yet, the Applying AI project demonstrates why this activity is justifiable in
academic and creative as well as purely commercial terms. As is shown below, the
success of the project hinged on the proper functioning of an interdisciplinary team
involving technical developers, businesspeople, academics, and creative writers. The
contribution of the RSE developer and creative writer were particularly essential;
their lack of progress or unavailability was enough to halt the entire production
process. It is not so much that Etzkowitz and Leydesdorff’s triple helix pulls
otherwise purely creative writers and academics into the commercial world, in other
words, as it creates a new set of creative challenges that require inter-disciplinary
approaches to resolve. Projects such as Applying AI allow those new approaches to be
trialled, for the benefit of commercial and research partners alike, creating
significant creative and intellectual insights as well as new career paths. It is
unreasonable to assume that cross-sector co-creation unwittingly acts as a Trojan
horse for capture and the warping of HE priorities.
This has unfortunately become a natural assumption in some areas of humanities and DH
scholarship. It is difficult to be sure of the precise intellectual genealogy, but it
appears to be built on methodological positions established in STS by scholars such
as Knorr-Cetina, who view “knowledge as practices - within
structures, processes, and environments that make up specific epistemic
settings”
[
Knorr-Cetina 1999]. This created an excellent framework for holistic
analysis of scientific and technological initiatives but negative perceptions of
applied research built on such work too, culminating in the work of Sheila Slaughter
who coined the term
academic capitalism and suggested it exposes the
“neoliberal tendenc[y] to treat higher education policy as a
subset of economic policy”
[
Slaughter and Leslie 2001]. Another strain of literature built directly
on Etzkowitz and Leydesdorff’s 1995 paper, criticising its emphasis on linear modes
of knowledge transfer that assumes the progressive evolution of concepts from R&D
to production without an allowance for iterative cycles, co-production, and failure.
Many humanists find it difficult to see the possibility of creativity or intellectual
freedom in such models.
The conversation is becoming increasingly nuanced, however. Carayannis and Campbell’s
Mode 3 Innovation Ecosystem emphasizes more fluid interaction between participants,
and what the authors refer to as “a multi-layered, multi-modal,
multi-nodal and multi-lateral system, encompassing mutually complementary and
reinforcing innovation networks and knowledge clusters consisting of human and
intellectual capital, shaped by social capital and underpinned by financial
capital”
[
Carayannis and Campbell 2009]. Revealingly, Carayannis and Campbell’s
analysis adds a fourth aspect to the triple helix in the form of feedback from the
public and media culture, in a complex “fractal”
innovation system that is self-correcting and constantly evolving [
Carayannis and Campbell 2009]. Sarah Atkinson and Helen Kennedy use the
term
ecosystem in a similar way, noting the early adopter role of London
theatre companies in the production of Virtual Reality (VR) experiences [
Atkinson and Kennedy 2018]. However conceived, STS research has made it
abundantly clear that policy changes at national level and economic pressures at a
global level since World War Two have resulted in fundamental changes to the
operational context of academic research, creating opportunities for new modes of
creative production and collaboration.
In the United Kingdom this crystalized in 1997 in the policies of the New Labour
government led by Prime Minister Tony Blair, which established a Creative Industries
Task Force (CITF) within its newly established Department of Culture, Media, and
Sport (DCMS). This taskforce, and the resulting Creative Industries Mapping Document
(CIMD), has been widely credited with establishing the modern notion of the
“creative industries” in the UK and positioning it at the forefront of
national and international efforts to extract economic value from cultural activity.
As Terry Flew notes, the CIMD defined the creative industries as “those activities which have their origin in individual creativity, skill and
talent and which have the potential for wealth and job creation through the
generation and exploitation of intellectual property”
[
Flew 2012]. New Labour subsequently positioned the creative industries
at the heart of national identity, using the “then-new concept of
convergence to argue that the future of arts and media in Britain lay in a
transformation of dominant policy discourses towards a productive engagement with
digital technologies, to develop new possibilities for the alignment of British
creativity and intellectual capital with these new engines of economic
growth”
[
Flew 2012].
Applying AI to Storytelling provides an excellent example of this new model of
government/creative industry interaction, contributing to policy changes designed to
enable economic and cultural “clusters” the government identified as key drivers
of Carayannis and Campbell’s fractal innovation ecosystem. In a 2009 article the
authors identified three different kinds of clusters of “fractally connected” organizations dependent upon rich cross-sector
interactions and complex transfer and production of R&D activity: geographic,
sectoral, and knowledge clusters [
Carayannis and Campbell 2009]. These
clusters - often overlapping and almost always interacting in complex
cross-functional configurations - include a “pluralism of a
diversity of agents, actors and organisations: universities, small and
medium-sized enterprises and major corporations, arranged along the matrix of
fluid and heterogeneous innovation networks”
[
Carayannis and Campbell 2009]. They function as what military analysts
might term force multipliers, aggregating diverse economic and methodological
capabilities in either real-world or virtual configurations and creating the
conditions for exponential levels of creative and technical innovation and economic
growth.
The concept of clusters clearly resonated with UK policy makers, who are eager to
ensure the continued competitive advantage of the country’s creative economy they
feel has been “changed beyond recognition”
[
Mateos-Garcia et al. 2013] by internet-based technologies: a £100bn sector
worth more than the automotive, aerospace, life sciences, oil and gas industries
combined [
Choudhury 2019]. The sector has grown at more than twice the
rate of the wider economy since 2010 [
HM Government 2018a]. Significant
funding streams were directed at creative clusters
[1]
in 2018, with £80m allocated to fund nine creative industries clusters across the
country [
HM Government 2018b]. In a clear signal that the country
acknowledges the synergistic relationship between scholarly analysis and economic
production, an associated Policy and Evidence Centre was funded to work out how to
best measure and enable future initiatives. A further £33m was allocated to an
Audiences of the Future programme designed to enable cross-sector R&D resulting
in virtual and augmented reality (XR) products [
UKRI 2018].
The Applying AI project was funded by a related Innovate UK scheme, similarly aimed
at enhancing knowledge and skills transfer between elements of the triple helix [
Innovate UK 2018]. The Creative Clusters and Innovate UK programmes
amount to an overt attempt to extend the United Kingdom’s competitive advantage in
the creative industries, in support of a plethora of flagship institutions such as
the BBC, the Victoria and Albert Museum, the British Museum, the Tate Modern, the
Royal Opera House and in the context of emerging technologies that at once threaten
and promise to transform their potential and scope. Revealingly, Carayannis and
Campbell’s Mode 3 model extends the innovation environment from these traditional
organisations to myriad small, medium, and large technology start-ups alongside
established vendors, freelance designers, and gaming studios. The complexity is
overwhelming but forecast disruption caused by emerging technologies and a related
skills shortage has prompted remarkably directed action [
Innovate UK 2018].
Section IV: Production Models for Immersive Engineering
Any robust analysis of Applying AI needs to take this policy context into account.
The primary aim of the project was to enable the commercial goals of Charisma
Entertainment, who were grappling with issues endemic to the creative
industries, beset by increasingly complex design and engineering problems at the
intersection of art and technology. Hall’s work with the charisma.AI platform
reflects this challenge: his work took on recursive characteristics as he moved
between the role of writer, developer, and product consultant. Working with an
evolving technical product, he realised its support for branching narratives would
not be able to support the complex plot and character developments required for
For King and Country. Using computational logic and
working closely with the core development and creative teams at Charisma
Entertainment, he surmised that the interface could be stretched to
accommodate a more complex hub and spokes plot configuration. This made management of
the various story nodes difficult, so he hacked together a basic external technical
solution (something that could later be integrated into the main product) to keep
track of the narrative.
High quality production models for immersive engineering need to account for this
symbiotic relationship between creative artists and technologies, enabling creative
collaborations between artists and machines [
Sandry 2017]. A 2018 UK
report into production pipelines for immersive engineering noted companies were
experiencing difficulties across the entire spectrum, from budgeting to managing
small teams, consumer expectations, design, dynamic workflows, inadequate and fast
evolving toolsets, and a lack of people with appropriate skills and experience [
Opposable Group and TechSpark 2018]. Stuart Whyte, Director of VR product
development at Sony’s SIE London Studio, framed the issue in stark terms, noting that
[i]t’s so new. I think in traditional games we’ve taken many
generations and iterations of designs, around control schemes, and so on. The
challenge and excitement around VR is that it is a very new space and some of
those designs and languages from other games just don’t work so well in this
medium and yet there are alternate ways you can do it that are even better (Whyte
cited in [Opposable Group and TechSpark 2018]).
Charisma Entertainment experienced much the same pressures as other
start-ups building products for the creative industries during the Applying AI
project (tight budget, small team, competing demands) and their production pipeline
was similarly challenged. Hard choices had to be made between additional R&D or
creative effort and technical investment, and the ultimate decision to transition
from production of the
For King and Country demo to
The Kraken Wakes required resolution of complex
business, technical, and creative issues.
At a more granular level, tension between creative and technical expression in
Applying AI was generated from the characters in the charisma.AI system being
“audience aware”. This means that they could: react to the language, mood and
conversations of the players; “remember” previous interactions with a player;
access a knowledge base of that character’s interactions with other players; and use
this data to provide responses tailored to the user (implicitly in the way a chatbot
learns from interactions, and explicitly, in directly challenging users with their
shared history e.g., “yesterday you said this.”). While it
might seem premature to read phenomenological significance into such interactions,
even with the critical apparatus provided by critics such as Aarseth [
Aarseth 1997] and those that followed him, it is worth noting the deep
ramifications of memory for our understanding of not only human experience, but the
nature of intelligence and artificial intelligence (AI). As Gisela Bahr et al note,
“[h]uman long-term memory is [merely] an example of naturally
occurring big data”
[
Bahr et al. 2017]. As rudimentary as contemporary chatbot and AI
technology currently may be, and as minuscule as contemporary computing storage is
next to the exabyte capacity of the human brain, the underlying mechanisms at work -
especially in use cases as fundamental to experience and identity as narrative -
provide valuable insights into the boundaries between human and machine intelligence.
The study of memory and forgetting, and its implications for next-generation
computing systems that struggle to manage the vast quantities of data being funnelled
into them, is just one example of the strangely synergistic tensions emerging between
the creative industries and industry proper. Production models for immersive
engineering are likely to offer practical applications in a range of industrial
settings.
Section V: Knowledge Transfer and Talent Pipelines
Production pipeline issues were not satisfactorily resolved by the Applying AI
project, although that was a key focus of the grant application. Progress was made
identifying issues, and acknowledging the complexity of issues in play, but the
problem is a generational one. Hall’s hybrid role as technical consultant and writer
greatly increased the technical and creative capacity of the charisma.AI platform but
was unsatisfactory as a long-term solution. Demand for hybrid talent is strong enough
that Hall became unavailable for future phases of the project, and the Applying AI
project made no provision for passing his skills on to another RSE. That issue was,
of course, intractable in the short time available for the project. Hall’s skills
have been developed over the course of more than a decade working as an RSE in
digital humanities and creative writing.
Developing workers with Hall's blend of technical and creative skill is something
that would ideally begin at primary and secondary school and be laced throughout
university experience as well. It may be that practical masters, doctoral and
post-doctoral fellowships will help in the interim, perhaps augmented by more
vocationally focused apprenticeships and on the job mentoring, but they will need to
be extremely flexible in their capacity to foster people with diverse talents
expressed with different emphasises on different areas. Hall’s balance between
technical and creative knowledge was useful but in many cases an inflection towards
one or the other would be optimal. It is unusual to find someone with equal parts
creative and technical talent, and more feasible to provide people with the skills
they need to contribute to multi-disciplinary teams. Similarly, it will often be
useful to involve people with theoretical academic knowledge as well: perhaps
possessing barely enough technical knowledge to work closely with engineering teams
but deep understanding of the creative and theoretical aspects to a problem.
An emphasis on teamwork skills, and confidence working across disciplinary boundaries
will resolve many issues — accepting the need for hybrid teams as well as people with
rare hybrid skills — but this will not resolve the deeper issues of narrative and
aesthetic quality demanded by next-generation creative products. At some point the
education system will need to adapt to the challenging requirements imposed by the
creative industries and develop graduates with myriad indefinable mixtures of talent,
fusing computational and creative skills along tailored learning pathways. The
transfer of knowledge between university research teams and the creative industries
is in this sense merely the first act in a wider process of a humanistic re-tooling
of (a growing part of) the engineering profession. The development of robust talent
pipelines from the education system into the creative industries should be the
medium-term goal.
Applying AI to Storytelling was an ambitious and complicated interdisciplinary
project undertaken in the context of a complex intellectual, creative, technical, and
policy development landscape. The United Kingdom’s cognizance of the difficulties
(and potential economic benefit) in the development of next-generation immersive
products, and their proactive funding of applied research activity between academic
and commercial partners, made it possible to engage in the relatively high-risk
R&D necessary to make progress but significantly more work is needed. A
full-spectrum approach is needed to understand the implications of immersive product
development in applied and theoretical terms. The goal must be to enable development
of more sophisticated products (dare we hope, on a par with the centuries of
tradition inherited by the creative industries), enabled by interdisciplinary
academic enquiry, improved design and engineering processes, more effective
pedagogical approaches, and enhanced career pipelines. The humanities, and DH in
particular, are well positioned to contribute to this process but experimentation
will need to occur across many concurrent areas of activity, and we will — in order
to work at the cutting edge of creative endeavour and have a voice in the development
of next-generation creative products — need to reconcile ourselves to contributing to
commercially driven endeavours.
Works Cited
Aarseth 1997 Aarseth, Espen J. 1997. Cybertext: Perspectives on Ergodic Literature. Baltimore:
The Johns Hopkins University Press.
Asdal 2018 Asdal, Kristin. 2018. “
‘Interested Methods’ and ‘Versions of Pragmatism,’
”
Science, Technology, & Human Values 43, no. 4
(July): 748–55.
Atkinson and Kennedy 2018 Atkinson, Sarah
and Helen W. Kennedy. 2018. “Extended Reality Ecosystems:
Innovations in Creativity and Collaboration in the Theatrical Arts.”
Refractory: A Journal of Entertainment Media 30 (July
14).
https://refractory-journal.com/30-atkinson-kennedy/.
Bahr et al. 2017 Bahr, Gisela Susanne, William H.
Allen, Philip J. Bernhard, and Stephen Wood. 2017. “The
Artificial Memory of Mr. Polly: Memory Simulation in Databases and the Emergence
of Knowledge.”
Leonardo 52, no. 3 (February 28): 300-304.
Carayannis and Campbell 2009 Carayannis, Elias G. and David F.J. Campbell. 2009. “
‘Mode 3’ and ‘Quadruple Helix’: Toward a 21st Century Fractal Innovation
Ecosystem.”
International Journal of Technology Management 46, no.
3/4: 201-234.
Chu 2014 Chu, Ted. 2014. Human
Purpose and Transhuman Potential: A Cosmic Vision of Our Future Evolution.
Chicago: Origin Press.
Ciccoricco 2007 Ciccoricco, David. 2007. Reading Network Fiction. Tuscaloosa: The University of
Alabama Press.
Demetriou 2018 Demetriou (Yiota), Panayiota A.
2018. “
‘Imagineering’ Mixed Reality (MR) Immersive Experiences in the Postdigital
Revolution: Innovation, Collectivity, Participation and Ethics in Staging
Experiments as Performances.”
International Journal of Performance Arts and Digital
Media 14, no. 2 (August): 169-186.
Etzkowitz and Leydesdorff 1995 Etzkowitz, Henry and Loet Leydesdorff. 1995. “The Triple Helix —
University-Industry-Government Relations: A Laboratory for Knowledge Based
Economic Development.” SSRN Scholarly Paper. New York: Social Science
Research Network.
Etzkowitz et al. 1998 Etzkowitz, Henry, Andrew
Webster, and Peter Healey. Capitalizing Knowledge: New
Intersections of Industry and Academia. New York: State University of New
York Press, 1998.
Flew 2012 Flew, Terry. “Origins of
Creative Industry Policy.” In The Creative Industries:
Culture and Policy. London: SAGE Publications Ltd, 2012.
Forrester 2019 Forrester, Katrina. 2019. In the Shadow of Justice: Postwar Liberalism and the Remaking of
Political Philosophy. Princeton: Princeton University Press.
Gross 2020 Gross, Jonathan. 2020. The Birth of the Creative Industries Revisited: An Oral History of the 1998 DCMS
Mapping Document. London: King's College London.
Haraway 1991 Haraway, Donna. 1991. “A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in
the Late Twentieth Century.” In Simians, Cyborgs and
Women: The Reinvention of Nature, 149–81. New York: Routledge.
Hayles 2008 Hayles, N. Katherine. 2008. Electronic Literature: New Horizons for the Literary. Notre
Dame: University of Notre Dame.
Hegel 2017 Hegel, Georg Wilhelm Friedrich. 2017. The Phenomenology of Spirit (1807). Edited by Terry P.
Pinkard. New York: Cambridge University Press.
Huizinga 1938 Huizinga, Johan. 1938. Homo Ludens. Haarlem: Tjeenk Willink & Zoon.
Ihde 1993 Ihde, Don. 1993. Postphenomenology: Essays in the Postmodern Context. Northwestern University
Studies in Phenomenology and Existential Philosophy. Evanston:
Northwestern University Press.
Ihde 2002 Ihde, Don. 2002. Bodies
in Technology. Minneapolis: University of Minnesota Press.
Ihde 2009 Ihde, Don. 2009. Postphenomenology and Technoscience: The Peking University Lectures. SUNY
Series in the Philosophy of the Social Sciences. Albany: SUNY Press.
Innovate UK 2008 Innovate UK. 2008. The Immersive Economy in the UK: The Growth of Virtual, Augmented
and Mixed Reality Technologies. London: HM Government.
Innovate UK 2018 Innovate UK. 2018. “Applying AI to Storytelling — Bringing Computational Research into
Creative Industries.”
UK Research and Innovation.
https://gtr.ukri.org/projects?ref=133687. Accessed 3 March 2020.
Knorr-Cetina 1981 Knorr-Cetina, Karin. 1981.
The Manufacture of Knowledge: An Essay on the Constructivist
and Contextual Nature of Science. Oxford: Pergamon Press.
Knorr-Cetina 1999 Knorr-Cetina, Karin. 1999.
Epistemic Cultures: How the Sciences Make Knowledge.
Cambridge, MA: Harvard University Press.
Latour and Woolgar 1979 Latour, Bruno, and
Steve Woolgar. 1979. Laboratory Life: The Construction of
Scientific Facts. Princeton University Press, 1979.
McDuff and Czerwinski 2018 McDuff,
Daniel, and Mary Czerwinski. 2018. “Designing Emotionally
Sentient Agents.”
Commun. ACM 61, no. 12 (November): 74–83.
Murray 2010 Murray, Fiona. 2010. “The Oncomouse That Roared: Hybrid Exchange Strategies as a Source of Distinction
at the Boundary of Overlapping Institutions.”
American Journal of Sociology 116, no. 2 (September):
341–88.
Nyssa 2019a Nyssa, Zoe. 2019a. “Building an STS of Practice & the Practice of STS.” Society for Social
Studies of Science, New Orleans, 4 September.
Nyssa 2019b Nyssa, Zoe. 2019b. “Lab Studies Reloaded? Machine Learning, Ethnography, and Critical STS.”
Society for Social Studies of Science, New Orleans, 4 September.
Opposable Group and TechSpark 2018 Opposable Group and TechSpark. 2018. Creative Tools and
Workflows for Immersive Content Creation. London: Digital Catapult.
Pickering 1995 Pickering, Andrew. 1995. The Mangle of Practice: Time, Agency, and Science. Chicago:
University of Chicago Press.
Rodriguez 2006 Rodriguez, Hector. 2006. “The Playful and the Serious: An Approximation to Huizinga's Homo
Ludens.”
Game Studies 6, no. 1 (2006).
Rosenberger and Verbeek 2015 Rosenberger, Robert and Peter-Paul Verbeek. 2015. “A Field Guide
to Postphenomenology.” In Postphenomenological
Investigations: Essays on Human-Technology Relations, edited by Robert
Rosenberger and Peter-Paul Verbeek, 9-41. Lanham: Lexington Books.
Sandry 2017 Sandry, Eleanor. 2017. “Creative Collaborations with Machines.”
Philosophy & Technology 30, no. 3 (September):
305–19.
Slaughter and Leslie 2001 Slaughter,
Sheila and Larry L. Leslie. 2001. “Expanding and Elaborating the
Concept of Academic Capitalism.”
Organization 8, no. 2 (May): 154–61.
Slaughter and Rhoades 2009 Slaughter,
Sheila and Gary Rhoades. 2009. Academic Capitalism and the New
Economy: Markets, State, and Higher Education. Baltimore: Johns Hopkins
University Press.
Smithies 2017 Smithies, James. 2017. The Digital Humanities and the Digital Modern. Basingstoke:
Palgrave Macmillan.
Taylor 1977 Taylor, Charles. 1977. Hegel. Cambridge: Cambridge University Press.