Eamon Schlotterback is a doctoral candidate in literature at Northeastern University. She studies life writing, queer and trans literature, and transgender theory. Schlotterback's dissertation project interrogates practices of self-creation in transgender memoir. She is lab coordinator at the Digital Transgender Archive and her writing has appeared in
Cailin Flannery Roles is a doctoral student in the English department at Northeastern university. Their fields of study include global animation and comics, cultural studies, and queer theory. They also conduct research on special projects for the Digital Transgender Archive, and they are Project Coordinator of the Homosaurus. In 2019, they received their MA in English with an emphasis in cultural studies at Kansas State University. Their writing has been published in
K.J. Rawson works at the intersections of the Digital Humanities and Rhetoric, LGBTQ+, and Feminist Studies. Focusing on archives as key sites of cultural power, he studies the rhetorical work of queer and transgender archival collections in both brick-and-mortar and digital spaces. He has co-edited special issues of
This is the source
In this case study of the Digital Transgender Archive, www.digitaltransgenderarchive.net, the authors discuss the results of a yearlong audit of the project. While scholarship in the field has focused on both the startup and sustainability stages of DH project lifecycles, far less attention has been paid to fostering healthy project growth and self-evaluation during the maintenance stage. After discussing the motivations for our audit and the methods we employ, this article offers five key observations and summarizes our responses to them. Our hope is that other DH project teams will find that some of these observations are applicable to their work and will then benefit from the responses we developed.
Discussing motivations for the audit of the Digital Transgender Archive and the methods employed.
Archival materials related to the experiences of transgender and gender-nonconforming
people are often difficult to locate due to the relative scarcity of these materials and
the access barriers presented by brick-and-mortar archives (e.g., geographic, financial,
environmental, intellectual)
The Digital Transgender Archive (DTA) is a free online collection of trans-related primary source historical materials and information on undigitized archival collections held throughout the world. The goal of the project is to leverage an open-access digital archive to mitigate barriers to research on transgender and gender-nonconforming people. When the DTA moved institutions in mid-2020, we chose this natural lull to conduct the project’s first audit. Our aim was to thoroughly examine every aspect of the project to identify our strengths and areas for growth while charting a course for continued development and vibrancy into the future. In preparation for this audit, we searched for published models, examples, or theorizations of auditing a digital project, but we found little scholarship to guide our work. To contribute to this area, this article presents a project report that shares the findings of our yearlong audit of the DTA. We begin with an introduction to the DTA alongside a discussion of existing scholarship on Digital Humanities project lifecycles and evaluative procedures. We then highlight five key observations that the audit revealed and how we addressed (or plan to address) those observations. In using this observation/response structure, we offer readers the distilled results of our audit in a format that we hope will be easy to apply to other Digital Humanities (DH) projects.
DH projects generally move through fairly predictable cycles, often including startup,
dissemination, maintenance, and sustainability (or abandonment) stages. Given that
cultural and institutional pressures guide digital humanities scholars towards
innovation specialize in imagining and launching digital projects
Like many DH projects, the startup phase of the DTA, from approximately 2013 to 2018, was a period of feverish growth. Following a model that will be familiar to readers, the DTA is a collaborative online aggregator of historical materials that are contributed by a range of cultural heritage institutions. Boosted by the support of a Digital Innovation grant from the American Council of Learned Societies, the DTA initially launched in January of 2016 while Rawson was a faculty member at the College of the Holy Cross. For the next several years, Rawson worked to build the collection with a dynamic team of undergraduate students and, when funding allowed, Digital Archivist Nicole Tantum. Software developer Steven Anderson, an external consultant funded through small internal grants, has also been key to our success as he is responsible for both the preliminary development and continued maintenance of the site. During this initial growth stage, we focused on deepening our collaborative networks and implementing a functional database that would have sufficient resources to attract researchers.
As we invested more time and resources into the project, our collaborative network expanded and the size of our collections grew. As of January 2022, we have 70 contributors from eight countries (Brazil, Canada, England, Ireland, Norway, Spain, South Africa, United States). These collaborators represent a range of cultural institutions––from grassroots archives and historical societies to university and public libraries––and most are primarily brick-and-mortar organizations. We currently have approximately 8,800 items available on our site, a small percentage of which were directly donated to the project and are not available for researchers to access in physical archives. The project’s primary purpose is to increase the accessibility of trans history. We accomplish this through our centralized database of primary source materials and our efforts to identify and include a wide range of materials fitting under a capacious banner of pre-2000, non-normative gender practices. In other words, our collections are not only focused on individuals who identify as transgender; instead, we include documentations of any experiences relating to transgressions of gender norms (in relation to particular cultural and historical contexts).
With our expanding collaborations and content, we became increasingly aware of the
responsibilities involved in maintaining this important resource. We attuned ourselves
to conversations in DH about orphaned projects and project sustainability, though we
noted the lack of consensus among digital humanists as to what sustainability means
exactly continuity of funding, data preservation (including choice
of standards and longevity of data) and the need for flexibility in order to change
business models,
particularly as digital platforms transform or become obsolete
While we found these discussions of the startup and sustainability stages of DH project
lifecycles to be quite helpful, we also observed that the maintenance stage has received
considerably less attention, perhaps because it is the least glamorous part of any DH
project. As Ashley Reed notes, there remain many opportunities to discuss how to
maintain a DH project and keep it healthy and thriving they work, the tools they build, the disciplines they connect to
With respect to the DTA, as we focused on developing and launching the platform and
meeting grant deliverables, we had little time to evaluate the project as it emerged.
Many of the DTA’s features, policies, and internal documents had not been reviewed since
the first few years of our startup phase. In some cases, policies that had become
integral to the project over the years had never been formally recorded. The majority of
the DTA’s materials have been processed by revolving teams of undergraduate students,
making it difficult to maintain consistent practices. These issues were exacerbated by
the DTA’s move to Northeastern in fall 2020 and the complexities of working remotely
during the COVID-19 pandemic, as our previous model for workflow depended on a physical
lab space in which new team members were trained by veteran ones. Because crucial
process-based work went undocumented, it also could not be critiqued or modeled by
community stakeholders or fellow DH practitioners, which is a noted problem in the field
When Rawson began a new position at Northeastern University (NU) in the fall of 2020, the move presented an ideal opportunity to hit pause on expanding the project and instead conduct a robust audit in order to adapt to a new institutional environment. The audit was also well-timed from a project development standpoint because the DTA had settled into collections development without conducting a thorough evaluation of our site, our holdings, or our planned growth. In order to move toward future grants and new collaborations with purpose and intention, it was important to fully and candidly consider where we stand now.
We understand audits as comprehensive and evaluative analyses of data collected on a
given project to gauge efficacy. Audits are common across many realms, and can be
particularly useful for digital humanists seeking to appraise a project’s success and
plan for future development. In auditing the DTA, we evaluated the site’s goals and the
degree to which it accomplishes those goals while also developing concrete strategies
for growth. Most importantly, we sought to ensure the DTA was adequately serving its
mission of making trans history accessible and identified ways to better serve that
mission. To conduct the audit, Rawson was able to enlist two NU Ph.D.
students––co-authors Eamon Schlotterback and Cailin Roles ––who led the audit as part of
their 10-hour-per-week graduate research assistant positions. In order to
comprehensively audit the project, we decided to spend the full year on this work (with
some of our time still focused on small-scale processing and supporting a 3-member team
of volunteers)
Throughout each step of the audit, we also employed a series of checks to ensure that the DTA site content and features aligned as much as possible with widely accepted standards for accessibility. Specifically, we compared our practices to those recommended in the Web Content Accessibility Guidelines (WCAG) 2.1 and ran the site through accessibility evaluation tools such as Functional Accessibility Evaluator 2.2 and WAVE Web Accessibility Evaluation Tool. We also reviewed blogs and forums written by and for screen reader users and downloaded an open-access screen reader, NV Access, to test the site. We plan to continue running these checks routinely to ensure the DTA site is as accessible to as many users as possible, and we encourage others to adopt similar practices.
In publishing our account of the audit and its results, our purpose is twofold: first, we want to share specific insights into the state of our project and the results of this maintenance audit; and second, we want to offer a model for DH practitioners who are performing their own evaluations of their projects or who may want to extrapolate some of our results to improve other DH projects.
Since the DTA initially launched in 2016, there have been significant developments in the understanding of trans history and the language used to describe gender-nonconforming (GNC) people and practices. With renewed scholarly and public interest in trans identities, there has also been an increasing push to digitize more trans-related historical materials, in part due to dissatisfaction with the oversaturation of materials relating to white trans women. Within this context, we spent a significant amount of time during the audit evaluating the DTA’s scope to better adapt to the shifting landscape of trans history.
We were interested in interrogating scope on a number of levels, including: genre of
materials; dates covered; race and ethnicities represented; trans identities
represented; and types of experiences documented. Among the elements of scope we
considered, race and ethnicity were particularly important to us. As VanHaitsma and
others have observed in reviewing the project, and as Rawson has discussed in interviews
and his own publications, the DTA’s holdings skew toward white U.S. and Canadian
subjects
As one of extraordinarily few archives focused on trans experiences, it is critical that
we attend to these issues of impact and access. Researchers interested in trans history
have had to negotiate a number of systematic challenges — including lack of
communication across archival institutions and the recent emergence and cultural
specificity of the term
invisibilized and ignored
and trans women glamourized and vilified
Before we began working to increase the number of people with marginalized identities
represented in the DTA, it was necessary to closely examine how and why we choose to
represent people’s identities overall. Our audit began at the tail end of the summer of
2020, when the United States was reckoning with its continued history of anti-Blackness
following the murders of George Floyd and Breonna Taylor, which deeply informed our
work. The audit became an opportunity to renew our approach to archiving objects
depicting BIPOC, to learn from Black and anti-racist archival practices, and reconsider
how we frame historically significant objects depicting racism
A central project completed over the course of our audit was our search for potential additions to our holdings, both to diversify the scope of the project going forward and to ensure that we provide broad access to as many trans-related historical materials as possible. We approached this search for materials with two objectives in mind: 1) to identify new materials at institutions we have existing relationships with; and 2) to locate new institutions and collections for future collaborations.
Among our partnered institutions’ collections, we found over 300 items that were already
digitized and could be made available on the DTA, ranging from nineteenth-century
medical literature to 1960s glamour shots. Of these, we prioritized certain materials
for immediate processing based on gaps in our current holdings. For instance, materials
related to incarceration and GNC children became higher priorities, whereas drag show
programs were less pressing since we already have a preponderance of those items. As we
filled some of the gaps in our collection, we sought opportunities to not simply add
items
Our search for new partners was fruitful beyond our aims. At the start of our audit, we had 64 contributing partners, but we have since added five more to the DTA site and identified an additional 15 that we plan to reach out to in order to form collaborations.These new institutions include organizations with significant holdings related to transmasculine individuals, Indigenous peoples of New Zealand, and Asian Americans. Our goal in collaborating with more archives, especially community- and trans- and BIPOC-led archives, is not only to make more items related to trans history available online, but also to build mutually beneficial relationships with our partners. Another result of these efforts is that we are currently pursuing funding for a large-scale project that will allow us to work with our partners to digitize thousands of pages of materials related to trans people of color, as well as improve our anti-racist description practices through a BIPOC-led steering committee.
Even six years since it went live, the DTA remains one of extraordinarily few archives
devoted to trans experiences, and many of our materials are only accessible to the
public through the DTA. Our position as one of the few freely available archival
resources on trans people necessitates that we expand our holdings to better represent
the full spectrum of trans experience and to continually strengthen our commitments to
ethical representation. Like many long-term DH projects, our work over the years has
often focused on expanding the quantity of materials rather than quality of
representation. This is not uncommon — as [equate] decolonization with the need for diversity in the field,
as if the
mere addition of ‘diverse’ bodies will transform the practices of digital humanities
and the traces of colonialism therein. Instead, as we evaluated the scope of the DTA, we
also worked to strengthen and extend our relationships with BIPOC-focused initiatives,
pursue funding for projects to enhance access to trans BIPOC histories, develop
antiracist description policies, and enact more informed mechanisms of
representation.
Assessing the DTA’s scope led us to larger questions about the ways we describe materials hosted on the site. Since the project’s start, we have attempted to maintain consistent descriptive practices by implementing a metadata application profile (MAP), a detailed document that outlines requirements and best practices for each metadata element (e.g., Title, Subject Coverage, Description). However, over the course of the DTA’s history, a number of different people from different backgrounds and levels of archival experience have participated in processing materials, which has resulted in discrepancies in the metadata for different objects.
Although DTA materials are discoverable through metadata and the terms used within the
materials, there were numerous inconsistencies regarding the usage of certain elements
and descriptive practices for trans identities and practices. For example, the most
commonly used term in subject coverage was crossdressers,
though in many instances the
materials predated the term and it was unlikely the subjects represented would identify
with it. This practice was in contrast to our policy, which cautions against assuming
individuals’ identities without self-acknowledgement. At the same time, this policy was
more consistently applied in regards to race and ethnicity, which meant that materials
focused on people of color were too often unmarked in the metadata. As a result, race
often became hidden or obscured while gender identity was frequently overdetermined.
Additionally, we noticed that in some cases, our metadata reproduced harmful language
practices from other institutions and that some items were published with little
description because team members were unsure how to interpret their content and account
for their complexity.
Our policies regarding privacy, explicit content, and descriptive practices have always
been publicly available on the project site, and the DTA has been widely praised for its
collaborative nature and continued accountability to the community
To get a better sense of what MAP and policy changes might be in order, we began by evaluating our collections and compiling a list of metadata-related concerns and best practices. Aware of our limitations as a U.S.-based team that is predominantly white, we researched the policies of a number of BIPOC-led digital archives, DH projects, and social justice organizations. In addition, we found Antracoli et al.’s
In some cases, addressing metadata issues on the DTA required simply correcting typos.
Other issues necessitated case-by-case analysis and editorial flexibility. For example,
we reached out to an individual represented in our materials who changed his name to ask
how we should refer to him when processing materials he created before the name change.
His answer — that we could either update his name or maintain both names — reflects his personal choice rather than archival best practices..dead names.
We also expanded our project-wide policies to clarify our processes for describing gender, race, ethnicity, and other identity markers. Though our metadata typically adopts language already used in the materials and language known to be used by the individuals represented, we often process items that cannot be neatly categorized, such as photographs with no extant context. The decisions we make in describing these items may not be immediately clear to visitors of the site, which could make them difficult to locate or result in misunderstanding of historical terminology. Therefore, as we evaluate ways to best represent individuals’ identities, we also turn a critical eye toward harmful language practices in the archival field. We have sought to disrupt racist and transphobic materials or descriptions by questioning whether to make available compelling but ethically questionable items, adding critical framing to descriptions of complicated items, and removing harmful descriptions and metadata inherited from contributing institutions. To support our team members in describing complex items, we began writing short reference documents that include language and framing suggestions as well as contextual information to guide them through the difficult and thorny process of respectfully and accurately representing trans and GNC subjects. We have also developed content warnings within the Description field for items with potentially harmful language and imagery.
As we resolved specific metadata issues and revised our policies and processing workflow, we also began to write a new edition of the DTA MAP that better reflects our current descriptive practices. This work includes clarifying how fields like Spatial Coverage become searchable so team members can apply it more effectively and updating Subject Coverage examples to match the newest version of Homosaurus, the LGBTQ+ linked data vocabulary we use to supplement the Library of Congress Subject Headings. While in the past the MAP has only been available to team members and contributors, we have chosen to make the DTA MAP 3.0 publicly accessible on the project website, both to encourage feedback from our stakeholders and to provide a model for other DH projects writing or revising their own MAPs.
In the six years since the DTA went live, the project and items from its collections have been taken up by academic researchers, pop culture journalists, intrigued readers, documentary filmmakers, and teachers of all kinds. We have taken great joy in seeing how the DTA has been utilized and engaged, often posting to our social media channels items that refer to the DTA and circulating these items among students working on the project. At no point, however, did we formally track such qualitative usage, even though measurable evidence of a project’s impact is critical for grant writing efforts. Additionally, while the DTA’s mission is to make trans history more accessible to a wide public audience, we had no dedicated methods for evaluating users’ experiences in accessing our materials.
Throughout the life of the project, we have relied on Google Analytics to track site
usage and Google Alerts to keep up-to-date with mentions of the DTA online.belongs to Google and is housed on U.S. servers, where data
may be subject to the legislation of that country,
raises serious concerns about
user privacy, particularly for a trans-focused project. As the DTA and its user base
grows, we find ourselves reevaluating whether Google Analytics fits into our project
values.
We initially considered conducting a typical usability test to better understand the project’s uptake and user interactions; however, we quickly recognized that soliciting test users during the pandemic would be difficult and potentially unsafe. Tabling the usability test for the time being, we decided instead to systematically evaluate usage of the project through reviews, discussions, and mentions of the DTA across academic databases, institutions’ websites, public forums, and social media platforms. In addition to bibliographic information, we tracked articles, chapters, theses and dissertations, resource lists, articles on pop culture outlets and non-academic websites, and social media posts. We noted what items people were using or responding to, paying particular attention to the DTA’s accessibility for young readers and individuals unfamiliar with digital archives. We also recorded feedback from formal reviews of the DTA, particularly reviewers’ observations regarding ways to expand the project’s scope and functionality and the need to clarify some site features.
Reviews of the DTA and articles highlighting its usage in classrooms by
outdatedterms are used on the site or how to effectively search using those terms. Recognizing that the creation of metadata is a highly subjective and political act and that trans-related terminology is both personal and contextual, we describe individuals’ identities and practices using terms provided within the materials, terms known to be used by individuals represented in the materials, and, when we have no other information, terms contemporaneous to the period or culture. Since this practice influences search results in ways that may not be immediately apparent, we developed a front-facing Search Tips & Terms page that explains how we use search terms and suggests terms for researchers to consider based on their interests.
Along with identifying areas for improvement, we learned a great deal about how the DTA
has been used in academic research. While scholars such as
In reviewing uptakes and usages of the DTA, it became clear that the project inspires a wealth of valuable engagement that is often overlooked by Google Alerts and other automated searches. Bearing this discovery in mind, we now dedicate lab hours at the end of each month to actively locate and record engagement to improve our understanding of users’ experiences and remain accountable to our communities’ needs.
Social media offers a useful tool for better connecting with potential users of the DTA,
particularly those outside of academia. As self-representation, expression or reflection and more organized forms of collaboration
and knowledge building.
The DTA maintains three social media accounts––Facebook,
Twitter, and Instagram. Early in the audit, we determined that our social media posts
had a few shortcomings: they were inconsistently published, their contents were not
standardized, they varied in the amount of information provided, and they were not
always fully accessible. Further, our engagement with our followers was minimal, often
occurring only through responses to comments rather than proactive communication. It was
unclear who our audience was or what they sought from the DTA on social media, which
made it difficult to foster a sense of community on these platforms.
In response to these concerns, we developed a Social Media Style Guide to foster more consistent and effective use of our social media presence across all three platforms (Facebook, Twitter, and Instagram). To create this guide, we evaluated platform-specific analytics to identify our audience’s demographics and track trends in post engagement. Next, we reviewed each platform’s accessibility guidelines and discussed ways to ethically present materials addressing discrimination and trauma. These data points allowed us to begin posting more accessible content (e.g., providing alternate text to describe images, using CamelCase for hashtags) that was better tailored to the interests of our social media communities. We also shifted the timing of our posts to our followers’ peak activity hours and developed a list of important annual dates to ensure that we commemorate events important to our work. Because some of our materials discuss violence and discrimination, the Social Media Style Guide includes best practices for content warnings.
Additionally, as a collaborative archive that relies upon the labor of countless individuals and organizations, we wanted to make sure we were consistently crediting our partnering institutions on social media posts, directing traffic to their websites and social media presences as well as our own, and calling attention to exciting new projects. Therefore, we created policies for citing the sources of our content and began sharing more posts about the archives and websites that inspire us.
Given that physical archives can be uniquely inaccessible to trans people, we have found that social media is one additional way we can ensure that we are reaching the people whose histories are the focus of our work. Already, we are seeing positive responses to these changes, including higher rates of post engagement, increased site traffic originating from social media, and more consistent responses from our communities. Our Social Media Style Guide functions as a living document that we continually revise as we receive feedback and track usage. By formalizing social media policies while retaining space for further experimentation, we are now better able to create informative outreach content that is consistent with our goals and ethical commitments.
Since the driving mission behind the DTA is making trans-related history more accessible, we have a strong commitment to ensuring that our website content is welcoming and functional for as many of our visitors as possible, irrespective of their academic background or level of experience with archival research. While the majority of our day-to-day work is focused on developing and expanding the database itself, there are a series of static pages on the site that serve as important features of the project. These include “About” pages explaining the history of the project, our team and board, and our policies, as well as “Learn” pages that offer a starter’s guide to the DTA, a race and ethnicity research guide, a glossary, and a list of global terms. Many of these pages were created in the early stages of the project and have not been consistently updated.
As new team members, Roles and Schlotterback brought fresh perspectives on the project, and they conducted a comprehensive reading and analysis of our website copy and images. They were specifically focused on consistency, accuracy, intellectual accessibility, and user experience. Their review ultimately determined that while we were generally successful at presenting our content in an easy-to-navigate format with a style and tone that was appropriate for a generalist audience, there were significant inconsistencies across our static pages, including some content that was concerningly outdated.
Identifying the areas of our static pages that needed further development led to a number of changes. Some of the edits were quite simple, such as fixing typos and broken links. Yet even such minor changes pointed to the need to clarify our editorial practices. To eliminate inconsistencies in grammar and terminology, we devised and implemented a Style Guide for the DTA’s copy. Though we decided to defer to the AP Style Book for many stylistic decisions, we collectively established best practices for the specificities of writing about sex and gender, topics often neglected by mainstream style guides.
To better facilitate user engagement with the various functions on the DTA site, we
developed explanatory and instructional copy for certain features, such as the Map tool.
Among the front-facing pages of the DTA, we identified a number of areas that required
more sustained care, such as the
Glossary and
Global Terms
pages. Rather than attempting a short-term, superficial revision during the first phase
of the audit, the Glossary was significantly expanded by Roles and Schlotterback during
the spring 2021 semester. In addition to revising existing definitions to better
represent the complexities of trans experiences, Roles and Schlotterback identified a
number of new terms to define, including the most commonly used words in DTA materials’
subject coverage, as well as historical terms like female impersonators
and
masquerading
that are less familiar to present-day readers. As we highlight in the
introduction of the revised Glossary, our goal in defining these terms is not to
prescribe a single, correct definition, but to clarify how the DTA understands and uses
terms throughout the project. We view this page as a living document that will be
revised and expanded upon as needed, and we encourage DTA visitors to offer suggestions
for terms to add or clarify. As of this writing, revision of the Global Terms page is
ongoing.
The five observations and responses discussed above capture the key results of our year-long audit of the Digital Transgender Archive. The results of our audit have already proven to be tremendously insightful. In brief, we accomplished the following: our project scope has been critically analyzed and we have identified areas for intentional growth moving forward; more of our project documentation is written and publicly available; our project values have been articulated and shared; many of our static pages have been significantly revised; our social media presence is now more consistent and purposeful; we have more qualitative and quantitative user data to include in our grants; and we directly responded to reviews and feedback on the project.
We have shared these results in the hopes that our observations resonate with other project teams and their work. Irrespective of the specific outcomes, we found the process of auditing the DTA to be a highly effective time investment for a project that had passed through the startup phase and was just beginning to settle into ongoing maintenance. While there is always some pressure to innovate and expand, spending most of our year auditing our work allowed us the space for genuine self-evaluation so that we can now move forward with clearer intention and purpose.