The Primeros Libros collection was built through a collaborative effort between institutions in the U.S. and Mexico to produce digital facsimiles of all surviving copies of all books printed in the Americas before 1601. Known, controversially, as the American incunables, these books document the establishment of this new technology for textual production in the nascent Spanish colony. Created by New Spain’s mendicant friars and secular clergy, they cover topics ranging from religious practice to anatomy and politics. They have been valued most highly, however, for their discussions of indigenous Mexican languages, which make them important resources for both linguists and historians. This also makes the Primeros Libros vital cultural heritage objects for the more than one million Nahuas living in Mexico today, who have been systematically denied access to historical documents written in their native language [McDonough 2014, 4]. For this reason, both the Primeros Libros project and the automatic transcription project are motivated by the desire to increase the accessibility and discoverability of these documents.

The two books that will be featured as examples in this article both center on indigenous language, culture, and history, though in different ways. The Advertencias para los confessores de los naturales (Warnings for the Confessors of Natives) is a multi-volume tome written by the Franciscan friar Juan Bautista and printed around 1601 at the Colegio Imperial de Santa Cruz in Tlatelolco, Mexico. The Advertencias was written to be a guidebook for new missionaries, providing a detailed discussion of the theological intricacies of taking confession from new converts, along with translations of religious texts and ideas into Nahuatl (the predominant language of the Nahuas, known popularly as the Aztecs) and extensive theological passages in Latin (often copied wholesale from European authors). In his later Sermonario, Bautista introduces in some detail nine indigenous scholars who provided him with both linguistic and cultural assistance; we can assume that some of them collaborated in the composition of this work as well [Christensen 2013, 32].[5]

The second book featured in this study is Antonio Rincón’s Arte Mexicana. Printed in 1595 in Mexico City by Pedro Balli, the Arte Mexicana is one of three important sixteenth-century grammars of the Nahuatl language.[6] It stands out because its indigenous author was one of the only ordained indigenous Jesuits, and was the first indigenous linguist in Mexico. Walter Mignolo and others have argued that the introduction of alphabetic writing to indigenous American communities functioned as a linguistic conquest that paralleled other kinds of colonization, based on the presumed superiority of Latin grammatical structures [Mignolo 1995]. In contrast, Rincón was unique in seeing Nahuatl as different from Latin, rather than deficient, suggesting that the language called for new grammatical models. “The alphabetic representation of the languages,” McDonough observes, “can indeed be seen as suppression or even an attempted destruction of one technology and the imposition of another. At the same time, Nahua appropriation of the alphabetic script, and of the Latin and Castilian languages, was just as much a ‘possession’”  [McDonough 2014, 47]. For McDonough, Rincón was an indigenous scholar who took possession over alphabetic writing and Spanish grammatical models.

These two texts emblematize the transcription challenges posed by the Primeros Libros collection: the presence of fluidly multilingual text and variable orthographies. They were also selected because this project was developed through close work with Kelly McDonough, Stephanie Wood, Sergio Romero, and Adam Coon, scholars of Nahuatl who were particularly interested in the ways that corpus analytics could provide new opportunities for research into indigenous Mexican language and literature. The focus of this article emerges from the research interests of our collaborators.

We chose Ocular as our preferred tool for transcribing the Primeros Libros because it is state-of-the-art in historical document transcription, designed specifically for documents printed using a hand press.[7] By taking into account the unique material factors affecting transcription of these printed books, Ocular improves significantly on tools that expect the stylistic consistency of modern printed books. Ocular works by combining two generative statistical models that represent how text should be. The first model, which is called the “font model,” focuses on the material qualities of the text: the shape of the font, the alignment of the type, the over- or under-inking that make text difficult to read visually. The second model, which is called the “language model,” focuses on the text itself. After analyzing a language sample, it builds a statistical model of six-character strings (known as six-grams): given any sequence of five characters, it is able to guess at what the sixth character should be. The result of this combination is a model that can identify clearly defined characters and use context to recognize an unclear image [Berg-Kirkpatrick et al. 2013].

Ocular is uniquely effective because it pays particular attention to the material conditions of the text. When working with the tool, however, we found that equal attention to the language model is necessary for a fully functioning OCR system. Ocular was originally designed for a nineteenth-century British corpus which was relatively monolingual and orthographically regular. In our sixteenth-century corpus, monolingual documents were not guaranteed: texts switch between languages at the level of the chapter, paragraph, sentence, and even word. This is illustrated, for example, in a passage from Rincón’s Arte mexicana in which he describes the Nahuatl use of the gerund, writing: “El gerundio, en do, ſe ſuple tambiẽ en dos maneras. Lo primero por la compoſicion de todos los verbos que ſignifican quietud o mouimiẽto v.g. ni tetlaçotlatica, eſtoy amando, nitetlaçotlatiuitz. vengo amando…” ([Rincón 1595, 24r], Nahuatl words have been italicized).[8] Elsewhere, quotations from Latin are incorporated smoothly into the Spanish prose, much like in this article.

Like language usage, sixteenth-century orthography was not consistent even within a single document, where printers might use three or four spellings for a single word, including common letter substitutions (a “u” in place of a “v”) or shorthand (the elision of the letters n and m). Neither of these challenges is unique to the Primeros Libros corpus: in his survey Natural Language Processing for Historical Texts, Michael Piotrowski describes these as two of the key challenges for transcribing and analyzing historical documents created throughout the early modern world [Piotrowski 2012, 11].

To handle these challenges, we modified Ocular by developing a multilingual model that allows it to identify the language of each character before attempting to transcribe it. We also added an interface for orthographic variability, which allows us to alter the system manually according to period-specific orthographic patterns. A technical description of the system can be found in the Proceedings of NAACL 2015 [Garrette 2015].[9]

As the brief history of the Primeros Libros described, the colonial effort to reframe Nahuatl as a Latin language has been understood as a process that both enacted and recorded broader processes of cultural exchange and (forced) assimilation in early colonial Mexico. We see this embedded in the very texts themselves, as shown by the example of Rincón’s creative reworking of Latin grammatical law, or in Bautista’s efforts to translate theologically precise concepts and texts into Nahuatl. The differences can be expressed, importantly, at the level of orthography: for example, the presence or absence of the letter “h”, used by some philologists to mark the glottal stop, may reflect different understandings of the language that are shaped by efforts to adhere to – or sway from – the Latin model [Lockhart 2001, 104].

Consider, for example, the facsimile shown in Figure 1. This page from Bautista’s Advertencias discusses efforts to communicate the concept of the holy trinity to new indigenous converts. The danger is that the converts will understand the trinity – meant to be three facets of a single god – as polytheistic. The solution, Bautista informs us, is to use the Nahuatl phrase Ca huel imeixtintzitzin, which signifies “todas tres personas son el verdadero” (all three people are the true [god] [Bautista 1600, 52r]). As Bautista describes, however, embedded in this Nahuatl phrase is an amphibologia (amphibology: a grammatically ambiguous phrase) which might lead the uninformed to the heretical belief that God is a person. Bautista explains: “Eſta ãphibologia no ay ẽ latin, por ſer diferẽte la terminacion” (“This amphibology does not exist in Latin because the ending [of the word] is different”  [Bautista 1600, 52r]). This is Bautista’s paradox: without properly explaining the concept of the holy trinity, new converts will believe Christianity is polytheistic. Yet due to a grammatical difference between Latin and Nahuatl, efforts to explain the concept lead, themselves, to the risk of heresy.

The process of automatic transcription can pose a translation problem that is analogous to that faced by Bautista. This translation problem is introduced to the system by way of the language model. Recall that the language model is a simple n-gram model based on “language data” provided by the user. To observe how this language data shapes the transcription, consider Table 2, which shows three variations of an Ocular transcription of the page from the Advertencias shown in Figure 1.

The three variations here show three ways of “reading” the facsimile page, each based on a different language model. The first variation uses a model based on the New York Times, similar to the Wall Street Journal model used by the original Ocular system. The second variation uses a language model based on the full text of Lewis Carroll’s Alice in Wonderland and Through the Looking Glass from Project Gutenberg.[12] The third shows a language model that draws on three historical corpora of Latin, Spanish, and Nahuatl.

Each variation is a kind of “dirty OCR”: a deformation of the original text that looks like gibberish. A closer examination, however, shows that there are patterns. Each variation is a reworking of Spanish and of Nahuatl that reflects the linguistic biases of the original. Ocular works by pairing a “font model” based on the visual appearance of the characters with a “language model” based on its knowledge of what language is supposed to look like. In these examples, the “font model” pulls the transcription towards the visual appearance of the text, while the language model pulls it towards the linguistic context of modern English, Victorian English, and multilingual New Spain. The result is a jabberwocky-esque transcription that looks like the Advertencias – like Spanish and Nahuatl – but appears in sequences characteristic of other times and places.

We would never use Alice in Wonderland as language data for the automatic transcription of the Primeros Libros. What these examples make explicit, however, is that the language data has a direct effect on the transcription. This is true even in the multilingual, early modern corpus that we used for our transcriptions. Given the wide variations in orthographic norms among regions and writers and the sparsity of language data relative to the requirements of the language model, it was impossible to build a language corpus that perfectly represented the context of our documents. Instead, our language data is more generalized, which has a homogenizing effect on regional variations. This homogenization is complicated by the fact that many of the transcriptions we used for our data are modernized versions of historical documents. Modifications of spelling, extension of shorthand, and standardization of character use are respected practices among documentary editors working to produce readable documents for a (relatively) general public. When they are embedded into the language data, however, they become unrecognized influences over the shape of the final text.

A closer consideration of the Nahuatl case shows how this homogenization or modernization can have a meaningful impact on the final transcription. Because alphabetic Nahuatl was still under development during the sixteenth century, orthographic difference can be an important marker of regional, religious, authorial, or class distinctions. Our language data for Nahuatl came primarily from scholars schooled in the orthographic tradition promoted by James Lockhart. This tradition, based on that developed by the seventeenth century philologist Horacio Carochi, was primarily documentary: it sought to reflect the styles of the original documents [Lockhart 2001, 109]. Modernization nevertheless occurred, as in the transcription of an unrenderable shorthand as “qz,” or the general adherence to standards that didn’t “jelly” (to use Lockhart’s term) until the 1560s or 70s, some thirty years into the Primeros Libros corpus.

In this case, the decision to draw on Lockhart’s documentation in our transcriptions was a conscious decision to bias the model towards these orthographic standards. Perhaps more significantly, however, the Lockhart examples often came from the legal archives, which had spelling conventions that often differed significantly from their ecclesiastical counterparts. As Lockhart writes, “when left to themselves […] Nahua writers had a very different outlook on what they were doing than their Spanish counterparts. Spaniards were spelling words; in general, they wrote a given word the same way every time they used it, employing the same standard spelling, in relative independence of how they might pronounce it. To the Nahuas, the word, insofar as they were even aware of it, was a constantly changing entity with fluid borders”  [Lockhart 2001, 111]. This could be represented by a difference in spacing, but could also appear in the form of phonetic spelling variations. It was also reflected at the level of the letter, through the presence and absence of the glottal stop as “h” and the “n” or “m” to signify nasal sounds. To impose this orthographic pattern onto the Primeros Libros documents, which were primarily ecclesiastical, is to erase important cultural differences between two forms of Nahuatl writing. Yet given the paucity of the Nahuatl corpus, distinguishing between the various forms of Nahuatl was not a real option.

I find in this intractable challenge an echo of the problem that Bautista encountered with translating the holy trinity. Bautista found himself trapped linguistically between two heresies: the heresy of polytheism or the heresy of deistic personhood. Though the orthographic variations in Ocular’s language data may not appear to carry the same theological weight, they do mark epistemological differences, the erasure of which may, among certain circles, come dangerously close to heresy. If we return to the scene of textual production, we recall that these texts are the products of relationships between the friars – Spanish and criollo – and the indigenous scribes. Orthographic homogenization can also present itself as the erasure of already-obscured indigenous voices, or of the growing influence of Spanish epistemologies. Both of these factors are consequential for our reading of the text.

Recognizing bias in the language data is intuitive: it makes sense that what you put into the system will affect what comes out of it. Less intuitive are the ways that the system itself can have a deterministic effect on the transcription. This deterministic effect is built into the relationship between the font model and the language model, which work in tandem to recognize characters. It is also present in the evaluation system that we use to measure Ocular’s accuracy.

The previous section described how Ocular’s transcription output responds to different orthographic patterns in the language data. This does not mean, however, that we can simply impose a transcription philosophy on our system by choosing the right texts for the language model. When human transcribers decide to replace an “x” with the more modern “j” in words like dixe (modern Spanish dije, “I said”), they do so based on an understanding of the historical relationship between the two characters. In contrast, when the system encounters dixe, the visual data from the font model makes the letter “j” highly improbable as a substitute. Instead, if the historical usage of the “x” in place of the “j” is not embedded in the language data, the system is likely to substitute a visually similar, but incorrect, letter.

Consider a similar example that we encountered in the Advertencias. Table 3 shows two variations of the Spanish word mentira (lie) that appear on a single page. The first variation follows modern spelling conventions. In the second variation, the “n” has been elided, as indicated by a tilde over the e (mẽtira). When we give the language model a standard corpus of early modern Spanish, the system misreads the second variant as merita, a statistically probable interpretation of the character string, but not a correct one. When we use the interface for orthographic variation that we built into the system to teach the program about character elisions, it’s able to read both words correctly. This points, again, to the relationship between language data and transcription output. But it also reveals one way that the system imposes a single transcription method onto the text. Ocular prefers – and in some ways, depends on - an ultra-diplomatic transcription.

This preference for the ultra-diplomatic model was not always duplicated by the evaluation system that we used to test Ocular. When presenting our modifications of Ocular to a scientific audience, we provided data in the form of a table of results, summarized in Table 4 [Garrette 2015, 1039].

Our results show an improvement over Ocular, which in turn showed an improvement over Tesseract, Google’s popular and freely available OCR tool [Berg-Kirkpatrick et al. 2013]. This improvement can be understood as evidence of what Julia Flanders elegantly describes as the “progressive momentum” of the digital humanities. In a now-classic article in Digital Humanities Quarterly, however, Flanders argues that “the digital humanities domain reflects the non-progressiveness of the humanities disciplines”  [Flanders 2009]. While the improvements that our system provides for automatic transcription are real, they are not the whole story.

The Ocular system is evaluated by measuring the correspondence between the characters output by the system and the characters typed by a human transcriber. This poses a scientific problem: how can we be sure that the human is correct? If one system is closer to the human system, does that mean the system is more accurate? Better? Are those always the same thing? In this case, we found that human transcribers struggled to determine where to put spaces between words, and how to encode unusual diacritics or orthographies. A smudged letter might be a “u” in modern Spanish, but in sixteenth century text it might just as likely be a “v.” If one tool output “u” and the other output “v”, the one that guessed closest to the human would earn a better evaluation, but might not be more “correct.”

The epistemological concerns embedded in this evaluation system are made most clear in the Nahuatl example. In our case, none of our transcribers were fluent in Nahuatl, though all read Spanish and Latin. As a result, while a transcriber could make a decision about an unclear Spanish word based on his knowledge of the language and the historical context, he could only make Nahuatl decisions based on his knowledge of Spanish and Latin. As a result, the evaluative system encouraged an output in which the Nahuatl looked more like a Romance language.

This effect is compounded when we consider the history of the Nahuatl documents themselves. As described previously, for the early Spanish linguists, Nahuatl’s value as a language was measured against a Latin standard, such that one Franciscan was able to remark “the Mexican language lacks seven letters”  [Mignolo 1995, 46]. For these early linguists, this lack articulated not just the paucity of the language, but by association, the weakness of the culture which produced it. At the same time, the imposition of Latin grammar, orthography, and textual ideology onto Nahuatl culture was itself a reframing of the relationship among speaker, language, and text which would have epistemological consequences. When these linguistic relations are duplicated by the Ocular evaluation system, the colonial legacy of the documents is again embedded in the system’s output.