In this paper, we introduce a new model for the automatic recognition of named entities in medieval Latin charters. The named entity concept involves all physical and real objects which can be designated with a specific name. A named entity recognition (NER) model detects entities in the text and classifies them (e.g., a person's name, a location name, an organization's name, a date). It becomes possible, then, to index the resulting information units and to extract relationships between them. In that sense, automatic NER systems can help to improve current information systems by building a knowledge base of all the spatial, social and personal entities in a set of documents; this enables a searchable and quotable corpus matrix using actors and places as dynamic keywords which help to provide meaningful answers to questions asked on a large textual database.

In recent years, some projects have entailed the production of structured data through manual tagging of named entities in thousand-item historical corpora. This manual operation takes years for an entire team and can obviously not be scaled up to the almost 500,000 (and increasing) medieval digitized documents. This lengthy process can be completed only with massive acceleration through a computational operation of converting automatically thousands of documents in raw data format into structured data in a short time.

In the present project we use one of these hand-annotated corpora as our dataset: the CBMA (Corpus Burgundiae Medii Aevi), which contains a collection of medieval charters and cartularies produced in Burgundy. From the 11th century on, many ecclesiastical institutions started to compose cartularies in which they transcribed their own documents, especially titles relating to their property and their land rights, in order to better preserve written deeds and keep legal proof of ecclesiastical possessions and social relationships. Today, these registers stand as one of the major sources for medieval studies. Among the texts collected in the CBMA we isolated a set of 5300-items (1,2 million words), which correspond mostly to cartularies from Cluny. This large sub-set contains mainly donations, exchanges and sales, that is, written records dealing with the transfer of lands, goods and rights between ecclesiastical institutions and between private and public actors. While these documents are relatively formulaic, with only limited vocabulary and a relatively small number of turns of phrase, their geographical provenance is as diverse (more than one hundred different places) as the chronological scope of their production is wide (from the early 10th century to the middle of the 13th century). Moreover, they include many different kinds of documents: charters, notices, bulls, diplomas, letters, lawsuits, etc.

As we will show, developing a model able to achieve entity recognition for this corpus presents several challenges. The need to choose a single corpus for stability and coherence can affect the ability of the model to be generalized to a wide range of documents, because it reduces the variety of samples and can hide more specific phenomena. Constructing a robust model requires finding a technical and intellectual compromise between collecting sufficiently varied data and avoiding extreme training on some similar structures [Plank 2016]. This issue becomes even more complicated if we consider the particular characteristics of our main corpora: on the one hand, diplomatic charters combine formulaic structure and variable lists of named entities [Zimmermann 2003], which can be an advantage regarding iteration in training. But it also means fitting problems of recognition over other corpora, because there are many formulaic traditions, different models in according to juridical actions and regional redactional dependencies. On the other hand, because Latin is a low-resource language in natural language processing, the work must be conducted using tools and vocabularies not initially developed for Latin or developed for classical Latin literature [Eger et al. 2015]; not to mention the technical adaptations required for the specificities of medieval Latin due to several linguistics alterations. Above all, less technical but more extended, pre-processing must be completed, such as the normalization of textual spelling, the manual validation of much inflected language arbitrariness, and the listing of difficulties due to overlapping and to textual ambiguities.

The construction of a proper validation environment, proving the robustness of our model, has led us to build several corpora and sub-corpora variations and to annotate manually other smaller corpora. We finally developed a validation protocol with an extensive set of evaluations in order to show that our model can be applied to a wide range of unannotated charters with different typologies and belonging to different scriptural traditions and regional origins.

This paper describes, first, the historical nature of our corpus and the features of our gold-standard corpus. It explains the method adopted in generating the model with a machine-learning approach based on conditional random fields (CRF). We detail the construction of the model and sub-models, as well as the results obtained by the application of our general model and by the different experiments using cross-validation. In the discussion, we try to develop a broader argument for results by referring to current onomastic studies, historical and scriptural usages, and language variations, thus offering a contextualized explanation that can help to clarify, from a non-technical and humanistic perspective, matches and errors obtained in the application of the model. The constitutive models of this project have been deployed in a web-based application adapted to expert and non-expert users located at https://entities-recognition.irht.cnrs.fr. An automatic and accessible web-based annotation workflow, based on models developed in this project, has been deployed at https://entities-recognition.irht.cnrs.fr. Finally, we propose some suggestions for the fruitful application of these models in the wider context of digital humanities.

Named entity recognition (NER) is one of the most promising technical tools in the digital humanities field. Some classical digital humanities projects have been conducted with NER, thereby opening new ways to explore and query large databases. Actively used in the medical, biological and journalistic domains [Abacha et al. 2011] [Eltyeb et al. 2014], named entities prove to be meaningful pieces of information intimately connected to the main issues addressed by humanistic studies. NER techniques are an excellent way to provide an overview of a corpus. On the one hand, they help to classify data into pre-defined categories, which is primordial to indexing and describing data and eventually to building structured databases. On the other hand, they can activate data exploration at different scope levels. For example, NER techniques can be used to identify core concepts and cluster them into vocabularies, ontologies, and actor's attributes on documentary series They are also indispensable in the pre-treatment of texts since they provide pieces of information that do not appear in the language dictionaries. Alternatively, at a corpora level, NER techniques can serve to build thick social, spatial and semantic relationship networks using named entities as nodes.

NER work in the digital humanities community consists of two subtasks: (i) detection and classification of named entities in classical and modern language texts and (ii) entity linking towards already existing knowledge databases. Concerning the first task, experiments have proven that supervised approaches are more suitable for large-scale databases than rule-based or unsupervised ones [Nadeau et al. 2007]. In these supervised methods, especially statistics-sequential methods, algorithms use labelled data and meta-information obtained by different types of syntactical analysis, such as POS tagging, parsing, chunking, from word-based to sentence-based levels. The system then builds a statistical model based on these labelled data and suggests the best labels for a new, unlabelled sequence of words. Some digital humanities projects in recent years [Curran et al. 2003] [Won et al. 2018] [Rayson et al. 2017] produce supervised models exceeding 80% on the F-measure (harmonic mean of precision and recall).

Nevertheless, because of the lack of annotated data, rule-based or dictionary-based models are still very popular, despite the fact that they generally achieve a low recall ratio (for the former) or a low precision ratio (for the latter). Training a dictionary-based recognition model against a list of names can lead to a high ratio of recognition for a particular corpus, but the model is often not robust when applied to unseen texts or different types of data. On the other hand, rule-based models trained using rule definitions and descriptions to categorize entities show a valid global recall, but a slight tendency to poor precision-ratio on unseen texts. In spite of that, these methods have proven very useful for rapid textual annotation of small corpora or more standardized corpora like those from literature and journals [Grover et al. 2008] [Mosallam et al. 2014] [Ehrmann et al. 2016].

Recent years have seen an increasing popularity of hybrid, semi-supervised approaches. Bootstrapping techniques use a small set of annotated data in order to obtain automatically tagged data by taking advantage of specific repetitive and contextual patterns. Some work on this topic has introduced interesting propositions with excellent results [Brooke et al. 2016] [Neelakantan et al. 2015] [Klein et al. 2014].

Both unsupervised and supervised methods need annotated corpora in order to build better linguistic analyzers and evaluation tools. This problem has motivated, in recent years, different actions in the digital humanities community. The main one is the production of handcrafted corpora with one or more fully annotated attributes, such as morphological features, semantical descriptions, syntactic relationships, named entities, etc. The medieval corpora made available by CBMA, CDLM, and SRCMF projects are among these, as well as examples of well-constructed lists of authorities, gazetteers and treebanks such as proposed by Pelagios, LASLA, and Perseus, which are used in many projects to improve semi-supervised methods specially addressed to classical and medieval literature. In addition, the growing number of platforms created to facilitate morphological analysis of new corpora, like Collatinus Lemlat, or GutenTag, is remarkable; these platforms offer a gateway to rapid extraction of a full range of linguistic attributes.

All these efforts are complemented by an increasing availability of programming packages including easy access to classical techniques like POS tagging, parsing, chunking and named entity resolution in combination with annotating tools based on treebanks-training such as TreeTagger and Lapos. They are contributing to the incorporation of NER in mid-range projects. Stanford CoreNLP, Freeling,, Natural Language Toolkit and Scikit-learn are among the most used set of tools and libraries.

Concerning the task of disambiguation (entity linking), projects are close to semantic Web and linked open data principles, based on interoperability and machine-readable data. Natural languages are ambiguous by definition, and retrieved entity mentions must be clearly identified. There are several propositions, especially from library domains, calling to use lists of authors and authoritative databases to link recovered entities with unique referents stocked in big databases like DBpedia [Lehmann et al. 2013], Freebase [Bollacker at al. 2008], or Yago [Hoffart et al. 2013] using URI’s (Uniform Resource Identifier) [Nouvel et al. 2016] [Rizzo et al. 2011]. Some very interesting works based on small-size corpora have been taking this direction, especially studies by literary scholars [Frontini et al. 2016] [Elson et al. 2010]. However, they are limited to databases containing knowledge about relevant people or mapping modern texts. For instance, these works are tightly focused on geographical disambiguation, because places do not show a high number of possibilities.

To address both problems, automatic entity detection and entity disambiguation, it is necessary to have large, hand-made annotated corpora whose production is highly time-consuming work. For instance, computer acceleration is the most promising solution for dealing with the large and growing number of available digital historical documents. Algorithm-based techniques can do intelligent tasks and provide a very valuable result. However, experience shows that high context dependency at different levels compels the combination of the automatic result with an academic methodology to define correct heuristics tasks and to incorporate the most adequate epistemological rules.

In this work, we use different types of information to build a model for automatic named entity recognition: grammatical categories, lemma, case, entity categorization and n-gram observations. A model based on a probabilistic CRF-approach that extracts correlations dependencies between this information and observations leads us to high performance in the process of categorizing entities. High percentages in the results, after having tested the model on corpora of different geographical origins and different periods, demonstrate that the linguistic and statistical algorithmic approach has proven to be robust. In fact, the results are quite conclusive, but the evaluation process involves an important level of heterogeneity needing an explanation that exceeds the most technical level and requires the use of humanistic knowledge. We need to answer why a model created from a regional corpus can obtain a high recognition rate on medieval documents from other regions, chronologies and even traditions.

Thus, references to the historical background, social usages and scriptural variations can provide a more meaningful explanatory context, enriching the evidence obtained by machine-learning methods. Reviewing the literature about historical names is a good first step. Anthroponymic studies suggest a double movement in Western Europe after the 10th century: first, a reduction of name stock and consequently a reduction of the variety of common names; and second, an extension of double names in variable forms: nomen paternum (name + name of father), locatives (name + place-name) and nicknames (name + profession, qualities or titles). In fact, the nature of the second element in bi-names distinguishes several naming traditions in Europe. In France, the meridional tradition incorporated the nomen paternum quickly, that is, the hereditary second name coming from the father, but central and northern regions (like Burgundy and Île-de-France) developed a preference for locatives and periphrastic forms [Bourin 1996].

Single-name forms disappeared in the 12th century, and traditional and local names were replaced by the universal Christian or princely names. At the same time, in the northern Hispanic regions and Castile, the tradition promoted an incorporation of nomen paternum, formed with the declension in the original name of the father (v.g. Gundisaluo Roderici, Rodericus Ferrandi) and, later, a composition of three names in association with locatives and nicknames (v.g Ferrando Sanchiz de Fenosa) [Sopena 1996]. Norman traditions heavily influenced the central and southern regions of England, which adopted double locative or triple names and nomen paternum formed by the second name of the father (v.g Guidonis Nerioli de Buxiaco, Aimo de Monte Pauonis) [Billy 1995]. The tradition operating in the Lombardy region presents the same phenomena of duplication in anthroponymic elements during the 11th century, but prefers a system that promotes a more broadly referential second name (v.g. Otonnis qui dicitur de Suso, Grioulum filium Lafranci Caipeni) which takes the form of a periphrastic composition promoting surnames, geographical origin, and ancestor’s filiations [Corrarati 1994].

Concerning geographical names, we must make a distinction between landscape entities and nominal entities. The first are used to describe or to locate a property, whereas the second appear as a locative complement of a compound name. Indeed, a large proportion of geographical entities is linked to the rise of double names between the 11th and 13th centuries, when the locative complement became a form of familial or personal social distinction. The expansion and precise definition in Europe of the name + locative combination through use of the de particle (genitive or nominative without preposition) produced a very strong increase in binomial name entities combining personal and geographical components. This is a process that quickly produces great variety by proposing the origin, residence, feudal, or ethnic place as locative complement (v.g. Iohannes Allemanus) completing the information about an individual person. In fact, all these locatives are generally real places, but they are usually defined as hagio-toponyms, old toponyms, and micro-toponyms hardly connected with real spaces on a map.

Nevertheless, most of the geographical entities are not associated with a personal name but correspond to the first above-mentioned type: landscape and territorial names. They are the names of a terra, a villa, a pagus, an ager, a river, a building, a church, a monastery, a so-called place, etc. Names and master words or co-occurrences are closely related in charters (see Table 5) because cartularies produce, by classifying records geographically and transcribing them in a codex, a spatial knowledge of ecclesiastical estates which uses many of the same global geographical entities [Chastang 2007]. The first effect of this is the constant reference in formulary charters to a well-delineated territorial space, a cadastral order that today is almost impossible to reconstruct due to numerous information gaps [Bange 1984].

According to the evaluation of our general model, the results obtained in B-LOC recognition (geographical Begin-entity) are just five points below (90% compared to 95%) the best performance obtained in B-PERS (personal Begin-entity). That means that performance on detection of geographical entities and the classification of single entities is almost as excellent as obtained on personal names. Geographical entities usually appear in less complex forms than the entities of the name, which raises the percentage of this first result. But performance decreases almost ten points if inside entities (I-PERS and I-LOC) are compared (90% and 80%), which indicates that elements involved in composed geographical entities are harder to model. This proportionality is more or less maintained in evaluating foreign corpora (see Tables 6 and 7). All results are, on average, acceptable, but, once more, the problems are concentrated on inside-entities, as is shown when comparing partial and exhaustive Brateval results.

The most common format for complex entities is the double name (bi-name), but as we have seen, our model can handle well the recognition of this prolific format and its variants, which suggests that we need to examine more complex forms of composed entities. Thus, regarding the apparition of complex geographical entities, we notice different situations depending on regional traditions. Concerning personal names, Castile and Southern England are regions where the triple name expansion takes place one century before it does in Central France. Both regions form bi-names with two nominal components, and the incorporation of locatives entails triple or quadruple names (v.g. Guilielmi de Sancto Satiro, Martinus Garciez de Stallaia). In Lombardy, the formation of complex names never takes root, but the addition of periphrastic composition (through dicitur, nominatur, filius, de loco, etc.) creates some long entities impossible to dissolve. Almost the same thing happens in central French regions where representative quantities in triple names are not present before the 13th century. But the use of locatives and periphrastic forms since the 11th century contributes to the apparition of complex entities of up to five elements (v.g. Adalbertus de vico Camonaco filii Iohanni).

Nonetheless, the most complex entity format is related to overlapped and nested entities (see Section 4.2). Personal entities, acting as possessive genitive or nominative, can be found fulfilling the function of a geographical entity in the land and boundary land descriptions in donations, purchases, inheritances, or legal disputes (v.g a mane terra Bertrannus; sub domi ipsius Ansaldi). Furthermore, since the 10th century, saints and ecclesiastical institutions were currently present in the formulae of the donationes pro anima, where they performed the function of intermediate receptors and defenders of properties. Their presence in charters contributes to creating complex entities with four or more elements taking, associating, or superimposing a saint’s name, an institutional name, and a geographical name.

In part, this complexity is, even more, a result of medieval variants of the Latin language. The gradual expansion of use of the genitive since the 11th century created large groups of entities without intermediate particles; at the same time, there was an increase in prepositions and the extinction of many Latin case endings, all of which lead the model, in many instances, to recognize the words of non-entities as named entities. Latin phrase order is irregular, and exceptions in medieval variants are almost infinite; consequently, training taking into account grammatical rules, co-occurrences, and context can generate many false positives even with an approach that is not rule-based.

The difficulty in recognizing entities involved in these complex phenomena lies not so much in their quantity as in their extensive consequences. The percentage of complex entities (more than three elements) does not exceed 11% of the total in our corpora, but the statistical impact on results due to bad recognition of such entities is more elevated when they are composed of more than four elements: B-PERS + (3) I-PERS (that means three inside-entity errors for each non-recovered entity). Moreover, the impact of these phenomena is increased if we consider that the original mark-up of our database, in which non-physical personal names and juridical actors are classified as place-names, can elevate the number of false positives in automatic recognition of complex place-names.

The influence of these circumstances is more evident in the different tests of temporal variability that we have performed. On one hand, the results of century models applied to European charters are very stable, which confirms, first, the presence of important homogeneous structures in name compositions and formulaic patterns and, second, the presence of a larger stock of name compositions than we would have expected in the Burgundy region in each century, which provides more variety in the training process. On the other hand, in all cases, the model developed with charters from the 10th century, when the double name was not yet established, is less adequate for recognition in other centuries; at the same time, the models from the 10th, 11th, and 12th centuries are less efficient when used on 13th century charters, because larger and more complex compositions increased in this century.

Finally, the excellent statistical results in spite of temporal variability also show that there is a high degree of iteration in late medieval formulaic discourse. This characteristic helped us to form a valid model starting from an incomplete and not very structured corpus. We obtained an acceptable ratio of recognition when we tested the model on charters representing different legal acts: donations, transactions, royal orders, deeds and notarial actions from four European regions. That leads us to assume a vast radius of application for our model on Western European legal acts of various types. In addition, single-century models do not show substantial improvements compared to the results of general models when applied to documents from each century, which confirms that our general model can provide the best ratios of recognition along a temporal line of at least four centuries, from the 10th to the 13th century.

We present a named entities recognition model applied to medieval Latin charters, and we implement an adequate evaluation environment. The model produced from long data sets of Burgundian medieval charters is able to accomplish a high ratio of recognition of the personal and geographical names of medieval Latin documentary sources, especially from the 10th to the 13th century. The evaluation is focused on demonstration of the robustness of the model in diverse situations. We performed different cross-validation experiments modifying size, chronology, typology, and regional origin in several sub-corpora, obtaining a very acceptable ratio of recognition in each case, which confirms that our model has a wide range of applications with respect to medieval documentary sources. By crossing results, we can confirm that the model supports an accurate semi-automatization of named entities recognition and, in consequence, that it can provide a primordial level of structuration of data, saving many human efforts.

Validation and cross-experiments also show us that questions about representativeness in the learning model do not generate insurmountable issues. Our discussion tries to confirm that the most significant problems in recognition are an expression of the intense dependency of data on complex historical, social and scriptural conditions, and not only on linguistic or statistical concerns. Several historical and philological conclusions helped us to refine and normalize algorithmic approaches and to understand numerical results provided by the machine. This also led to corrections of the original dataset and to complement it by the integration of the new annotated datasets that we have produced during this project.

The use of this NER model can be fruitful in different fields of research in the Humanities, including, in particular, recovery information systems, automatic indexing and distant reading. Since named entities are not subject to a lot of variation, they can be used as keywords and meaningful terms improving internal workflows of search engines. A NER model which can automatically spot these keywords within a large dataset, becomes a useful tool to speed up, classify and refine requests on ancient documents that are available in plain text editions. The recovery of entities, acting as nodes of action, can serve as a first step in the production of historical GIS-maps, and the reconstruction of event timelines or social networks. These widely used forms of data visualization can benefit from automatic tagging while automatic tagging can also help to solve questions concerning the production or the dating of ancient texts by providing data that scholars have traditionally used in their textual or manuscripts studies – mention of persons, places, words or formulas. Since it enriches datasets with specific linguistic and semantic features, our NER model can even be integrated into a pipeline of tasks, such as topic classification of ancient texts, automatic handwriting recognition or word sense disambiguation.

* This project is supported by the "IDI 2016" project funded by the IDEX Paris-Saclay, ANR-11-IDEX-0003-02