The National Library of Finland has digitized a large proportion of the historical newspapers published in Finland between 1771 and 1910[1] [Bremer-Laamanen 2014] [Kettunen et al. 2014]. This collection contains 1,960,921 million pages in Finnish and Swedish. The Finnish part of the collection consists of about 2.4 billion words. The National Library’s Digital Collections are offered via the digi.kansalliskirjasto.fi web service, also known as Digi. Part of the newspaper material (years 1771–1874) is freely downloadable in The Language Bank of Finland provided by the FIN-CLARIN consortium.[2] The collection can also be accessed through the Korp[3] environment, developed by Språkbanken at the University of Gothenburg and extended by FIN-CLARIN team at the University of Helsinki to provide concordances of text resources. A Cranfield[4] style information retrieval test collection has been produced out of a small part of the Digi newspaper material at the University of Tampere [Järvelin et al. 2016]. An open data package of the whole collection was released in the first quarter of 2017 [Pääkkönen et al. 2016].

The web service digi.kansalliskirjasto.fi is used, for example, by genealogists, heritage societies, researchers, and history-enthusiast laymen. There is also an increasing desire to offer the material more widely for educational use. In 2016 the service had about 18 million page loads. User statistics of 2014 showed that about 88.5% of the usage of the Digi came from Finland, but an 11.5% share of use was coming outside of Finland.

Digi is part of the growing global network of digitized newspapers and journals, and historical newspapers are considered more and more as an important source of historical knowledge. As the amount of digitized data accumulates, tools for harvesting the data are needed to gather information and to add structure to the unstructured mass [Marrero et al. 2013]. Named Entity Recognition has become one of the basic techniques for information extraction of texts since mid-1990s [Nadeau and Sekine 2007]. In its initial form, NER was used to find and mark semantic entities like persons, locations, and organizations in texts to enable information extraction related to these kinds of entities. Later on other types of extractable entities, like time, artefact, event, and measure/numerical, have been added to the repertoires of NER software [Nadeau and Sekine 2007][Kokkinakis et al. 2014].

Our goal using NER is to provide users of Digi better means for searching and browsing the historical newspapers (i.e. new ways to structure, access, and, possibly, enhance information). Different types of names, especially person names and names of locations, are frequently used as search terms in different newspaper collections [Crane and Jones 2006]. They can also provide browsing assistance to collections if the names are recognized, tagged in the newspaper data, and put into the index [Neudecker et al. 2014]. A fine example of applying name recognition to historical newspapers is La Stampa’s historical newspaper collection[5]. After basic keyword searches, users can browse or filter the search results by using three basic NER categories of person (authors of articles or persons mentioned in the articles), location (countries and cities mentioned in the articles) and organization. Thus named entity annotations of newspaper text allow a more semantically-oriented exploration of content of the larger archive. Another large scale (152M articles) NER analysis of the Australian historical newspaper collection (Trove) with usage examples is described in Mac Kim and Cassidy [Mac Kim and Cassidy 2015].

Named Entity Recognition is a tool that needs to be used for some useful purpose. In our case, extraction of person and place names is primarily a tool for improving access to the Digi collection. After getting the recognition rate of the NER tool to an acceptable level, we need to decide how we are going to use extracted names in Digi. Some exemplary suggestions are provided by the archives of La Stampa and Trove Names [Mac Kim and Cassidy 2015]. La Stampa's usage of names provides informational filters after a basic search has been conducted. User can further look for persons, locations, and organizations mentioned in the article results. This kind of approach expands browsing access to the collection and possibly enables entity linking [Bates 2007] [Toms 2000] [McNamee et al. 2011]. Trove Names’ name search takes the opposite approach: users first search for names and the search reveals articles where the names occur. We believe that the La Stampa's use of names in the GUI of the newspaper collection is more informative and useful for users, as the Trove style can be achieved with the normal search function in the GUI of the newspaper collection.

If we consider possible uses of presently evaluated NER tools–FiNER, the FST, and Connexor’s tagger–for our newspaper collection, they only perform basic recognition and classification of names, which is the first stage of entity handling [McNamee et al. 2011]. To be of practical use names would need both intra-document reference entity linking, as well as multiple document reference entity linking [McNamee et al. 2011] [Ehrmann et al. 2016b]. This means that all the occurrences of equal names either in a single document or several documents, would be marked as occurrences of the same name. ARPA’s semantic entity linking is of broader use, and entity linking to external knowledge sources, such as Wikipedia, has been used, for example, in the Europeana newspaper collection with names [Neudecker et al. 2014] [Hallo et al. 2016].

One more possible use for NER involves tagging and classifying images published in the newspapers. Most of the images (photos) have short title texts. It seems that many of the images represent locations and persons, with names of the objects mentioned in the image title. As image recognition and classifying low-quality print images may not be very feasible, image texts may offer a way to classify at least a reasonable part of the images. Along with NER, topic detection could also be done to the image titles. Image content tagging thus could be one clear application for NER.

Our main research question in this article is, how well or poorly names can be recognized in an OCRed historical Finnish newspaper collection with readily available software tools. The task has many pitfalls that will affect the results. First, the word level quality of the material is quite low [Kettunen and Pääkkönen 2016], which is common for OCRed historical newspaper collections. Second, the language technology tools that are available are made for modern Finnish. Third, there is neither comparable NER data of historical Finnish, nor a standard evaluation corpus. Thus our results form a first baseline for NER of historical Finnish. One would be right to expect that results will not be very good, but they will give us a realistic empirical perspective on NER’s usability with our data [Kettunen and Ruokolainen 2017].

We are using five readily available tagging tools for our task. By using a set of different types of tools we are able to pinpoint common failures in NE tagging our type of material. Observations of error analysis of the tools will help us possibly to improve further NE tagging of historical OCRed material of Finnish. Differences in tagging between these tools also help us to analyze further the nature of our material.

We will not provide a review of basic NER literature; those who are interested in getting an overall picture of the topic, can start with Nadeau and Sekine [Nadeau and Sekine 2007] and Marrero et al. [Marrero et al. 2013], who offer historical and methodological basics as well as critical discussion of the theme. References in Nadeau and Sekine and Marrero et al. as well as references in the current paper allow further familiarization. Specific problems related to historical language and OCR problems are discussed in the paper in relation to our data.

The structure of the paper is following: first we introduce our NER tools, our evaluation data and the tag set. Then we will show results of evaluations, analyze errors of different tools, and finally discuss the results and our plans for using NER with the online newspaper collection.

For recognizing and labelling named entities in our evaluation we use FiNER software as a baseline NER tool. Our second main tool, SeCo’s ARPA, is a different type of tool, mainly used for Semantic Web tagging and linking entities [Mäkelä 2014][6], but it could be adapted for basic NER, too. Besides these two tools, three others were also evaluated briefly. Connexor[7] has NER for modern Finnish, which is commercial software. The multilingual package Polyglot[8] also works for Finnish and recognizes persons, places, and organizations. A semantic tagger for Finnish [Löfberg et al. 2005] also recognizes the three types of names.

All our taggers have been implemented as analysers of modern Finnish, although ARPA’s morphological engine is able to deal with 19th century Finnish, too. As far as we know there is no NE tagger for historical Finnish available. Before choosing FiNER and ARPA we also tried a commonly used trainable free statistical tagger, Stanford NER, but were not able to get reasonable performance out of it for our purposes although the software has been used successfully for other languages than English. Dutch, French and German named entity recognition with the Stanford NER tool has been reported in the Europeana historical newspaper project, and the results have been good [Neudecker et al. 2014] [Neudecker 2016]. However, we have been able to use Stanford NER with our material after this evaluation was finished, but results of this are not available yet.

As far as we now, besides the five tools evaluated in this paper, there are not many other existing tools to do NER analysis for Finnish[9]. FiNER has an updated version, but it does not seem to perform any better than the version we are using in our evaluation. In our discussion section we will get back to possibility of using a more historically aware statistical NE tagger. It should also be noted that all the tools except ARPA were used as is. That is, we did not enhance their performance with any additional tools or modifications, and the tools were treated as black boxes.

We have shown in this paper evaluation results of NER for historical Finnish newspaper material from the 19th and early 20th century with two main tools, FiNER and SeCo’s ARPA. Besides these two tools, we briefly evaluated three other tools: a Finnish semantic tagger, Polyglot’s NER and Connexor’s NER. We were not able to train Stanford NER for Finnish. As far as we know, the tools we have evaluated constitute a comprehensive selection of tools that are capable of named entity recognition for Finnish although not all of them are dedicated NER taggers.

Word level correctness of the whole digitized newspaper archive is approximately 70–75% [Kettunen and Pääkkönen 2016]; the evaluation corpus had a word level correctness of about 73%. Regarding this and the fact that FiNER and ARPA and other tools were developed for modern Finnish, the newspaper material makes a very difficult test for named entity recognition. It is obvious that the main obstacle of high class NER in this material is the poor quality of the text. Also historical spelling variation had some effect, but it should not be that high, as late 19th century Finnish is not too far from modern Finnish and can be analyzed reasonably well with modern morphological tools [Kettunen and Pääkkönen 2016]. Morphological analyzers used in both FiNER and ARPA seem to be flexible and are able to analyze our low quality OCRed texts with a guessing mechanism too. The FST and Connexor’s NER also performed quite well with morphology.

NER experiments with OCRed data in other languages usually show some improvement of NER when the quality of the OCRed data has been improved from very poor to slightly better [Packer et al 2010] [Marrero et al. 2013] [Miller et al. 2000]. Results of Alex and Burns (2014) imply that with lower level OCR quality (below 70% word level correctness) name recognition is harmed clearly [Alex and Burns 2014]. Packer et al. (2010) report partial correlation of Word Error Rate of the text and achieved NER result; their experiments imply that word order errors are more significant than character level errors [Packer et al 2010]. Miller et al. (2000) show that rate of achieved NER performance of a statistical trainable tagger degraded linearly as a function of word error rates [Miller et al. 2000]. On the other hand, results of Rodriguez et al. (2012) show that manual correction of OCRed material that has 88–92% word accuracy does not increase performance of four different NER tools significantly [Rodriguez 2012].

As the word accuracy of our material is low, it would be expected that better recognition results would be achieved if the word accuracy was around 80–90% instead of 70–75%. Our tests with different quality texts suggest this too, as do the distinctly different unrecognition rates with correctly and incorrectly tagged words.

Better quality for our texts may be achievable in the near future. Promising results in post-correction of the Finnish historical newspaper data have been reported recently: two different correction algorithms developed in the FIN-CLARIN consortium achieved correction rate of 20–35 % [Silfverberg et al. 2016]. We are also progressing in re-OCRing tests of the newspaper data with open source OCR engine, Tesseract[21], and may be able to improve the OCR quality of our data [Kettunen et al. 2016] [Koistinen et al. 2017]. Together improved OCR and post-correction may yield 80+% word level recognition for our data. Besides character level errors our material also has quite a lot of word order errors which may affect negatively the NER results [Packer et al 2010]. Word order of the material may be improved in later processing of the XML ALTO and METS data, and this may also improve NER results. It would also be important that word splits due to hyphenation could be corrected in the data [Packer et al 2010].

Four of the five taggers employed in the experiments are rule-based systems utilizing different kinds of morphological analysis, gazetteers, and pattern and context rules. The only exception was Polyglot. However, while there has been some recent work on rule-based systems for NER [Kokkinakis et al. 2014], the most prominent research on NER has focused on statistical machine learning methodology [Nadeau and Sekine 2007]. Therefore, we are currently developing a statistical NE tagger for historical Finnish text. For training and evaluating the statistical system, we are manually annotating newspaper and magazine text from the years 1862–1910 with classes for person, organization, and location. The text contains approximately 650,000 word tokens. After annotation we can utilize freely available toolkits, such as the Stanford Named Entity Recognizer, for teaching the NE tagger. We expect that the rich feature sets enabled by statistical learning will alleviate the effect of poor OCR quality on the recognition accuracy of NEs. Preliminary unpublished results comparing FiNER and adjusted Stanford NER for modern Finnish business newspaper data show that Stanford NER outperforms FiNER with about 30% units in combined results of five name categories. For recent work on successful statistical learning of NE taggers for historical data, see [Neudecker 2016].

On general level, there are a few lessons to be learned from our experiments for those that are working with other small languages that do not have a well-established repertoire of NER tools available. Some of them are well known, but worth repeating, too. First and most self-evidently, bad OCR was found to be the main obstacle for good quality NER once again. This was shown clearly in section 2.10. The implications of this are clear: better quality data is needed to make NER working well enough to be useful. Second, slightly surprisingly, we noticed that differences in lexical coverage of the tools will not show that much in the NER results. ARPA had clearly the best lexical coverage of the tools and FST the worst coverage, but their NER performance with locations are quite equal. This could imply that very large lexicons for a NE tagger are not necessary and a good basic coverage lexicon is enough, but this could also be language specific. Third, we showed that historical language can be processed to a reasonable extent with tools that are made for modern language if nothing else is available. However, if best possible results need to be achieved, more historical data oriented tools need to be used. It is also possible that the quite short time frame of our material enhances performance of our tools. Fourth, results of The Finnish Semantic Tagger showed that NER does not need to be only a task for dedicated NER tools. This has also been shown with modern Finnish [Kettunen and Löfberg 2017]. FST performed almost as well in a modern Finnish NER evaluation task as FiNER. Thus, at least in some cases, one can have available a suitable tool for doing NER even if the tool does not look like a NE tagger. Fifth, results of Polyglot hint that general multilingual packages do not work very well with data of small languages if their training data was not suitable for the task at hand. A lesson that we learned too was, that too detailed named entity classification (original FiNER labeling) is probably not a good solution for our purposes. In the future we shall stick to the use of persons, locations, and organizations.

Our main emphasis with NER will be to use the names with the newspaper collection as a means to improve structuring, browsing, and general informational usability of the collection. A good enough coverage of the names with NER also needs to be achieved for this use, of course. A reasonable balance of P/R should be found for this purpose, but also other capabilities of the software need to be considered. These lingering questions must be addressed if we are to connect some type of functional NER to our historical newspaper collection’s user interface.

First and third author were funded by the Academy of Finland, project Computational History and the Transformation of Public Discourse in Finland 1640–1910 (COMHIS), decision number 293 341.

Thanks to Heikki Kantola and Connexor Ltd. for providing the evaluation data of Connexor’s NE tagger.

Corresponding author: Kimmo Kettunen, ORCID: 0000-0003-2747-1382