One advantage of the detection method presented here is its universal applicability.[5] In principle, any (Latin) text can be used as an input, as long as it is available as an appropriately prepared .txt file. However, this also means that accessibility to electronic full texts is essential for the compilation of specific research corpora.[6] That said, at the present time, the acquisition of the required digital texts is not always easy and even structurally problematic.

The databases accessible without license agreements usually offer the text of older editions, which does not always reflect current research. The quality of the texts usually required in the discipline is more likely to be ensured in the case of licensed databases since these offer more up-to-date editions. Here, however, time-consuming and (in some cases) cost-intensive coordination and negotiation processes with rights holders associated with various institutes are necessary. In addition — both in the case of license-free and in the case of licensed texts — the accompanying information from critical apparatus is (mostly) missing.

For the current project, the required classical texts of current editions (without critical apparatus) were procured on the one hand through an adaptation of an already existing database license, on the other hand through renegotiating a license for the specific use case with regards to methods of text-data-mining.[7]

From a science perspective, however, the use of licensed text editions structurally prevents the transparency and reproducibility of the digital analysis processes actually envisaged by the project. After all, in order to keep the citation analysis processes replicable, the texts used should also be allowed to be disclosed.[8]

All texts provided by the publishers had to be converted into a uniform format for entry into the algorithmic processing. This was done using custom Python code to transform and (re)organize all the required paratextual information from the XML and .txt formats into the final .txt file formats used.[9] The described detection process is based on the Python programming language in version 3.8.[10]

The concept of quoting is based on the lexical understanding of citations as it is applied within the existing research tradition (i.e., the ancient concept of loci similes). The entire filtering process is thus fundamentally based on finding shared words in Jerome’s corpus of letters as the target text on the one hand and in the works of Virgil, Cicero, and other classical authors as source texts on the other. According to the loci similes concept, shared words are identical word forms in a certain number.[11]

The starting point of the calibration of the analytical toolkit is the so-called gold standard of citations collected in (manual-hermeneutic) research.[12] Potential new quotations are thus those found matches that emerge as additional findings (total findings minus the gold standard contained therein) from the computer-assisted detection process. From these, the actual new quotations are ultimately selected or confirmed by means of hermeneutic close reading.

In order to be able to perform the final evaluation (and classification), the digital preselection — by means of the filters we discuss below — has to be optimized in such a way that the number of “false” found matches is kept as low as possible.

The texts, in the form of .txt files, are subdivided by chapters (for prose) or by verses (for poetry). It is important to emphasize that the analysis process can be carried out with any source text.[13]

These “raw” texts are examined for two-word congruences by means of the script-based tool Simillima. These are pairs in which both the source text and the target text contain at least two (distinct) words in exactly the same word form.[14] The actual comparison of texts is based, in essence, on a “matching of certain units” of target and source text: each unit of Jerome’s corpus of letters, for example, is compared with each unit of the Eclogae. The whole sentence is taken as the matching unit.[15] The output file of the Simillima program contains for each pair a match number, the actual text reference,[16] the respective texts (the shared words being highlighted), and finally the shared words themselves.

The Complura filter returns matches whose shared word material consists of at least four consecutive shared words. The pairs found in this way are at the same time already final (albeit partial) results: due to the accumulation of shared words, they are unambiguous quotations that do not require further automated analysis.[17] Especially in the case of clustered synsemantic words (grammatical words such as conjunctions, subjunctions, prepositions, pronouns, auxiliary verbs) such citations would also not be detectable in the further digital analysis process.[18] An example of this phenomenon is a quotation of Virgil in Jerome’s letter 52, which has been identified by traditional scholars:

The shared words here mainly consist of stopwords that are already present at the very beginning of a Virgil-Epistulae-specific stoplist, with the exception of possumus.[21] Therefore, without passing through the Complura filter routine, this well-known quotation would be sorted out as the analysis proceeds.

Since after the first matching process the number of pairs of sentences is clearly too high for the phase of close reading (for example, the matching of the corpus of Jerome’s letters with the Eclogae yields 408,393 results, that with the Georgica 1,217,441 results, and that with the Aeneid 4,525,752 results), filtering criteria of semantic quality and distance are applied, as already in Revellio (2022).[22] First, synsemantic words that are usually not sufficient for proving a text–text relationship (i.e. a quotation) are removed with the help of stoplists.[23] If two shared words are still contained, their distance is calculated. Then, if a maximum of two words are found between them, the matched pair is kept and written into the output file.[24]

After processing this step the algorithm outputs several result lists (one for each stoplist). One goal in the development of the detection process is to determine the one stoplist for which the highest possible number of actual quotations can be found among returned matches, but for which, at the same time, the surplus is as small as possible.[25] The number of potential new quotations remains comparatively high after this step: for the three mentioned works of Virgil alone, the selection of stoplist 6 (at least 100 occurrences) results in 862 pairs for review. Although the number of potential new quotations can thus be decimated compared to the first matching process, the algorithm proves to be deficient at this stage — especially in view of significantly larger text corpora such as those of late-antique authors.

The Auctoritas filter bundles these potential new findings that are clearly recognizable as intentional citations by explicit reference to authors, works, and the like. Thus, for instance, the phrase sceleratum frigus (the villainous cold) in Jerome’s letter 121 is explicitly marked as a citation because the name Uergilius is mentioned and because there is actually a citation marker in the form of the predicate appellet (calls):

For efficiency reasons, the filter is applied after the separation of sentences and text comparison (Simillima).[26] Similar to the output of the Complura filter, the result list of the Auctoritas filter contains (partially) final results. These are to be classified as quotations due to the explicit reference to the respective source text and therefore do not have to go through any further selection processes.

The “Scissa” filter eliminates potential new quotations whose shared words — in at least one of the two texts (source or target) — do not belong to one syntactic unit. The indicators for separation of the word material are “soft” punctuation marks (comma, colon, and semicolon).[27] Since colons, commas, and semicolons indicate “weaker” syntactic separations than periods, question marks, and exclamation marks, they are not used in Step 1 to separate sentences, in order to prevent the heavy fragmentation of source and target text. Nevertheless, the caesurae marked in this way can — in this phase of more detailed analysis — be used as indicators for syntactic separation of quoted material. This is illustrated by a match resulting from the matching of Jerome’s letters with Virgil’s Georgics as an example:

The shared words (pluuia; laeta)[28] are embedded in an asyndetic structure in Jerome: pluuia is the subject of abiit, while laeta characterizes the subject implicitly contained in respondit. The fact that these shared words are thus bound to two different predicate verbs (which is indicated accordingly by commas in the underlying print edition) rules out a citation.[29] Having been applied, the “Scissa” filter reduced the number of potential new quotations by 28% for the Eclogues, by 22% for the Georgics and by 30% for the Aeneid.[30]

The HTRG filter developed by Revellio is placed after the “Scissa” filter:

Potential findings from computer-assisted text comparison, whose matching word material consists of at least two nouns or two verbs as well as a combination of these two word classes, [are] particularly suited to establish a meaning-producing text-to-text relationship.[32]  [Revellio 2022, 151]

Accordingly, the shared words of each match are analyzed by part-of-speech tagging. Information on whether the combination of shared words satisfies the HTRG rule (N–N; V–V; N–V; V–N) is added to each match. Thus, this filter does not eliminate matches but increases the “resolution” of the output data.[33]

The steps of the digital analysis tools described so far are already performed in an automated way. In what follows, new optimization strategies are presented that are currently in an experimental stage but seem to be target-oriented and promising. By implementing these approaches into the routine, the automation process shall be extended and the amount of work in the close-reading phase shall thus be made manageable by further reducing the number of returned matches.

Short quotations make up only a small proportion of the total amount of quotations of Virgil in Jerome’s corpus of letters, a fact that is evident from analyzing the research-based gold standard of published Virgilian quotations from the Eclogues, Georgics, and the Aeneid. At the same time, precisely these short quotations are the core result of the digital detection process. Because of this discrepancy, the results have to be considered all the more critically. In order to filter out further non-relevant (i.e., false) matches prior to the phase of hermeneutic close reading, further strategies are presented below, derived from the phenomenological characteristics of known and published (i.e., contained in the gold standard) short quotations of Virgil in Jerome. The pairs of quotations discussed below — together with the quotation of Verg. georg. 2,256b–257a in Hier. epist. 121,10,5 discussed in connection with the Auctoritas filter — represent the complete list of Virgilian quotations in Jerome that have two exactly identical word forms and can thus be termed short.[34]

It should be noted that a match with two shared words can in actual fact have other “shared” words of identical form. However, these often belong to the category of synsemantics and are subsequently removed by applying the stoplists. In the case of the Eclogues, Georgica and Aeneid, many of the potential new quotations selected by the algorithm do indeed share other words in addition to two shared words [35], which, however, are mostly irrelevant for the evaluation of the match as a quotation. [36] This disregard of synsemantic words among additional shared words indeed does mimic a “manual” hermeneutic reading but still justifies the use of the term “short” instead of the term “two-word citation” (where it would always need to be stated whether only shared words or all identical word forms are meant).

The presented algorithm successfully outputs the quotation of Verg. ecl. 4,60–63 in Hier. epist. 130,16,3, which is contained in the gold standard, as a match; the shared words (risu; matrem) are retained.[37]

The syntactic relationship of the two nouns, which are in ablative and accusative case respectively, remains unclear at first. Nevertheless, this match can undoubtedly be regarded as an actual quotation. This is confirmed by the surrounding text environment. In it, additional “material” from Virgil’s poem is “recycled” (but not in exactly the same form!): infans paruulus can be traced back to the twice appearing parue puer, but neither the synonym (infans for puer) nor the diminutive (paruulus < paruus) are “visible” for the algorithm. Moreover, the verb cognoscere is present in the target text, albeit in inflected form, which, in both target and source text, forms the syntactic connection between the shared words (matrem: direct object; risu: instrumental ablative).

This results in the optimization strategy of making “invisible” language material, which can be traced back to the target text, “visible” to the algorithm. As the above short quotation shows, this additional material in the target text can be connected to the forms of the source text in three different ways: by lemmatization (cf. cognoscat – cognoscere), by finding synonyms (cf. infans – puer), and by detecting etymologically related forms (cf. paruulus – paruus). Through these approaches more quoted words can be detected. By marking those quotations that consist of only two shared words but have further similar meaningful forms in the same sentence, these filters could significantly support the close reading.

In book 9 of the Aeneid, Iris urges the Rutulian king Turnus to attack the Trojan camp with the words rumpe moras omnis et turbata arripe castra (break off delay, and seize the bewildered camp, 13). The phrase rumpere moras, which is frequently employed by poets, is used by Jerome in letter 130, where he writes: rumpe moras omnes (abandon all procrastination). Hagendahl cautiously counts this passage among the “[l]ess obvious reminiscences of Virgil” and thus points to the difficulty of an exact attribution [Hagendahl 1958].[38]

However, here again the “additional” linguistic material is striking: Jerome does not only write rumpe moras, but strengthens his request — like Virgil in Verg. Aen. 9,13 — with omnes (= omnis [39]). Both instances thus have the same syntactic complexity (predicate + object + attribute adjective). Other instances of the phrase rumpere moras therefore seem to be rather unlikely as pre-texts, such as Verg. Aen. 4,569 (heia age, rumpe moras – Up then, break off delay) and Verg. georg. 3,43 (rumpe moras; uocat ingenti clamore Cithaeron – break with slow delay! With mighty clamour Cithaeron calls), the earliest instance of the phrase.

The following returned match, which cannot be considered a quotation based on these considerations, illustrates ex negativo that recycling syntactic complexity increases the plausibility of a quotation:

It is obvious that despite the presence of two shared words (summo; culmine) there is no reference to Virgil’s line. This is due to the different nature of the prepositional phrases in which the shared words are found. Besides the semantic difference in the prepositions themselves (de vs. in), it is then, above all, the greater syntactic complexity of Jerome’s phrasing; the identical culmine is complemented by an attribute genitive (uirtutum). In particular, the addition of this attribute genitive “activates” the figurative meaning of culmen (summit), whereas in Virgil culmen refers to a roof ridge. [40]

The analysis of syntactic complexity thus highlights a difference of source and target text,[41] which seems to make semantic shifts detectable for the algorithm. Based on these considerations, a filter (to be implemented after part-of-speech tagging) is being developed that analyzes the syntactic structure underlying the shared words and checks whether the syntactic complexity is similar in both source and target text. Increased syntactic complexity in one of the two texts considerably reduces the plausibility of a citation. Additionally, the absence of complexity (i.e., especially the absence of one or more attributes) seems to be an indication that there is no citation, especially in simple adverbial ablative phrases like sole orto and toto orbe. Frequent adverbials of place and time could be eliminated in this way.

A fundamental problem that largely eludes the digital analysis tools is the consideration of the manuscript tradition. The algorithm presented here can only take into account a single wording. For example, for the citation pairs of Hier. epist. 126,2,2/129,4,3 = Verg. Aen. 4,42–43 shown below, the digital analysis produces only two shared words (lateque; Barcaei). In fact, the manuscripts of the Aeneid also give the reading uagantes (roaming) in addition to furentes (raging), which Jerome obviously refers to[42] — and not just in this passage.[43] If uagantes was chosen in the underlying digital text of the Aeneid, the digital analysis process could have detected (depending on the respective stoplist) up to three shared words.

Both pairs of quotations show that especially in the phase of manual-hermeneutical evaluation of potential new quotations special attention must also be paid to the possibility of different readings within the manuscript tradition. Thus, consultation of the critical apparatus is essential.[44] The case of the variants furentes/uagantes shows that divergences within the manuscript tradition can go far beyond similar lemmata and synonyms. Digital texts that preserve the breadth of the manuscript tradition as much as possible and present it in an algorithm-compatible form can thus be seen as an urgent desideratum in the field of digital text analysis. Since the current digital Teubner editions do not (yet) meet these specific needs, we test the definition of minimal deviations in letter sequences as an immediately implementable replacement strategy. It is also possible that this approach will prove to be more efficient from a procedural point of view.[45]

The phrase experto credite (trust the one who has experienced it) from book 11 of the Aeneid is used by Jerome in Hier. epist. 84,3,5 as credite experto.

Due to the shortness of the phrase — and also since it is not the only possible source[46] — this match does not look like an obvious quotation at first glance. Nevertheless, the instance in Jerome is accepted as a quotation of Virgil by scholars. The key indicator for this verdict is the reuse of the quotation, albeit in shorter form: Jerome has already quoted these lines of the Aeneid in full in Hier. epist. 50,4,2. For the evaluation of potential new quotations, this observation results in the strategy of comparing several instances (if any) of short quotations. If a short quotation occurs elsewhere as part of a longer quotation, familiarity with the pre-text seems plausible and thus increases the match’s potential to be an actual quotation.