Annotating, Projecting, and Interpreting Named Entities in Digital Scholarly Editions with LLMs

Annotating, Projecting, and Interpreting Named Entities in Digital Scholarly Editions with LLMs | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Annotating, Projecting, and Interpreting Named Entities in Digital Scholarly Editions with LLMs Selina Galka, Georg Vogeler This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7175875/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 28 Jan, 2026 Read the published version in International Journal of Digital Humanities → Version 1 posted 7 You are reading this latest preprint version Abstract This paper explores the use of large language models (LLMs) to enhance semantic annotation and annotation projection in digital scholarly editions (DSEs), focusing on historical ego-documents. Using the TEI/XML-encoded French memoirs of Countess Luise Charlotte of Schwerin (1684–1732) and their German translation as a case study, we evaluate LLMs for Named Entity Recognition (NER), annotation transfer across aligned bilingual texts, and the extraction of interpersonal relationships. A comparative analysis with a traditional NER framework shows that LLMs significantly outperform baseline models, particularly in recognizing complex person references, such as non-rigid designators and nested entities. For annotation projection, we demonstrate that LLMs can reliably transfer entity annotations between French and German texts without intermediate alignment layers, achieving over 97% of correct projected entities using zero-shot prompting. Additionally, a pilot experiment illustrates the potential of LLMs for structured relationship modeling. The analysis of the errors puts further emphasis on the question of our intentions as editors when translating and indexing texts. Digital Edition Memoirs Named Entity Recognition Annotation Projection Large Language Models Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Digital Scholarly Editions (DSEs) aim to make (historical) sources accessible according to digital paradigms. They often go beyond presenting a single text, encompassing multiple textual variants and, in many cases, translations as well. Especially in the editing of ego–documents such as memoirs, letters, or diaries, considerable effort is devoted to close reading of the text, identifying referenced entities such as persons and places, and encoding them using TEI/XML. Sometimes translations are also included to enhance accessibility and enable comparative analysis. This type of work has also been undertaken in the digital edition of the memoirs of Countess Schwerin (1684-1732), which serves as the basis for the topic discussed in this paper. The project 1 aimed to make the memoirs – originally written in French and transmitted through two distinct manuscripts – accessible in a semantically enriched digital format. The memoirs offer rare, detailed insight into the life of an aristocratic woman in the early 18th century and reveal female networks, agency, and courtly life at Berlin and Vienna, which are otherwise underrepresented in sources from this period (Tersch 2004, 738). They also contribute to an understanding of informal and kinship–based power structures within the Habsburg and Prussian aristocracy. The persons mentioned in the memoirs of Countess Louise Charlotte of Schwerin are of particular interest for several reasons – e.g., because she was personally connected to the Prussian and Habsburg elites, including courtiers, diplomats, clergy, and even the emperor, and describes interactions with other women at court, which give new insight into female influence at court. Following that, all mentioned persons are systematically encoded in the TEI/XML edition, linked to external identifiers where possible, and researched in depth. In the Memoirs of Countess Louise Charlotte of Schwerin, individuals are referenced in diverse and historically significant ways. Many people appear only through indirect expressions like “mon père” (my father) or “ma belle-mère” (my mother-in-law or my stepmother). These indirect references are crucial for understanding the kinship–based and informal networks that structure courtly life. From a theoretical perspective, these indirect mentions correspond to what Saul Kripke (1982) defined as non-rigid designators: expressions whose referent can vary across possible worlds or contexts. Unlike rigid designators, which refer to a specific individual in all possible contexts, non-rigid expressions rely on contextual, relational knowledge. Also, mentioned person names can be quite complex, including various role names or titles, like “monsieur le baron de Danckelmann”. Additionally, named entities may frequently occur as so-called “nested entities” – for example, a person name that includes a place name, such as “madame d'Amtitz”. Although the memoirs of Countess Schwerin are written in French, translating them into German is particularly meaningful given the historical and cultural context. The Countess spent a significant part of her life in Vienna, where German was the dominant language of court, society, and administration. A German translation not only makes the text accessible to a broader regional audience but also aligns with the cultural and historical connections between the content of the memoirs and the German-speaking world. This paper primarily aims to demonstrate and discuss the application of large language models (LLMs) for Named Entity Recognition (NER) on the normalized historical French text of Countess Schwerin’s memoirs. The focus lies on annotating mentions of persons and places, as well as transferring these annotations to a German translation of the text — a process known as annotation projection. In addition, the paper includes a brief example showcasing the potential of LLMs for information extraction, specifically for identifying (implicit) personal relationships mentioned in the text. Theoretical foundations Large Language Models and their (possible) use in digital editions Large language models (LLMs) are advanced neural network architectures trained on vast amounts of textual data to generate, interpret, and manipulate human language. The use of LLMs is currently being explored across a wide range of applications, including those relevant to digital editions (Pollin et al. 2025). This section does not attempt to provide a comprehensive review of the research literature, but rather aims to offer an overview of potential areas of application. One such area is text creation, which encompasses transcription tasks. For example, Humphries et al. (2024) tested LLMs on a corpus of English handwritten documents from the 18th and 19th centuries. Their study demonstrated that LLMs can effectively interpret handwritten content, outperforming established Handwritten Text Recognition (HTR) systems in both accuracy and efficiency, while also being more cost-effective. In addition to transcription, LLMs are also being evaluated for post-OCR processing, where they are used to correct errors and improve the quality of OCR outputs (see e.g., Thomas/Gaizauskas/Lu 2024). Also, the application for normalization is tested (see e.g., Scholger et al. 2025). Another important area is annotation, where best practices are discussed (see e.g., Törnberg 2024), and LLMs are evaluated on specific annotation tasks, such as those in social media research (Pangakis/Wolken 2025). In the context of digital editions, the annotation of named entities – such as persons or places – is often a central focus. The application of LLMs to NER has been the subject of numerous studies, including work focused on historical texts (e.g., Hiltmann et al. 2025, Tudor/Megyesi/Östling 2025, Greif/Griesshaber/Greif 2025, González–Gallardo et al. 2024, Santos et al. 2024). Fang et al. (2025) also demonstrated that LLMs outperform traditional NLP methods on tasks such as word segmentation, part-of-speech tagging, and NER, particularly when applied to historical Chinese texts. Xu et al. (2024) provide a comprehensive review of how generative LLMs are used to extract structured data – such as entities, relations, and events – from unstructured textual input. LLMs can offer notable advantages over traditional information extraction methods or NLP methods in terms of flexibility and generalization. They can perform multiple tasks – such as NER, relation extraction, and event extraction – within a single model architecture using task-specific prompts, thereby eliminating the need for separate task-specific pipelines. Another key benefit can be their capacity for few-shot and zero-shot learning: they can achieve effective performance with minimal labeled data, whereas conventional systems typically require extensive task-specific training. In addition, generative LLMs are capable of producing structured outputs relevant for DSEs (e.g., JSON, TEI/XML or RDF) directly, which facilitates seamless integration into digital edition workflows. Named Entity Recognition Named Entity Recognition (NER) refers to the identification and classification of named entities such as persons, places, organizations, and dates within a text. A variety of methods are available for this task, ranging from traditional rule–based approaches – which rely on regular expressions and gazetteers – to machine learning pipelines such as those implemented in frameworks like spaCy . More recent developments include transformer-based models such as BERT and flair , which have significantly improved performance by leveraging contextual embeddings. Building on this progress, prompt-based NER using LLMs has recently emerged as a possible alternative. Traditional NER approaches often struggle with historical texts due to their lack of standardized spelling and syntax, domain and genre variation, and limited availability of annotated training data (see Ehrmann et al. 2023). Frameworks like spaCy and flair perform well on modern, normalized corpora but are less effective when applied to historical sources. Moreover, Nadeau and Sekine defined in their survey of NER applications and methods, that only referents with rigid designators (as defined by Kripke 1982) are considered in the identification and classification of Named Entities (2007). Since the advent of ChatGPT and other LLMs, interest in zero- and few-shot NER without fine-tuning has increased. However, earlier evaluations (e.g., González-Gallardo et al. 2023; Tang et al. 2024) reported inconsistent or weak performance of LLMs for NER, primarily due to insufficient prompt engineering, a lack of domain adaptation, and inadequate evaluation strategies – as noted by Hiltmann et al. (2025). In their study on Baedeker travel guides for Berlin (1921), Hiltmann et al. systematically evaluated different prompting strategies (zero-, few-, and many-shot) and techniques (e.g., penalization/reward heuristics) using ChatGPT-4o. Their approach employed a proprietary annotation scheme adapted from Wang (2025) and they demonstrated that LLMs can perform NER on historical travel guides with remarkable accuracy. In contrast, Tudor et al. (2025) tested NER on historical newsletters and demonstrated that, despite some success – particularly with multilingual models such as Aya – LLMs frequently produce hallucinated outputs, which limits their reliability. The authors emphasize that while prompt-based LLMs offer notable flexibility and reduce the need for costly annotation, they currently underperform compared to fine-tuned models and require further methodological refinement. It should be noted, however, that the Aya models are open-source models, which facilitates reproducibility and transparency, although they may lag behind in performance due to limited computational resources compared to proprietary alternatives. Annotation Projection Annotation projection is the process of transferring markup from a well-annotated source text to a corresponding target text, such as a translation. Its main motivation is efficiency: once annotations are created in one language, replicating them manually in others is labor-intensive. Yarowsky and Ngai (2001) were early proponents, applying projection for POS tagging from English to French. Bamman, Babeu, and Crane (2010) projected TEI/XML annotations from Ancient Greek and Latin texts to their English translations, aiming to build a multilingual digital library based on curated sources. They transferred structural and named entity markup across over 13,000 tags, achieving 88.2% accuracy using Moore’s sentence aligner and MGIZA++ for word alignment. Ermann, Turchi, and Steinberger (2011) applied annotation projection to multilingual NER corpora. English texts annotated for named entities were translated using PBSMT, with projection performed via string-matching and edit-distance methods. They reported up to 95% recall for Romance and Germanic languages, and over 86% for inflectional languages. Steinbach and Rehbein (2019) developed a modular NLP pipeline to project quotation and speaker annotations from English novels to German translations, using BLEUAlign and fast_align for alignment. Galassi et al. (2019) addressed non-literal translation in legal texts, projecting annotations for unfair clauses between English and German using sentence embeddings and Dynamic Time Warping, yielding accurate and efficient results. Yousef et al. (2023) projected named entity annotations from English to Ancient Greek and Latin via sentence-aligned Bible corpora and neural word alignment with XLM-R. Their heuristic handled varied alignment scenarios, reaching over 82% accuracy. Senator et al. (2025) used ChatGPT for cross-lingual annotation projection and zero-shot annotation in Arabic, focusing on emotion recognition and semantic role labeling. Results showed high accuracy, particularly on simpler texts, with human validation still essential for nuanced content. Numerous other studies have addressed the topic of annotation projection (see, e.g., García-Ferrero/Agerri/Rigau, 2022; Kuchmiichuk 2023; Zahgir et al. 2023; Rai/Chatterji 2023). Many of these approaches involve aligning the source and target languages at the word, sentence, or phrase level, followed by the evaluation of alignment quality (see Behzad et al. 2023, p. 244). In contrast, Chen et al. (2023) adopted a different strategy by first inserting special markers around annotated spans in the source text and subsequently translating the marked text – a method referred to as mark-then-translate . The COMUTE project, running from 2024-2027, funded by the DFG, addresses the challenge of aligning non-parallel multilingual text versions; Levchenko (2024) explored the use of translation alignment algorithms for constructing a Multilingual Digital Edition (MDE) of I promessi sposi , highlighting both the limitations of existing methods in handling literary translations and proposing an automated pipeline and evaluation metrics tailored to multilingual literary corpora. When it comes to tools and frameworks for annotation projection, Akbik and Vollgraf (2018) introduced ZAP , an open-source multilingual framework, which provides methods for preprocessing, computing word alignments, transferring annotations, and visualizing projection results. In contrast to these prior approaches, this study applies LLMs directly to TEI/XML-encoded historical texts and their translation, without intermediate word or sentence alignments, using structured prompts and markup-conforming output. Practical application When it comes to the use of LLMs there are several areas in which users can influence the model’s performance. Nina Rastinger listed these in regard to NER: when it comes to the model itself, you can choose the model, the access type and the model parameters. When it comes to the data, the input characteristics of the data can be adjusted, like input format or amount of input data. A central component is the prompt design, where you can define the output format, perform demonstrations, use consistency strategies, describe the task, prompt the roles or decompose the task. (Rastinger 2024) The following applications involving LLMs were conducted using Claude Sonnet 4 for NER and additionally Llama-4-Maverick-17B-128E-Instruct-FP8 and GPT-4.1 for annotation projection tasks. 2 Previous tests involving NER and the evaluation of three LLMs 3 demonstrated that Claude Sonnet 3.5 consistently delivered the most accurate and reliable results (Galka 2025). This assessment was further supported by the presentations at the workshop “Digital source editions and AI. Current trends, challenges and problems” held in Wolfenbüttel in April 2025, where Claude's performance was also rated highest in comparison with other models. For the use of the LLMs, the TEI/XML files were provided directly as input. Prior experiments had demonstrated that LLMs are capable of interpreting TEI/XML structure and generating corresponding, well-formed XML output. Rastinger (2024) likewise concluded that large language models (LLMs) perform Named Entity Recognition more effectively within the paradigm of annotation than within that of extraction , particularly when the task is framed as a structured markup or editing operation rather than a simple list-based retrieval. The output was returned in TEI/XML format, maintaining consistency with the original schema and facilitating integration into the digital edition workflow. The data of the following experiments can be found in the GitHub-Repository 4 . A comparison of NER with BiLSTM and an LLM NER on the memoirs of the countess was evaluated using both the flair -Framework (Akbik et al. 2019) and the LLM Claude Sonnet 4 , applied to a human annotated 5 sample of 50 paragraphs drawn from the orthographically normalized French source text. The french flair model (ner-french) is based on Flair embeddings and BiLSTM-CRF (Akbik/Blythe/Vollgraf 2018). Data This sample included 444 annotated person entities and 81 annotated place entities. Although the text has undergone orthographic normalization as part of the digital edition process, several historical linguistic challenges persist. These include nested entities – such as place names embedded within person names (e.g., “madame d'Amtitz”) – as well as complex entities involving role titles or honorifics. Nested entities are not annotated as such in the digital edition; instead, they are uniformly tagged as person entities. Particularly noteworthy in the context of ego-documents is the frequent occurrence of non-rigid designators for persons, such as “mon père” or “ma belle-mère” in the memoirs of the countess. Methods First, NER was performed with the available French flair NER model. The model was applied to a TEI-encoded French text via a script that extracts each paragraph, converts it into plain text using the standoffconverter library – which maintains a mapping between plain text and XML structure – and then re-integrates the detected entities back into the TEI as inline tags. All entity spans categorized as MISC were excluded from the evaluation. In a next step, NER was also performed using the LLM Claude Sonnet 4 . The TEI/XML file was used directly as input, as prior small-scale experiments had already demonstrated that LLMs are capable of processing this format effectively. Each paragraph was submitted to the model individually. The prompt instructed the model to tag only person and place names using and elements, respectively, while preserving all other existing TEI structures, including nested tags such as , , and . The prompt contained several illustrative input-output examples (few-shot-prompt), covering both explicit and implicit references (e.g., “ma tante”), as well as compound titles (e.g., “comte de Schwerin”). To facilitate the evaluation of named entity annotations, the annotated TEI/XML-file was first transformed into a plain-text format using a custom XSLT-stylesheet that encodes entities with proprietary markers. This intermediate format was then normalized to ensure consistent tokenization across the dataset. During evaluation, both the system-generated and the human-annotated versions were converted into BIO-labeled 6 sequences, allowing for direct token-level comparison. The seqeval library, specifically tailored for sequence labeling tasks, was employed to compute entity-level precision, recall, and F1-scores based on these BIO–tags. Results NER with flair shows relatively stronger recognition of location entities (see table 1). While many place entities were correctly annotated, components of personal names – such as “Schwerin” in “Comte de Schwerin” – were frequently annotated as LOC. Other sources of error were rare. In contrast, the model struggles significantly with person entities – especially with complex names, e.g., excluding roles or titles and missed most of the entities with non-rigid designators. 150 entities were identified and classified as MISC – these annotations contained actual person names (rigid or sporadic with non-rigid designators), religious entities (“Dieu”) and many text spans which do not include any entities and were not included in the evaluation. Regarding Claude , all tags from the input document were preserved in the LLMs annotated TEI/XML output. Minor alterations were observed at the textual level: apostrophes were consistently replaced with a typographically distinct variant, and in a few isolated cases, expressions such as “du comte de Schwerin” were reformulated as “de le comte de Schwerin.” Despite these minor shifts, no content was lost in the transformation. The NER process using Claude achieved high performance in detecting location entities; in comparison the recognition of person entities yielded lower metrics (see table 1), but it is significantly better than the NER with flair. A manual inspection revealed that the actual performance of the model is also even better than the strict evaluation metrics suggest. This discrepancy arises primarily from the evaluation method’s sensitivity to exact string matches, which does not account for minor surface variations such as the inclusion or omission of definite articles. A substantial portion of the mismatched entities stem from cases where the gold standard includes forms such as “le comte de Schwerin”, whereas Claude annotated simply “comte de Schwerin”, or vice versa. It should also be noted that such variations are not standardized within the ground truth itself. Table 1 NER-Results flair Claude Sonnet 4 LOC Precision 0.50 0.89 LOC Recall 0.72 0.91 LOC F1-score 0.59 0.90 PER Precision 0.30 0.72 PER Recall 0.19 0.74 PER F1-score 0.23 0.73 Discussion The comparative evaluation of NER using the flair framework and the LLM Claude Sonnet 4 demonstrates a clear performance gap, particularly in the recognition of person entities. While flair achieved modest results in recognizing location entities, its performance with person entities was significantly weaker. This discrepancy can largely be attributed to the model’s limitations in handling nested structures, complex honorifics, and non-rigid designators. In contrast, Claude Sonnet 4 demonstrated much higher overall precision and recall for both person and location entities. Additionally, the LLM’s performance with indirect references (e.g., “ma belle-mère”), compound titles and nested entities suggests a greater contextual understanding. However, even the Claude model’s performance was not flawless. The evaluation revealed that many discrepancies resulted from minor formal mismatches – such as the presence or absence of definite articles – which had a disproportionately negative effect on automated metrics like F1-score. Claude Sonnet clearly outperformed flair in accuracy and structural fidelity, confirming that well-prompted LLMs offer a strong alternative to traditional NER pipelines. While prompt design and edge cases remain challenging, the results suggest LLMs are well-suited for annotating historical, referentially dense texts in digital scholarly editions. Annotation Projection with Large Language Models Annotation Projection has not been a major area of research within DSEs so far, but it can offer significant benefits and potentially save considerable annotation and/or programming effort. Additionally, there is potential to simplify the task – for example, prior sentence or word alignment may not be necessary to perform annotation projection with LLMs; although some kind of previous alignment (e.g., paragraph–based) could probably increase the performance. During the translation process in the project, the German text was already aligned with the French original on a paragraph-by-paragraph basis. In the context of digital scholarly editions, human-controlled translations are generally preferred over purely machine-generated ones, as they allow for greater sensitivity to nuances and historical context. Source texts often already exhibit a structural organization – such as chapters, sections, or paragraphs – which can already be mirrored in the translation-process, facilitating both alignment and annotation. Data The dataset consists of the normalized French text, richly encoded in TEI/XML with extensive structural and semantic markup (e.g., named entities, structural tags, text versions), and a corresponding German translation. The French text was normalized only in regard to orthography, but not in wording, since eighteenth-century French exhibits a linguistic stage that is largely comparable to modern French. The translation is aligned with the French source text at the paragraph level and is likewise encoded in TEI/XML, although it contains no additional annotations beyond the paragraph structure. The German version strives to remain as close as possible to the French original in terms of content, while rendering it in fluent and readable contemporary German. While it is important to preserve the historical character of the source, the translation deliberately avoids imitating historical German. As a result, certain deviations from the French sentence structures are necessary, since a direct one-to-one transfer is often not feasible and not all nuances can be retained. A literal translation would therefore be inadequate. For the purposes of this study, 101 paragraphs from the normalized and extensively annotated French text of the Memoirs of Countess Schwerin were used as a basis for projecting person and place annotations onto the corresponding German translation. The French normalized text also records the differences between Manuscript A and Manuscript W using and elements. However, the German translation is not yet complete, not annotated and has not undergone a final round of corrections up to this point. Furthermore, no final decision has been made on how to handle the textual variants in the German translation. Due to the large number of textual variants, no automatic machine translation method could be applied so far, which leads to convincing results. Also, the focus of the DSE was not on a literal translation, but rather on providing a modern, easily readable version. Therefore, the annotations of the textual variants were not taken into account for the annotation projection, and the “rough” human translation 7 was used. The French normalized and annotated text sample contained 761 annotated person entities and 104 annotated place entities, already linked to a project-specific index file that also records authority data such as GND or GeoNames identifiers. Additionally, the French source text contained work entities (), references to the transcription of MS W (), annotations of person relationships (), and annotations of dates (). While these elements were present in the TEI/XML input, they were not taken into account in the annotation projection process. Method In order to automate the transfer of named entity annotations (persons and places) from a French TEI/XML edition to its German counterpart, three LLMs were tested and compared: Claude Sonnet 4 , Llama-4-Maverick-17B-128E-Instruct-FP8 , GPT 4.1 and Qwen 3 - 14B . Claude Sonnet 4 (Anthropic) is a commercially available model offered with a 200K–token context. GPT-4.1 (OpenAI) is a model available through paid API and ChatGPT Plus, supporting up to 1 million tokens and Llama-4-Maverick-17B-128E-Instruct-FP8 is an open–source model from Meta. The Llama -model is significantly smaller, using only 17 billion active parameters, whereas Claude Sonnet 4 and GPT-4.1 are larger, dense models. The annotation projection was also tested with OLMo-7B-Instruc t – currently one of the most open open-source models – but the results were unsatisfactory and significantly below those of other models evaluated. 8 After loading the French and German TEI documents, all paragraph elements () from both texts are extracted and then iterated through them in parallel. For each paragraph pair, a prompt is constructed that includes the French paragraph in its original, annotated TEI/XML form, along with the unannotated German TEI/XML–paragraph. The prompt instructs the model to replicate the person and place annotations from the French source text in the German version, including the corresponding references (i.e., @ref attributes containing project-specific identifiers), while preserving the exact wording of the German translation. In this experiment, the initial results obtained using a zero-shot prompt with Claude Sonnet 4 were of such high quality that no further prompt optimization was pursued. This means that no examples are given in the prompt; rather, the model is expected to generalize the task based solely on the instruction. The models were configured with a temperature of 0.0 to make it as deterministic as possible and avoid hallucination of entities or structure. The exact prompt used e.g., for Claude was as follows 9 : The output is validated based on two criteria: first, whether all annotations have been correctly transferred – including the links to the appropriate entries in the index files – and second, whether they have been applied to the correct spans in the text. First the annotations between the French and the German TEI/XML file were compared. A script extracts all entity references based on tags and @ref-attributes and records any entities missing a @ref, counts valid entity occurrences, and aligns corresponding paragraphs across both languages. For each shared paragraph, it identifies discrepancies, such as missing or extra references in the German version compared to the French source. The discrepancies are reviewed manually and a manually reviewable HTML interface is generated, in which annotated entities in both languages are displayed side by side. When hovering over an entity, corresponding entries in the other language are highlighted. The application of the projected annotations to the correct spans in the German translation was manually reviewed, as no ground truth reference was available for evaluation. Results The models reliably followed instructions and produced well-structured, valid TEI/XML annotations without requiring additional examples or iterative refinement. The annotated output was validated with regard to text integrity and annotation accuracy. A comparison of the annotated paragraphs from Claude revealed minor and punctual discrepancies in six cases between the source and target text, including missing, altered and additional words (e.g., in P.39: "Jesus." vs. "Jésus." or in P.88: "ewährt" vs. "Gewährt"). The output–text from GPT differed in six small cases from the TEI/XML-input text. In the output generated by Llama, a greater number of textual modifications were observed – approximately 36 instances – including changes to punctuation as well as, in some cases, longer text passages. The French annotated input text contained 865 annotated entities (761 persons and 104 places). The following table presents the discrepancies found in the German annotated output in comparison to these French annotations: Table 2 Annotation Projection Results Claude Sonnet 4 Llama 4 Maverick GPT 4.1 Qwen 3 - 14B False Negatives (Persons) 16 38 20 86 False Negatives (Places) 6 6 7 13 False Positives (Persons) 6 57 26 1 False Positives (Places) 1 9 10 0 Additional tags 6 (, , ) 43 (, , , , , <seg) 2 () 20 (, , , , ) Valid TEI/XML yes yes yes yes Text lost/changed yes, but very little (6 small adjustments) yes, about 36 changes yes, 9 small adjustments yes, including corrections of mistakes (e.g. inserting missing whitespaces) Regarding Claude, out of 865 annotated French entities (761 persons and 104 places), a total of 22 references were not successfully transferred to the German version – 16 person and 6 place annotations are missing in the annotated German output of the LLM. In contrast, the German annotated output text contains 7 additional entity annotations (6 persons and 1 place) not present in the French original. This corresponds to a transfer rate of over 97% for person and place annotations from the French source text to the German translation. A manual inspection confirmed that all transferred annotations were applied to the correct spans in the German text. In the few cases where discrepancies occurred, they typically involved minor variations such as the inclusion or omission of definite articles – an inconsistency that also appears in the French source and does not significantly affect semantic interpretation. Additionally, all @ref–attributes containing the project-specific identifiers for the entities have been transferred correctly. The following examples illustrate effective annotation projection from French source texts to their German translations, particularly in semantically and structurally complex cases. “P.xx” refers to the paragraph number or the corresponding @xml:id in the TEI–encoded text: P. 102, French: la demoiselle de Wylich–Lottum / German: Fräulein von W. P. 14, French: ce gouverneur / German: diesen Hofmeister / Erzieher / Lehrer P. 16, French: un des premiers de ce lieu / German: einem der der Vornehmsten dort P. 18, French: un homme d’un plus grand caractère qu’ / German: ein Mann mit einem höheren Rang P. 28, French: une fille fort fidèle qui m’avait élevée / German: sehr zuverlässigen Mädchen, das mich erzogen hatte P. 62, French: la plus belle terre dans le royaume / German: das schönste Landgut im Königreich P. 97, French: celle qui était dans la maison / German: jener, die im elterlichen Hause verblieb The following observations concern instances where persons annotations were not projected from the French source text to the German translation, along with several cases of additional or spurious annotations. Five instances were not projected due to complex structures, particularly involving and elements in the French text, which are only partially reflected or simplified in the German translation (e.g., P.81). One instance involved a shift from a named entity (“ comte de Schwerin” ) to a pronominal reference (“ dessen” ) in German, impeding direct annotation transfer (curved brackets indicating annotations and bold highlighting indicating absent annotations): French annotated text: Il fallut lui donner toutes mes lettres du {comte de Schwerin} pour les porter à {mon père}, afin qu’il jugeât de ce que le {comte de Schwerin} avait dit de {ceux qui m’élevaient}, les pouvait toucher. German human translation for the edition, annotated with Claude: Ich musste ihr alle meine Briefe des {Grafen von Schwerin} übergeben, damit sie sie {meinem Vater} bringen und dieser beurteilten konnte, ob dessen Äußerungen über {diejenigen, die mich aufgezogen hatten}, sie betreffen könnten. Another case in P.76 featured the pronoun “er” in place of the explicit mention “der Graf von Schwerin”. In this case one annotation was missing and one incorrectly added – “der Graf von Schwerin” was tagged even though the reference was to his father: French annotated text: Après quoi {le comte de Schwerin le père} vint me faire un grand compliment et m’embrassa comme sa chère fille [...] German human translation for the edition, annotated with Claude: Der Vater des {Grafen von Schwerin} kam herein, überschüttete mich mit Komplimenten und umarmte mich, als wäre ich seine eigene Tochter. In one case in P.63 “la maison de mon père” was rendered as “mein Elternhaus”, thus eliding the explicit person reference. In one case in P.83 the model correctly refrained from projection: the German text erroneously mentions “der Graf von Schwerin” instead of “die Gräfin von Schwerin” (the translation has not undergone a final correction round yet). In another case in P.96 the phrase “ministres du Roi l’acc” (two entities) was only partially rendered in German as “königliche Beamte”, tagged as a single generic entity. In another case (P.98) “mon beau–frère” is omitted in the translation and in P.83 “der Graf von Schwerin” was not projected; it appears in square brackets in the German translation. In two cases the entity is not mentioned in the german text. Among the seven additional annotations identified in the German translation, four are attributable to duplicated sentences – passages in which different versions of the translation coexist. In these cases, it remains to be determined which version will be retained as the final target segment, a decision that will directly impact the annotation projection. Only two annotations were incorrectly added in the LLM-generated TEI/XML output. An inspection of the six place annotations that were not projected from the French source text to the German translation reveals the following pattern: in four of the six cases, the respective place names (“Wesel”, P.22; “Berlin”, P.31; “Schlobitten”, P.89; “Berlin”, P.92) are simply absent in the German translation. A representative example is found in paragraph P.22, where “Wesel” appears twice in the French source but only once in the German version, preventing full projection: French annotated text: Nous retournâmes à Wesel peu de jours après à Wesel et feu S.M. le Roi de Prusse y vint avec toute sa Cour. German human translation for the edition, annotated with Claude: Wenige Tage später kehrten wir nach Wesel zurück, und Seine verstorbene Königliche Hoheit, der König von Preußen kam mit seinem gesamten Hof dorthin. Another one was a very complex case (“y” from the french was not projected to “wo”, engl. “where”) in the german translation: French annotated text: Le {père} de {ma belle–mère} {y} était gouverneur. German human translation for the edition, annotated with Claude: [...] wo der {Vater} {meiner Stiefmutter} Gouverneur war. The last missing annotation was the following, where “Pologne” has been translated with “polish conditions”: French annotated text: Dieu me préserve des modes de Pologne. German human translation for the edition, annotated with Claude: Gott bewahre mich vor solchen p[olnischen] Zuständen. This section analyzed the annotation projection results achieved with Claude Sonnet 4 , which showed the highest performance among the compared models. Most discrepancies stemmed from complex alignment issues, such as structural mismatches, paraphrased expressions, or omitted entities in the translation. Compared to Claude , the annotation projection results with Llama were notably lower. False Negatives included simple correspondences like "Mon père perdit ma mère [...]." and “Mein Vater verlor meine Mutter [...]” (P.6). that were left unannotated in the german output from the LLM, but also cases where entities like “mon beau–père une des sœurs” (two entities) in the source were only partially tagged in the target as one entity. False Positives included both plausible annotations that were just not considered in the French text, but also implausible annotations. Additionally, more inconsistencies in the tagging of the actual text spans can be observed, like in some cases only “Tante” instead of “meine Tante” has been annotated. Otherwise, the model struggled with the same cases as Claude , but also did not hallucinate entities that do not appear in the translation because of rephrasing. Overall, Llama ’s projections were less consistent and accurate than those produced by Claude, but it also has to be noted that the model is smaller, and also open-source. The results of the annotation projection using GPT-4.1 positioned its performance between that of Claude and Llama. Overall, GPT performed well, successfully projecting a substantial number of annotations and correctly applying them to the appropriate text spans – often including articles, pronouns, or capturing longer and more complex entities. Regarding false positives, GPT’ s output resembled that of Llama , occasionally annotating terms such as “Hof” or “Landleben” (P.11), as well as religious references like “Gott” or “Jesus” (P.5) which were not annotated in the original French input text. Regarding false negatives, GPT exhibited similar patterns as observed with the other models, particularly in cases involving rephrasings, missing entities in the translation, or merged entities – such as annotating “mon mari le comte de Schwerin” as a single entity rather than two, like in the French annotated input. The annotations transferred from the French annotated source text to the German translation by the LLM Qwen were mostly correctly tagged (see e.g. figure 5). The model inserted 20 additional tags from the French text, such as or . In comparison to the other models, it did not transfer a significantly higher number of person annotations (101), but it produced also markedly fewer false positives – only one was recorded. The projections that were not transferred also included many simple instances, such as “mon père”, “cette fille”, and “ma tante”. In some cases, the annotation was transferred to a completely incorrect text span—for example, in paragraph P.34, the ID for the aunt was assigned to the phrase “mich verlobt hätte” (engl. “would have engaged me”). Discussion As in many semi–automatic annotation workflows, a human-in-the-loop remains essential to review, adjust, and validate projected annotations – particularly in cases of missing content, complex textual variation, or ambiguous translation. Nonetheless, the evaluation demonstrates that annotation projection using a LLM can be highly effective. E.g., using Claude, over 97% of the annotated person and place entities in the French source text were successfully transferred to the German translation to the correct spans. These strong results are especially notable given that the translation is not literal but often paraphrased or adapted for fluency. Three recurring patterns account for most of the remaining unprojected annotations: Variant text spans : The French source text sometimes contains multiple textual variants encoded with parallel structures (e.g., and ), while the German translation retains only one version. This structural mismatch makes the full projection of all annotated variants not necessary and impossible. Simplified renderings for readability : In several cases, entities are not projected onto the German version due to minor rephrasings or deliberate omissions aimed at improving the translation’s naturalness or readability. Paraphrased entities : The German translation occasionally expresses the same referent using reworded or generalized phrases. While such paraphrases often maintain semantic equivalence, they do not support direct one–to–one annotation projection, and a few references were not captured. Beyond these particular complexities, the models reliably followed the annotation schema and produced well-structured, valid TEI/XML output with minimal intervention. Only two annotations were incorrectly added, and all projected spans were aligned correctly, with only minor deviations (e.g., the inclusion or omission of articles). Importantly, all @ref attributes containing project–specific identifiers were preserved during transfer. In sum, LLM-based annotation projection provides a powerful method for accelerating bilingual corpus annotation. While it is often noted that human involvement remains essential for many tasks related to digital editions, this may not necessarily apply in the present case, as even human input is prone to minor errors. Greater control and refinement are especially required when employing smaller or open-source models such as Llama . The challenges observed for the LLMs point into a different future field of discussion: editorial self-reflection. What are our intentions in the translations and the annotations? Do we prefer readability and naturalness over text surface based semantic annotations? Should we replace annotation with information extraction? Extraction of interpersonal relationships In this small trial, a prompt-based extraction strategy was employed to derive interpersonal relationships from a TEI/XML-encoded paragraph (P.86) 10 of the French normalized text. The source material, enriched with elements containing @ref attributes, which refer to the @xml:id of the entries in the person index, was processed using Claude Sonnet 4 (the Chat–Bot) to generate RDF triples. 11 The prompt instructed the model to identify relationships between the narrator – the Countess of Schwerin – and the entities mentioned in the text. The aim was to structure the information contained in the text into interoperable semantic data. The specific prompt used for this task was as follows: Please extract person relationships from this TEI/XML paragraph in RDF format. Use the IDs from the -Elements and also store the names of the persons and the gender. The narrator is the countess of Schwerin. For relationships, which are true in both directions (like siblings) only state the triple once. For in-law relationships use mother-in-law or father-in-law. Please also provide descriptions of the relationships you used: {french paragraph} The results of the chat can be accessed in the GitHub repository. The extracted data comprises named person entities annotated with FOAF (foaf:Person) types, along with literal values for their names and gender, and descriptive comments offering biographical context. Relationships among these individuals are encoded using the Relationship Ontology and a custom property (ex:livesWithOf), capturing various interpersonal ties including marital (rel:spouseOf), sibling (rel:siblingOf), in-law (rel:childInLawOf), and cousin (rel:cousinOf) relations. A human inspection of the paragraph reveals the following relationships: P153 (the countess, the narrator) is the spouse of P162 (the Count of Schwerin, Friedrich Wilhelm) P158 (Quadt Wickradt, Ermgardt Maria of; married name Countess of Schwerin) is the spouse of P169 (Count of Schwerin, Otto) P158 (Quadt Wickradt, Ermgardt Maria of) is the mother–in–law of the countess P169 (Count of Schwerin, Otto) is the father–in–law of the countess P170 (Countess of Schwerin, Ottonette Wilhelmine) is the sister of P162 (Count of Schwerin) P155 (Countess of Schwerin, Dorothea Maria) is the sister of P162 (Count of Schwerin) P399 (Wittenhorst-Sonsfeld, Hermine Alexandrine of) is the sibling of P205 (Wittenhorst-Sonsfeld, Dorothea Luise von) P399 (Wittenhorst-Sonsfeld, Hermine Alexandrine of) is the cousin of P162 (Count of Schwerin) P208 (Wylich-Lottum, Miss of) is the cousin of P162 (Count of Schwerin) A manual inspection of the RDF output generated by the LLM confirms the accuracy and completeness of the relationship extraction. All major interpersonal ties expressed in the TEI/XML paragraph have been correctly identified and semantically modeled in general (see figure 6): However, it is important to underscore that some of the properties employed – such as “cousinOf” or “childInLawOf” – are not part of the official Relationship Ontology. Although the relationships have been modelled accurately, aligning them with an established ontology remains a task that requires human editorial intervention. Nevertheless, this process might be facilitated through the use of targeted prompting strategies or by enriching the modelling phase with domain-specific information on relationship ontologies. Conclusion This paper has shown that LLMs, though originally developed for text generation, can be effectively repurposed for tasks central to digital scholarly editing. Numerous studies have demonstrated that LLMs often outperform traditional state-of-the-art NLP methods in many of these tasks. Moreover, they tend to be significantly easier to use. In the context of digital editions, which frequently rely on TEI/XML encoding, applying NLP techniques presents additional challenges. TEI/XML is not a standard input format for most NLP tools, necessitating substantial preprocessing and the involvement of pipelines to reintegrate NLP outputs back into the structured XML data. The application of LLMs to NER on 18th-century French TEI/XML texts has yielded promising results. In particular, the task of annotation projection was executed with remarkable results, only using zero-shot prompting, demonstrating the model's ability to transfer complex, semantically rich annotations across languages while preserving structural and contextual integrity; even though the target text was not a literal translation. However, LLMs also offer other new challenges. LLM output must be validated for tag integrity and text preservation, as subtle changes in wording, punctuation, or markup may occur or elements may be lost. Second, while LLMs can reduce the manual workload involved in annotating and projecting semantic content, the role of the human editor remains indispensable for verification and quality control. Strategies, how to make this efficient, are not yet available: Are we doing everything twice? Are we creating harder work for humans, similar to post-editing of machine translation (e.g., Tora et al. 2018)? Do we have to reconsider our own intentions in the editorial tasks? For instance, it is important to emphasize that, although considerable effort is devoted to textual annotation within digital editions, in the domain of ego-documents, the majority of editorial time is spent on interpretation, in particular identifying entities such as named persons (i.e., entity linking). Third, the use of LLMs in scholarly contexts raises broader technical and ethical considerations. These include the opacity of model decisions, the reliance on commercial APIs, the potential for output drift across model versions, and the carbon cost of large-scale inference. This study has not addressed the challenge of entity linking, but it points to a productive direction for further research. Supplying models with curated contextual sources could improve their ability to suggest relevant associations, especially when tracing historical actors and their relationships. In conclusion, this paper has explored the potential of large language models (LLMs) as tools for semi-automated annotation in digital scholarly editions. Rather than assuming that such models "understand" historical texts in a human sense, we suggest that their usefulness lies elsewhere: in their capacity to generate plausible linguistic associations that can be productively harnessed within human-guided editorial workflows. When carefully integrated into such frameworks, LLMs may indeed support and extend editorial practice – not by replacing interpretation, but by provoking it. Declarations Acknowledgements and Credits Selina Galka conceptualized and performed all experiments, did the background research, and wrote the text. Georg Vogeler contributed to the conceptualization and minor parts of the text. We would like to thank Suzana Sagadin for carefully reading the manuscript, providing valuable comments, and assisting with the setup of the LLMs Qwen and Olmo . We are also grateful to Ines Peper for her insightful content-related feedback on the source material. This research is part of the FWF project “ ‘ Tout Vienne me riait ’. Family and court relations in the memoirs of Countess Louise Charlotte von Schwerin (1684−1732)” (PI Ines Peper), Grant-DOI: 10.55776/P34943. Project blog: https://memoiren.hypotheses.org/. Data Availability Data that support the findings of the this study can be find in this GitHub-Repository: URL: https://github.com/selinagalka1/dses_and_llms References Akbik, Alan, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. ‘FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP’. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations) , edited by Waleed Ammar, Annie Louis, and Nasrin Mostafazadeh. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-4010. Akbik, Alan, Duncan Blythe, and Roland Vollgraf. 2018. ‘Contextual String Embeddings for Sequence Labeling’. In Proceedings of the 27th International Conference on Computational Linguistics , edited by Emily M. Bender, Leon Derczynski, and Pierre Isabelle. Association for Computational Linguistics. https://aclanthology.org/C18-1139/. Akbik, Alan, and Roland Vollgraf. 2018. ‘ZAP: An Open-Source Multilingual Annotation Projection Framework’. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) , edited by Nicoletta Calzolari, Khalid Choukri, Christopher Cieri, et al. European Language Resources Association (ELRA). https://aclanthology.org/L18-1344/. Bamman, David, Alison Babeu, and Gregory Crane. 2010. ‘Transferring Structural Markup across Translations Using Multilingual Alignment and Projection’. Proceedings of the 10th Annual Joint Conference on Digital Libraries , 11–20. https://doi.org/10.1145/1816123.1816126. Behzad, Shabnam, Seth Ebner, Marc Marone, Benjamin Van Durme, and Mahsa Yarmohammadi. 2023. ‘The Effect of Alignment Correction on Cross-Lingual Annotation Projection’. Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII) , 244–51. https://doi.org/10.18653/v1/2023.law-1.24. Chen, Yang, Chao Jiang, Alan Ritter, and Wei Xu. 2023. ‘Frustratingly Easy Label Projection for Cross-Lingual Transfer’. In Findings of the Association for Computational Linguistics: ACL 2023 , edited by Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-acl.357. COMUTE Project. Funded by the DFG. Running from 2024-2027. URL: https://www.comute-project.de/projekt/index.html Ehrmann, Maud, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, and Antoine Doucet. 2023. ‘Named Entity Recognition and Classification in Historical Documents: A Survey’. ACM Computing Surveys 56 (2): 27:1-27:47. https://doi.org/10.1145/3604931. Ehrmann, Maud, Marco Turchi, and R. Steinberger. 2011. ‘Building a Multilingual Named Entity-Annotated Corpus Using Annotation Projection’. Paper presented at Recent Advances in Natural Language Processing. https://www.semanticscholar.org/paper/Building-a-Multilingual-Named-Entity-Annotated-Ehrmann-Turchi/5f55aa811875e9789aad1c30b41ee819db4c4f15. Fang, Zhao, Liang-Chun Wu, Xuening Kong, and Spencer Dean Stewart. 2025. ‘A Comparative Analysis of Word Segmentation, Part-of-Speech Tagging, and Named Entity Recognition for Historical Chinese Sources, 1900-1950’. Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities , 1–6. Galassi, Andrea, Kasper Drazewski, Marco Lippi, and Paolo Torroni. 2020. ‘Cross-Lingual Annotation Projection in Legal Texts’. Proceedings of the 28th International Conference on Computational Linguistics , 915–26. https://doi.org/10.18653/v1/2020.coling-main.79. Galka, Selina. 2025. Blog Post: Named Entity Recognition using Large Language Models – A comparison. Project Blog: The Memoirs of Countess Schwerin. 2025. Accessed 2025-07-09. https://doi.org/10.58079/146m.j García-Ferrero, Iker, Rodrigo Agerri, and German Rigau. 2022. ‘T-Projection: High Quality Annotation Projection for Sequence Labeling Tasks’. Version 2. https://doi.org/10.48550/ARXIV.2212.10548. GitHub-Repository for this paper. URL: https://github.com/selinagalka1/dses_and_llms/tree/main . González-Gallardo, Carlos-Emiliano, Emanuela Boros, Nancy Girdhar, Ahmed Hamdi, Jose G. Moreno, and Antoine Doucet. 2023. ‘Yes but.. Can ChatGPT Identify Entities in Historical Documents?’ 2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL) , June, 184–89. https://doi.org/10.1109/JCDL57899.2023.00034. González-Gallardo, Carlos-Emiliano, Hanh Thi Hong Tran, Ahmed Hamdi, and Antoine Doucet. 2024. ‘Leveraging Open Large Language Models for Historical Named Entity Recognition’. In Linking Theory and Practice of Digital Libraries , edited by Apostolos Antonacopoulos, Annika Hinze, Benjamin Piwowarski, et al., vol. 15177. Lecture Notes in Computer Science. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-72437-4_22. Greif, Gavin, Niclas Griesshaber, and Robin Greif. 2025. Multimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents . Version 1. https://doi.org/10.48550/arxiv.2504.00414. Hiltmann, Torsten, Martin Dröge, Nicole Dresselhaus, et al. 2025. ‘NER4all or Context Is All You Need: Using LLMs for Low-Effort, High-Performance NER on Historical Texts. A Humanities Informed Approach’. arXiv:2502.04351. Preprint, arXiv, February 4. https://doi.org/10.48550/arXiv.2502.04351. Humphries, Mark, Lianne C. Leddy, Quinn Downton, et al. 2024. ‘Unlocking the Archives: Large Language Models Achieve State-of-the-Art Performance on the Transcription of Handwritten Historical Documents’. SSRN Scholarly Paper 5006071. Social Science Research Network, October 24. https://doi.org/10.2139/ssrn.5006071. Kripke, Saul. 1982. Naming and Necessity. Harvard University Press, Cambridge. Kuchmiichuk, Pavlo. 2023. ‘Silver Data for Coreference Resolution in Ukrainian: Translation, Alignment, and Projection’. Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP) , 62–72. https://doi.org/10.18653/v1/2023.unlp-1.8. Levchenko, Maria. 2024. ‘Automatic Translation Alignment Pipeline for Multilingual Digital Editions of Literary Works’. CHR 2024: Computational Humanities Research Conference , December 4–6 (Aarhus, Denmark), 1086–104. Nadeau, David, and Satoshi Sekine. 2007. ‘A Survey of Named Entity Recognition and Classification’. Lingvisticæ Investigationes 30 (1): 3–26. https://doi.org/10.1075/li.30.1.03nad. ner-french, french flair model. URL: https://huggingface.co/flair/ner–french. Pangakis, Nick, and Sam Wolken. 2025. ‘Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI’. Proceedings of the International AAAI Conference on Web and Social Media 19 (June): 1471–92. https://doi.org/10.1609/icwsm.v19i1.35883. Pollin, Christopher, Franz Fischer, Patrick Sahle, Martina Scholger, and Georg Vogeler. 2025. „When it was 2024 - Generative AI in the field of digital scholarly editions“. Zeitschrift für digitale Geisteswissenschaften , 10. https://doi.org/10.17175/2025_008. Rai, Pooja, and Sanjay Chatterji. 2023. ‘Annotation Projection-Based Dependency Parser Development for Nepali’. ACM Transactions on Asian and Low-Resource Language Information Processing 22 (2): 1–19. https://doi.org/10.1145/3542696. Rastinger, Nina. 2024. Named Entity Recognition with LLMs. Presentation at the workshop series of the “DHd–AG Angewandte Generative KI in den Digitalen Geisteswissenschaften” (AGKI–DH). URL: https://agki–dh.github.io/pages/webinar/page–7.html. Relationship Ontology. URL: https://vocab.org/relationship/. Santos, Joaquim, Helena Freire Cameron, F. Olival, Fátima Farrica, and Renata Vieira. 2024. ‘Named Entity Recognition Specialised for Portuguese 18th-Century History Research’. Paper presented at International Conference on Computational Processing of the Portuguese Language. https://www.semanticscholar.org/paper/Named-entity-recognition-specialised-for-Portuguese-Santos-Cameron/b1efd28f5ae7faeaff6e01a640497ac15d9d8028. Scholger, Martina, Elisa Cugliana, Franz Fischer, Christopher Pollin, Patrick Sahle, and Georg Vogeler. 2025. „Bias Und Evaluation – Beiträge Zur Kritik Der Digitalen Edition Mit Generativer KI“. Workshop: Digitale Quelleneditionen und KI – Aktuelle Tendenzen, Herausforderungen und Probleme, Wolfenbüttel, 10. April 2025, accessed 2025-07-12. https://zenodo.org/records/15299432. Senator, Ferial, Abdelaziz Lakhfif, Imene Zenbout, Hanane Boutouta, and Chahrazed Mediani. 2025. ‘Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection’. IEEE Access 13: 3707–25. https://doi.org/10.1109/ACCESS.2025.3525493. seqeval. URL: https://pypi.org/project/seqeval/ . spaCy. URL: https://spacy.io/ . standoffconverter. URL: https://github.com/standoff–nlp/standoffconverter. Steinbach, Uli, and Ines Rehbein. 2019. ‘Automatic Alignment and Annotation Projection for Literary Texts’. In Proceedings of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature , edited by Beatrice Alex, Stefania Degaetano-Ortlieb, Anna Kazantseva, Nils Reiter, and Stan Szpakowicz. Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-2505. Tang, Xuemei, Zekun Deng, Qi Su, Hao Yang, and Jun Wang. 2024. ‘CHisIEC: An Information Extraction Corpus for Ancient Chinese History’. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , edited by Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue. ELRA and ICCL. https://aclanthology.org/2024.lrec-main.283/. Tersch, Harald. 2004. ‘Frühneuzeitliche Selbstzeugnisse’. In Quellenkunde der Habsburgermonarchie (16.-18. Jahrhundert): Ein exemplarisches Handbuch , edited by Josef Pauser, Martin Scheutz, and Thomas Winkelbauer, vol. 44. MIÖG Ergänzungsband. Böhlau Verlag. Thomas, Alan, Robert Gaizauskas, and Haiping Lu. 2024. ‘Leveraging LLMs for Post-OCR Correction of Historical Newspapers’. In Proceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-COLING-2024 , edited by Rachele Sprugnoli and Marco Passarotti. ELRA and ICCL. https://aclanthology.org/2024.lt4hala-1.14/. Toral, Antonio, Martijn Wieling, and Andy Way. 2018. ‘Post-Editing Effort of a Novel With Statistical and Neural Machine Translation’. Frontiers in Digital Humanities 5 (May). https://doi.org/10.3389/fdigh.2018.00009. Törnberg, Petter. 2024. ‘Best Practices for Text Annotation with Large Language Models’. Sociologica 18 (2): 2. https://doi.org/10.6092/issn.1971-8853/19461. Tudor, Crina, Beata Megyesi, and Robert Östling. 2025. ‘Prompting the Past: Exploring Zero-Shot Learning for Named Entity Recognition in Historical Texts Using Prompt-Answering LLMs’. In Proceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2025) , edited by Anna Kazantseva, Stan Szpakowicz, Stefania Degaetano-Ortlieb, Yuri Bizzoni, and Janis Pagel. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.latechclfl-1.19. Wang, Shuhe, Xiaofei Sun, Xiaoya Li, et al. 2025. ‘GPT-NER: Named Entity Recognition via Large Language Models’. In Findings of the Association for Computational Linguistics: NAACL 2025 , edited by Luis Chiruzzo, Alan Ritter, and Lu Wang. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-naacl.239. Xu, Derong, Wei Chen, Wenjun Peng, et al. 2024. ‘Large Language Models for Generative Information Extraction: A Survey’. Frontiers of Computer Science 18 (6): 186357. https://doi.org/10.1007/s11704-024-40555-y. Yarowsky, David, and Grace Ngai. 2001. ‘Inducing Multilingual POS Taggers and NP Bracketers via Robust Projection Across Aligned Corpora’. Paper presented at NAACL 2001. Second Meeting of the North American Chapter of the Association for Computational Linguistics . https://aclanthology.org/N01-1026/. Yousef, Tariq, Chiara Palladino, Gerhard Heyer, and Stefan Jänicke. 2023. ‘Named Entity Annotation Projection Applied to Classical Languages’. Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature , 175–82. https://doi.org/10.18653/v1/2023.latechclfl-1.19. Footnotes The digital edition is being developed within the framework of an FWF-funded research project (2022–2026). The project is jointly based at the Institute for the Study of the Habsburg Monarchy and the Balkan Region of the Austrian Academy of Sciences in Vienna, and the Department of Digital Humanities at the University of Graz. Project blog: https://memoiren.hypotheses.org/ . Exact model version: claude-sonnet-4-20250514. GPT-4, LLaMA 3.3 (70B) und Claude Sonnet 3.5. GitHub-Repository accompanying this paper, URL: https://github.com/selinagalka1/dses_and_llms/tree/main . The annotation of the text was carried out by Ines Peper, Michael Pölzl, and Chiara Petrolini. The BIO (Begin–Inside–Outside) format is a common tagging scheme used in sequence labeling tasks such as named entity recognition. Each token in the text is labeled as either the beginning of an entity (B), inside an entity (I), or outside of any entity (O). In this context, 'rough' refers solely to the presence of doubled sentences offering alternative readings, the inconsistent transfer of variant text from different manuscripts, and the fact that the text has not yet undergone final correction. See European Open Source AI Index. URL: https://osai-index.eu/the-index . E.g. the model often returned multiple paragraphs instead of a one-to-one projection, produced variant versions, included lists of extracted entities instead of inline annotations, or even inserted French entities into the German translation. However, it should be noted that OLMo-7B is a relatively small model, and larger versions could be explored in future work to assess potential improvements. The prompts for GPT and Llama were slightly more structured, incorporating bullet points, but conveyed the same instructions. The paragraph can be found in the input document containing the french text of the memoirs for the Annotation Projection in the GitHub-Repository. At the current stage of the edition, the @ref attribute contains only the identifier (ID). During the final transformation and publication phases, it will be expanded to include the full URI referencing the corresponding person entries, which will also be utilized in the RDF representation. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 28 Jan, 2026 Read the published version in International Journal of Digital Humanities → Version 1 posted Editorial decision: Revision requested 08 Aug, 2025 Reviews received at journal 08 Aug, 2025 Reviewers agreed at journal 07 Aug, 2025 Reviewers invited by journal 07 Aug, 2025 Editor assigned by journal 07 Aug, 2025 Submission checks completed at journal 23 Jul, 2025 First submitted to journal 21 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7175875","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":497473425,"identity":"f35b3eb3-b331-4504-8804-c5358ffe9235","order_by":0,"name":"Selina Galka","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIie2QMUvDQBiG33BwWVJuPanYv3DSKaDJX0kJuAlOIih6pZAuoms2f4H/IfLBdSnOEVy66KoUJCAUL1ppl7OOgvcs3zd8D+97B3g8fxKmq88ZcuAFiH6hBEuFcQTlmiJ/UvCtsPUIp9IbD4d0dIbkesQH8/3icRvidvbaIDl3KWp6p6k0yEvi1D0sniLI576UyJ0pSg40RRw5KNRWoSitDaQCcxe7mVllgbxH4eg9tgqkYU2GC/fz60BTp0CiiJtu0Cqi4LICuYtNbbHOlcx2iR/El/dtCuexVpMt7So2JppHb3vpzqTo180xpRCGPTQnp8JZ7Av7Cas9a9M3CJZ0tYpq87nH4/H8Kz4AHjxQyrXw2P4AAAAASUVORK5CYII=","orcid":"","institution":"University of Graz","correspondingAuthor":true,"prefix":"","firstName":"Selina","middleName":"","lastName":"Galka","suffix":""},{"id":497473426,"identity":"8a52a5da-2f0c-45d0-b35f-4a3de625e86f","order_by":1,"name":"Georg Vogeler","email":"","orcid":"","institution":"University of Graz","correspondingAuthor":false,"prefix":"","firstName":"Georg","middleName":"","lastName":"Vogeler","suffix":""}],"badges":[],"createdAt":"2025-07-21 09:53:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7175875/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7175875/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s42803-025-00114-8","type":"published","date":"2026-01-28T15:59:22+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":89066601,"identity":"49456b3d-d4af-4980-b03a-27e2f96467a1","added_by":"auto","created_at":"2025-08-14 10:42:45","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":421591,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt for Annotation Projection\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7175875/v1/856080d292bfc6383e72cfd8.png"},{"id":89066056,"identity":"97cad588-b91c-4123-aa63-95b6b1d16ddf","added_by":"auto","created_at":"2025-08-14 10:42:30","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":334092,"visible":true,"origin":"","legend":"\u003cp\u003eInterface for manual validation\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7175875/v1/216298007bd9db828cf4b33a.png"},{"id":89066109,"identity":"1b8e4bda-eab7-454c-af31-606a5aa33115","added_by":"auto","created_at":"2025-08-14 10:42:35","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":636420,"visible":true,"origin":"","legend":"\u003cp\u003eFrench text and annotated german translation by\u003cem\u003e Qwen\u003c/em\u003e(only the “the count” in the last sentence has not been tagged by the model)\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7175875/v1/b94a883b752f6003acf75874.png"},{"id":89066612,"identity":"eec5d8a8-9ce1-4dde-8f31-5d80ee98fb54","added_by":"auto","created_at":"2025-08-14 10:42:45","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":105441,"visible":true,"origin":"","legend":"\u003cp\u003eLLM-Output, person relationships modelled in Turtle using project-specific identifiers and the Relationship Ontology\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7175875/v1/c5038f1a2441cf3153626bde.png"},{"id":101690662,"identity":"8948cc87-6c7e-4418-a131-11859c8111ea","added_by":"auto","created_at":"2026-02-02 16:07:02","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1957394,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7175875/v1/1a076d26-753c-483d-b814-4b42542a47e8.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Annotating, Projecting, and Interpreting Named Entities in Digital Scholarly Editions with LLMs","fulltext":[{"header":"Introduction","content":"\u003cp\u003eDigital Scholarly Editions (DSEs) aim to make (historical) sources accessible according to digital paradigms. They often go beyond presenting a single text, encompassing multiple textual variants and, in many cases, translations as well. Especially in the editing of ego\u0026ndash;documents such as memoirs, letters, or diaries, considerable effort is devoted to close reading of the text, identifying referenced entities such as persons and places, and encoding them using TEI/XML. Sometimes translations are also included to enhance accessibility and enable comparative analysis.\u003c/p\u003e\n\u003cp\u003eThis type of work has also been undertaken in the digital edition of the memoirs of Countess Schwerin (1684-1732), which serves as the basis for the topic discussed in this paper. The project\u003ca href=\"#_ftn1\" name=\"_ftnref1\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e aimed to make the memoirs \u0026ndash; originally written in French and transmitted through two distinct manuscripts \u0026ndash; accessible in a semantically enriched digital format. The memoirs offer rare, detailed insight into the life of an aristocratic woman in the early 18th century and reveal female networks, agency, and courtly life at Berlin and Vienna, which are otherwise underrepresented in sources from this period (Tersch 2004, 738). They also contribute to an understanding of informal and kinship\u0026ndash;based power structures within the Habsburg and Prussian aristocracy. The persons mentioned in the memoirs of Countess Louise Charlotte of Schwerin are of particular interest for several reasons \u0026ndash; e.g., because she was personally connected to the Prussian and Habsburg elites, including courtiers, diplomats, clergy, and even the emperor, and describes interactions with other women at court, which give new insight into female influence at court. Following that, all mentioned persons are systematically encoded in the TEI/XML edition, linked to external identifiers where possible, and researched in depth.\u003c/p\u003e\n\u003cp\u003eIn the Memoirs of Countess Louise Charlotte of Schwerin, individuals are referenced in diverse and historically significant ways. Many people appear only through indirect expressions like \u0026ldquo;mon p\u0026egrave;re\u0026rdquo; (my father) or \u0026ldquo;ma belle-m\u0026egrave;re\u0026rdquo; (my mother-in-law or my stepmother). These indirect references are crucial for understanding the kinship\u0026ndash;based and informal networks that structure courtly life. From a theoretical perspective, these indirect mentions correspond to what Saul Kripke (1982) defined as non-rigid designators: expressions whose referent can vary across possible worlds or contexts. Unlike rigid designators, which refer to a specific individual in all possible contexts, non-rigid expressions rely on contextual, relational knowledge. Also, mentioned person names can be quite complex, including various role names or titles, like \u0026ldquo;monsieur le baron de Danckelmann\u0026rdquo;. Additionally, named entities may frequently occur as so-called \u0026ldquo;nested entities\u0026rdquo; \u0026ndash; for example, a person name that includes a place name, such as \u0026ldquo;madame d\u0026apos;Amtitz\u0026rdquo;.\u003c/p\u003e\n\u003cp\u003eAlthough the memoirs of Countess Schwerin are written in French, translating them into German is particularly meaningful given the historical and cultural context. The Countess spent a significant part of her life in Vienna, where German was the dominant language of court, society, and administration. A German translation not only makes the text accessible to a broader regional audience but also aligns with the cultural and historical connections between the content of the memoirs and the German-speaking world.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis paper primarily aims to demonstrate and discuss the application of large language models (LLMs) for Named Entity Recognition (NER) on the normalized historical French text of Countess Schwerin\u0026rsquo;s memoirs. The focus lies on annotating mentions of persons and places, as well as transferring these annotations to a German translation of the text \u0026mdash; a process known as annotation projection. In addition, the paper includes a brief example showcasing the potential of LLMs for information extraction, specifically for identifying (implicit) personal relationships mentioned in the text.\u0026nbsp;\u003c/p\u003e"},{"header":"Theoretical foundations","content":"\u003ch2\u003eLarge Language Models and their (possible) use in digital editions\u003c/h2\u003e\n\u003cp\u003eLarge language models (LLMs) are advanced neural network architectures trained on vast amounts of textual data to generate, interpret, and manipulate human language. The use of LLMs is currently being explored across a wide range of applications, including those relevant to digital editions (Pollin et al. 2025). This section does not attempt to provide a comprehensive review of the research literature, but rather aims to offer an overview of potential areas of application.\u003c/p\u003e\n\n\u003cp\u003eOne such area is text creation, which encompasses transcription tasks. For example, Humphries et al. (2024) tested LLMs on a corpus of English handwritten documents from the 18th and 19th centuries. Their study demonstrated that LLMs can effectively interpret handwritten content, outperforming established Handwritten Text Recognition (HTR) systems in both accuracy and efficiency, while also being more cost-effective. In addition to transcription, LLMs are also being evaluated for post-OCR processing, where they are used to correct errors and improve the quality of OCR outputs (see e.g., Thomas/Gaizauskas/Lu 2024). Also, the application for normalization is tested (see e.g., Scholger et al. 2025).\u003c/p\u003e\n\n\u003cp\u003eAnother important area is annotation, where best practices are discussed (see e.g., T\u0026ouml;rnberg 2024), and LLMs are evaluated on specific annotation tasks, such as those in social media research (Pangakis/Wolken 2025). In the context of digital editions, the annotation of named entities \u0026ndash; such as persons or places \u0026ndash; is often a central focus. The application of LLMs to NER has been the subject of numerous studies, including work focused on historical texts (e.g., Hiltmann et al. 2025, Tudor/Megyesi/\u0026Ouml;stling 2025, Greif/Griesshaber/Greif 2025, Gonz\u0026aacute;lez\u0026ndash;Gallardo et al. 2024, Santos et al. 2024). Fang et al. (2025) also demonstrated that LLMs outperform traditional NLP methods on tasks such as word segmentation, part-of-speech tagging, and NER, particularly when applied to historical Chinese texts. Xu et al. (2024) provide a comprehensive review of how generative LLMs are used to extract structured data \u0026ndash; such as entities, relations, and events \u0026ndash; from unstructured textual input.\u003c/p\u003e\n\n\u003cp\u003eLLMs can offer notable advantages over traditional information extraction methods or NLP methods in terms of flexibility and generalization. They can perform multiple tasks \u0026ndash; such as NER, relation extraction, and event extraction \u0026ndash; within a single model architecture using task-specific prompts, thereby eliminating the need for separate task-specific pipelines. Another key benefit can be their capacity for few-shot and zero-shot learning: they can achieve effective performance with minimal labeled data, whereas conventional systems typically require extensive task-specific training. In addition, generative LLMs are capable of producing structured outputs relevant for DSEs (e.g., JSON, TEI/XML or RDF) directly, which facilitates seamless integration into digital edition workflows.\u003c/p\u003e\n\u003ch2\u003eNamed Entity Recognition\u003c/h2\u003e\n\u003cp\u003eNamed Entity Recognition (NER) refers to the identification and classification of named entities such as persons, places, organizations, and dates within a text. A variety of methods are available for this task, ranging from traditional rule\u0026ndash;based approaches \u0026ndash; which rely on regular expressions and gazetteers \u0026ndash; to machine learning pipelines such as those implemented in frameworks like \u003cem\u003espaCy\u003c/em\u003e. More recent developments include transformer-based models such as BERT and \u003cem\u003eflair\u003c/em\u003e, which have significantly improved performance by leveraging contextual embeddings. Building on this progress, prompt-based NER using LLMs has recently emerged as a possible alternative. \u003c/p\u003e\n\n\u003cp\u003eTraditional NER approaches often struggle with historical texts due to their lack of standardized spelling and syntax, domain and genre variation, and limited availability of annotated training data (see Ehrmann et al. 2023). Frameworks like \u003cem\u003espaCy\u003c/em\u003e and \u003cem\u003eflair\u003c/em\u003e perform well on modern, normalized corpora but are less effective when applied to historical sources. Moreover, Nadeau and Sekine defined in their survey of NER applications and methods, that only referents with rigid designators (as defined by Kripke 1982) are considered in the identification and classification of Named Entities (2007).\u003c/p\u003e\n\n\u003cp\u003eSince the advent of ChatGPT and other LLMs, interest in zero- and few-shot NER without fine-tuning has increased. However, earlier evaluations (e.g., Gonz\u0026aacute;lez-Gallardo et al. 2023; Tang et al. 2024) reported inconsistent or weak performance of LLMs for NER, primarily due to insufficient prompt engineering, a lack of domain adaptation, and inadequate evaluation strategies \u0026ndash; as noted by Hiltmann et al. (2025). In their study on Baedeker travel guides for Berlin (1921), Hiltmann et al. systematically evaluated different prompting strategies (zero-, few-, and many-shot) and techniques (e.g., penalization/reward heuristics) using ChatGPT-4o. Their approach employed a proprietary annotation scheme adapted from Wang (2025) and they demonstrated that LLMs can perform NER on historical travel guides with remarkable accuracy.\u003c/p\u003e\n\n\u003cp\u003eIn contrast, Tudor et al. (2025) tested NER on historical newsletters and demonstrated that, despite some success \u0026ndash; particularly with multilingual models such as \u003cem\u003eAya\u003c/em\u003e \u0026ndash; LLMs frequently produce hallucinated outputs, which limits their reliability. The authors emphasize that while prompt-based LLMs offer notable flexibility and reduce the need for costly annotation, they currently underperform compared to fine-tuned models and require further methodological refinement. It should be noted, however, that the \u003cem\u003eAya\u003c/em\u003e models are open-source models, which facilitates reproducibility and transparency, although they may lag behind in performance due to limited computational resources compared to proprietary alternatives.\u003c/p\u003e\n\u003ch2\u003eAnnotation Projection\u003c/h2\u003e\n\u003cp\u003eAnnotation projection is the process of transferring markup from a well-annotated source text to a corresponding target text, such as a translation. Its main motivation is efficiency: once annotations are created in one language, replicating them manually in others is labor-intensive. Yarowsky and Ngai (2001) were early proponents, applying projection for POS tagging from English to French. \u003c/p\u003e\n\u003cp\u003eBamman, Babeu, and Crane (2010) projected TEI/XML annotations from Ancient Greek and Latin texts to their English translations, aiming to build a multilingual digital library based on curated sources. They transferred structural and named entity markup across over 13,000 tags, achieving 88.2% accuracy using Moore\u0026rsquo;s sentence aligner and MGIZA++ for word alignment. Ermann, Turchi, and Steinberger (2011) applied annotation projection to multilingual NER corpora. English texts annotated for named entities were translated using PBSMT, with projection performed via string-matching and edit-distance methods. They reported up to 95% recall for Romance and Germanic languages, and over 86% for inflectional languages. Steinbach and Rehbein (2019) developed a modular NLP pipeline to project quotation and speaker annotations from English novels to German translations, using BLEUAlign and fast_align for alignment. Galassi et al. (2019) addressed non-literal translation in legal texts, projecting annotations for unfair clauses between English and German using sentence embeddings and Dynamic Time Warping, yielding accurate and efficient results. Yousef et al. (2023) projected named entity annotations from English to Ancient Greek and Latin via sentence-aligned Bible corpora and neural word alignment with XLM-R. Their heuristic handled varied alignment scenarios, reaching over 82% accuracy. Senator et al. (2025) used ChatGPT for cross-lingual annotation projection and zero-shot annotation in Arabic, focusing on emotion recognition and semantic role labeling. Results showed high accuracy, particularly on simpler texts, with human validation still essential for nuanced content. Numerous other studies have addressed the topic of annotation projection (see, e.g., Garc\u0026iacute;a-Ferrero/Agerri/Rigau, 2022; Kuchmiichuk 2023; Zahgir et al. 2023; Rai/Chatterji 2023). \u003c/p\u003e\n\u003cp\u003eMany of these approaches involve aligning the source and target languages at the word, sentence, or phrase level, followed by the evaluation of alignment quality (see Behzad et al. 2023, p. 244). In contrast, Chen et al. (2023) adopted a different strategy by first inserting special markers around annotated spans in the source text and subsequently translating the marked text \u0026ndash; a method referred to as \u003cem\u003emark-then-translate\u003c/em\u003e. The COMUTE project, running from 2024-2027, funded by the DFG, addresses the challenge of aligning non-parallel multilingual text versions; Levchenko (2024) explored the use of translation alignment algorithms for constructing a Multilingual Digital Edition (MDE) of \u003cem\u003eI promessi sposi\u003c/em\u003e, highlighting both the limitations of existing methods in handling literary translations and proposing an automated pipeline and evaluation metrics tailored to multilingual literary corpora.\u003c/p\u003e\n\u003cp\u003eWhen it comes to tools and frameworks for annotation projection, Akbik and Vollgraf (2018) introduced \u003cem\u003eZAP\u003c/em\u003e, an open-source multilingual framework, which provides methods for preprocessing, computing word alignments, transferring annotations, and visualizing projection results. \u003c/p\u003e\n\u003cp\u003eIn contrast to these prior approaches, this study applies LLMs directly to TEI/XML-encoded historical texts and their translation, without intermediate word or sentence alignments, using structured prompts and markup-conforming output.\u003c/p\u003e"},{"header":"Practical application","content":"\u003cp\u003eWhen it comes to the use of LLMs there are several areas in which users can influence the model\u0026rsquo;s performance. Nina Rastinger listed these in regard to NER: when it comes to the model itself, you can choose the model, the access type and the model parameters. When it comes to the data, the input characteristics of the data can be adjusted, like input format or amount of input data. A central component is the prompt design, where you can define the output format, perform demonstrations, use consistency strategies, describe the task, prompt the roles or decompose the task. (Rastinger 2024)\u003c/p\u003e\n\u003cp\u003eThe following applications involving LLMs were conducted using \u003cem\u003eClaude Sonnet 4\u0026nbsp;\u003c/em\u003efor NER and additionally \u003cem\u003eLlama-4-Maverick-17B-128E-Instruct-FP8\u003c/em\u003e and \u003cem\u003eGPT-4.1\u003c/em\u003e for annotation projection tasks.\u003ca href=\"#_ftn1\" name=\"_ftnref1\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e2\u003c/sup\u003e Previous tests involving NER and the evaluation of three LLMs\u003ca href=\"#_ftn2\" name=\"_ftnref2\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e demonstrated that \u003cem\u003eClaude Sonnet 3.5\u003c/em\u003e consistently delivered the most accurate and reliable results (Galka 2025). This assessment was further supported by the presentations at the workshop \u0026ldquo;Digital source editions and AI. Current trends, challenges and problems\u0026rdquo; held in Wolfenb\u0026uuml;ttel in April 2025, where \u003cem\u003eClaude\u0026apos;s\u003c/em\u003e performance was also rated highest in comparison with other models.\u003c/p\u003e\n\u003cp\u003eFor the use of the LLMs, the TEI/XML files were provided directly as input. Prior experiments had demonstrated that LLMs are capable of interpreting TEI/XML structure and generating corresponding, well-formed XML output. Rastinger (2024) likewise concluded that large language models (LLMs) perform Named Entity Recognition more effectively within the paradigm of \u003cem\u003eannotation\u003c/em\u003e than within that of \u003cem\u003eextraction\u003c/em\u003e, particularly when the task is framed as a structured markup or editing operation rather than a simple list-based retrieval. The output was returned in TEI/XML format, maintaining consistency with the original schema and facilitating integration into the digital edition workflow. The data of the following experiments can be found in the GitHub-Repository\u003ca href=\"#_ftn3\" name=\"_ftnref3\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e4\u003c/sup\u003e.\u003c/p\u003e\n\u003ch2\u003eA comparison of NER with BiLSTM and an LLM\u003c/h2\u003e\n\u003cp\u003eNER on the memoirs of the countess was evaluated using both the \u003cem\u003eflair\u003c/em\u003e-Framework (Akbik et al. 2019) and the LLM \u003cem\u003eClaude Sonnet 4\u003c/em\u003e, applied to a human annotated\u003ca href=\"#_ftn4\" name=\"_ftnref4\" title=\"\"\u003e\u003csup\u003e\u003c/sup\u003e\u003c/a\u003e\u003csup\u003e5\u003c/sup\u003e sample of 50 paragraphs drawn from the orthographically normalized French source text. The french flair model (ner-french) is based on Flair embeddings and BiLSTM-CRF (Akbik/Blythe/Vollgraf 2018).\u003c/p\u003e\n\u003ch3\u003eData\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eThis sample included 444 annotated person entities and 81 annotated place entities. Although the text has undergone orthographic normalization as part of the digital edition process, several historical linguistic challenges persist. These include nested entities \u0026ndash; such as place names embedded within person names (e.g., \u0026ldquo;madame d\u0026apos;Amtitz\u0026rdquo;) \u0026ndash; as well as complex entities involving role titles or honorifics. Nested entities are not annotated as such in the digital edition; instead, they are uniformly tagged as person entities. Particularly noteworthy in the context of ego-documents is the frequent occurrence of non-rigid designators for persons, such as \u0026ldquo;mon p\u0026egrave;re\u0026rdquo; or \u0026ldquo;ma belle-m\u0026egrave;re\u0026rdquo; in the memoirs of the countess.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eMethods \u003c/h3\u003e\n\u003cp\u003eFirst, NER was performed with the available French \u003cem\u003eflair\u003c/em\u003e NER model. The model was applied to a TEI-encoded French text via a script that extracts each paragraph, converts it into plain text using the \u003cem\u003estandoffconverter\u003c/em\u003e library \u0026ndash; which maintains a mapping between plain text and XML structure \u0026ndash; and then re-integrates the detected entities back into the TEI as inline \u0026lt;rs\u0026gt; tags. All entity spans categorized as MISC were excluded from the evaluation.\u003c/p\u003e\n\u003cp\u003eIn a next step, NER was also performed using the LLM \u003cem\u003eClaude Sonnet 4\u003c/em\u003e. The TEI/XML file was used directly as input, as prior small-scale experiments had already demonstrated that LLMs are capable of processing this format effectively. Each paragraph was submitted to the model individually. The prompt instructed the model to tag only person and place names using \u0026lt;persName\u0026gt; and \u0026lt;placeName\u0026gt; elements, respectively, while preserving all other existing TEI structures, including nested tags such as \u0026lt;app\u0026gt;, \u0026lt;rdg\u0026gt;, and \u0026lt;ptr\u0026gt;. The prompt contained several illustrative input-output examples (few-shot-prompt), covering both explicit and implicit references (e.g., \u0026ldquo;ma tante\u0026rdquo;), as well as compound titles (e.g., \u0026ldquo;comte de Schwerin\u0026rdquo;).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo facilitate the evaluation of named entity annotations, the annotated TEI/XML-file was first transformed into a plain-text format using a custom XSLT-stylesheet that encodes entities with proprietary markers. This intermediate format was then normalized to ensure consistent tokenization across the dataset. During evaluation, both the system-generated and the human-annotated versions were converted into BIO-labeled\u003ca href=\"#_ftn1\" name=\"_ftnref1\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e6\u003c/sup\u003e sequences, allowing for direct token-level comparison. The \u003cem\u003eseqeval\u003c/em\u003e library, specifically tailored for sequence labeling tasks, was employed to compute entity-level precision, recall, and F1-scores based on these BIO\u0026ndash;tags.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eResults\u003c/h3\u003e\n\u003cp\u003eNER with \u003cem\u003eflair\u003c/em\u003e shows relatively stronger recognition of location entities (see table 1). While many place entities were correctly annotated, components of personal names \u0026ndash; such as \u0026ldquo;Schwerin\u0026rdquo; in \u0026ldquo;Comte de Schwerin\u0026rdquo; \u0026ndash; were frequently annotated as LOC. Other sources of error were rare. In contrast, the model struggles significantly with person entities \u0026ndash; especially with complex names, e.g., excluding roles or titles and missed most of the entities with non-rigid designators. 150 entities were identified and classified as MISC \u0026ndash; these annotations contained actual person names (rigid or sporadic with non-rigid designators), religious entities (\u0026ldquo;Dieu\u0026rdquo;) and many text spans which do not include any entities and were not included in the evaluation.\u003c/p\u003e\n\u003cp\u003eRegarding \u003cem\u003eClaude\u003c/em\u003e, all tags from the input document were preserved in the LLMs annotated TEI/XML output. Minor alterations were observed at the textual level: apostrophes were consistently replaced with a typographically distinct variant, and in a few isolated cases, expressions such as \u0026ldquo;du comte de Schwerin\u0026rdquo; were reformulated as \u0026ldquo;de le comte de Schwerin.\u0026rdquo; Despite these minor shifts, no content was lost in the transformation.\u003c/p\u003e\n\u003cp\u003eThe NER process using \u003cem\u003eClaude\u003c/em\u003e achieved high performance in detecting location entities; in comparison the recognition of person entities yielded lower metrics (see table 1), but it is significantly better than the NER with \u003cem\u003eflair.\u003c/em\u003e A manual inspection revealed that the actual performance of the model is also even better than the strict evaluation metrics suggest. This discrepancy arises primarily from the evaluation method\u0026rsquo;s sensitivity to exact string matches, which does not account for minor surface variations such as the inclusion or omission of definite articles. A substantial portion of the mismatched entities stem from cases where the gold standard includes forms such as \u0026ldquo;le comte de Schwerin\u0026rdquo;, whereas \u003cem\u003eClaude\u003c/em\u003e annotated simply \u0026ldquo;comte de Schwerin\u0026rdquo;, or vice versa. It should also be noted that such variations are not standardized within the ground truth itself.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eTable 1\u0026nbsp;\u003c/strong\u003eNER-Results\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"602\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e\u003cem\u003eflair\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e\u003cem\u003eClaude Sonnet 4\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003eLOC Precision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.89\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003eLOC Recall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003eLOC F1-score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003ePER Precision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.72\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003ePER Recall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.74\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003ePER F1-score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 201px;\"\u003e\n \u003cp\u003e0.73\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch3\u003eDiscussion\u003c/h3\u003e\n\u003cp\u003eThe comparative evaluation of NER using the \u003cem\u003eflair\u003c/em\u003e framework and the LLM \u003cem\u003eClaude Sonnet 4\u003c/em\u003e demonstrates a clear performance gap, particularly in the recognition of person entities. While \u003cem\u003eflair\u003c/em\u003e achieved modest results in recognizing location entities, its performance with person entities was significantly weaker. This discrepancy can largely be attributed to the model\u0026rsquo;s limitations in handling nested structures, complex honorifics, and non-rigid designators. In contrast, \u003cem\u003eClaude Sonnet 4\u003c/em\u003e demonstrated much higher overall precision and recall for both person and location entities. Additionally, the LLM\u0026rsquo;s performance with indirect references (e.g., \u0026ldquo;ma belle-m\u0026egrave;re\u0026rdquo;), compound titles and nested entities suggests a greater contextual understanding. However, even the \u003cem\u003eClaude\u003c/em\u003e model\u0026rsquo;s performance was not flawless. The evaluation revealed that many discrepancies resulted from minor formal mismatches \u0026ndash; such as the presence or absence of definite articles \u0026ndash; which had a disproportionately negative effect on automated metrics like F1-score.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eClaude Sonnet\u003c/em\u003e clearly outperformed flair in accuracy and structural fidelity, confirming that well-prompted LLMs offer a strong alternative to traditional NER pipelines. While prompt design and edge cases remain challenging, the results suggest LLMs are well-suited for annotating historical, referentially dense texts in digital scholarly editions.\u003c/p\u003e\n\u003ch2\u003eAnnotation Projection with Large Language Models\u003c/h2\u003e\n\u003cp\u003eAnnotation Projection has not been a major area of research within DSEs so far, but it can offer significant benefits and potentially save considerable annotation and/or programming effort. Additionally, there is potential to simplify the task \u0026ndash; for example, prior sentence or word alignment may not be necessary to perform annotation projection with LLMs; although some kind of previous alignment (e.g., paragraph\u0026ndash;based) could probably increase the performance.\u003c/p\u003e\n\u003cp\u003eDuring the translation process in the project, the German text was already aligned with the French original on a paragraph-by-paragraph basis. In the context of digital scholarly editions, human-controlled translations are generally preferred over purely machine-generated ones, as they allow for greater sensitivity to nuances and historical context. Source texts often already exhibit a structural organization \u0026ndash; such as chapters, sections, or paragraphs \u0026ndash; which can already be mirrored in the translation-process, facilitating both alignment and annotation.\u003c/p\u003e\n\u003ch3\u003eData\u003c/h3\u003e\n\u003cp\u003eThe dataset consists of the normalized French text, richly encoded in TEI/XML with extensive structural and semantic markup (e.g., named entities, structural tags, text versions), and a corresponding German translation. The French text was normalized only in regard to orthography, but not in wording, since eighteenth-century French exhibits a linguistic stage that is largely comparable to modern French. The translation is aligned with the French source text at the paragraph level and is likewise encoded in TEI/XML, although it contains no additional annotations beyond the paragraph structure. The German version strives to remain as close as possible to the French original in terms of content, while rendering it in fluent and readable contemporary German. While it is important to preserve the historical character of the source, the translation deliberately avoids imitating historical German. As a result, certain deviations from the French sentence structures are necessary, since a direct one-to-one transfer is often not feasible and not all nuances can be retained. A literal translation would therefore be inadequate.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFor the purposes of this study, 101 paragraphs from the normalized and extensively annotated French text of the \u003cem\u003eMemoirs of Countess Schwerin\u003c/em\u003e were used as a basis for projecting person and place annotations onto the corresponding German translation. The French normalized text also records the differences between Manuscript A and Manuscript W using \u0026lt;app\u0026gt; and \u0026lt;rdg\u0026gt; elements. However, the German translation is not yet complete, not annotated and has not undergone a final round of corrections up to this point. Furthermore, no final decision has been made on how to handle the textual variants in the German translation. Due to the large number of textual variants, no automatic machine translation method could be applied so far, which leads to convincing results. Also, the focus of the DSE was not on a literal translation, but rather on providing a modern, easily readable version. Therefore, the annotations of the textual variants were not taken into account for the annotation projection, and the \u0026ldquo;rough\u0026rdquo; human translation\u003ca href=\"#_ftn1\" name=\"_ftnref1\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e7\u003c/sup\u003e was used.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe French normalized and annotated text sample contained 761 annotated person entities and 104 annotated place entities, already linked to a project-specific index file that also records authority data such as GND or GeoNames identifiers. Additionally, the French source text contained work entities (\u0026lt;bibl\u0026gt;), references to the transcription of MS W (\u0026lt;anchor\u0026gt;), annotations of person relationships (\u0026lt;seg\u0026gt;), and annotations of dates (\u0026lt;date\u0026gt;). While these elements were present in the TEI/XML input, they were not taken into account in the annotation projection process.\u003c/p\u003e\n\u003ch3\u003eMethod\u003c/h3\u003e\n\u003cp\u003eIn order to automate the transfer of named entity annotations (persons and places) from a French TEI/XML edition to its German counterpart, three LLMs were tested and compared: \u003cem\u003eClaude Sonnet 4\u003c/em\u003e, \u003cem\u003eLlama-4-Maverick-17B-128E-Instruct-FP8\u003c/em\u003e, \u003cem\u003eGPT 4.1 and Qwen 3 - 14B\u003c/em\u003e. \u003cem\u003eClaude Sonnet 4\u003c/em\u003e (Anthropic) is a commercially available model offered with a 200K\u0026ndash;token context. GPT-4.1 (OpenAI) is a model available through paid API and ChatGPT Plus, supporting up to 1 million tokens and\u003cem\u003e\u0026nbsp;Llama-4-Maverick-17B-128E-Instruct-FP8\u003c/em\u003e is an open\u0026ndash;source model from Meta. The \u003cem\u003eLlama\u003c/em\u003e-model is significantly smaller, using only 17 billion active parameters, whereas \u003cem\u003eClaude Sonnet 4\u003c/em\u003e and \u003cem\u003eGPT-4.1\u003c/em\u003e are larger, dense models. The annotation projection was also tested with \u003cem\u003eOLMo-7B-Instruc\u003c/em\u003et \u0026ndash; currently one of the most open open-source models \u0026ndash; but the results were unsatisfactory and significantly below those of other models evaluated.\u003ca href=\"#_ftn2\" name=\"_ftnref2\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e8\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eAfter loading the French and German TEI documents, all paragraph elements (\u0026lt;p\u0026gt;) from both texts are extracted and then iterated through them in parallel. For each paragraph pair, a prompt is constructed that includes the French paragraph in its original, annotated TEI/XML form, along with the unannotated German TEI/XML\u0026ndash;paragraph. The prompt instructs the model to replicate the person and place annotations from the French source text in the German version, including the corresponding references (i.e., @ref attributes containing project-specific identifiers), while preserving the exact wording of the German translation.\u003c/p\u003e\n\u003cp\u003eIn this experiment, the initial results obtained using a zero-shot prompt with \u003cem\u003eClaude Sonnet 4\u003c/em\u003e were of such high quality that no further prompt optimization was pursued. This means that no examples are given in the prompt; rather, the model is expected to generalize the task based solely on the instruction. The models were configured with a temperature of 0.0 to make it as deterministic as possible and avoid hallucination of entities or structure. The exact prompt used e.g., for Claude was as follows\u003ca href=\"#_ftn3\" name=\"_ftnref3\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e9\u003c/sup\u003e:\u003c/p\u003e\n\u003cp\u003eThe output is validated based on two criteria: first, whether all annotations have been correctly transferred \u0026ndash; including the links to the appropriate entries in the index files \u0026ndash; and second, whether they have been applied to the correct spans in the text. First the annotations between the French and the German TEI/XML file were compared. A script extracts all entity references based on tags and @ref-attributes and records any entities missing a @ref, counts valid entity occurrences, and aligns corresponding paragraphs across both languages. For each shared paragraph, it identifies discrepancies, such as missing or extra references in the German version compared to the French source. The discrepancies are reviewed manually and a manually reviewable HTML interface is generated, in which annotated entities in both languages are displayed side by side. When hovering over an entity, corresponding entries in the other language are highlighted. The application of the projected annotations to the correct spans in the German translation was manually reviewed, as no ground truth reference was available for evaluation.\u003c/p\u003e\n\u003cp\u003eResults\u003c/p\u003e\n\u003cp\u003eThe models reliably followed instructions and produced well-structured, valid TEI/XML annotations without requiring additional examples or iterative refinement. The annotated output was validated with regard to text integrity and annotation accuracy.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eA comparison of the annotated paragraphs from Claude revealed minor and punctual discrepancies in six cases between the source and target text, including missing, altered and additional words (e.g., in P.39: \u0026quot;Jesus.\u0026quot; vs. \u0026quot;J\u0026eacute;sus.\u0026quot; or in P.88: \u0026nbsp;\u0026quot;ew\u0026auml;hrt\u0026quot; vs. \u0026quot;Gew\u0026auml;hrt\u0026quot;). The output\u0026ndash;text from GPT differed in six small cases from the TEI/XML-input text. In the output generated by Llama, a greater number of textual modifications were observed \u0026ndash; approximately 36 instances \u0026ndash; including changes to punctuation as well as, in some cases, longer text passages.\u003c/p\u003e\n\u003cp\u003eThe French annotated input text contained 865 annotated entities (761 persons and 104 places). The following table presents the discrepancies found in the German annotated output in comparison to these French annotations:\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eTable 2\u0026nbsp;\u003c/strong\u003eAnnotation Projection Results\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"621\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e\u003cem\u003eClaude Sonnet 4\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u003cem\u003eLlama 4 Maverick\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e\u003cem\u003eGPT 4.1\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e\u003cem\u003e\u0026nbsp;Qwen 3 - 14B\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eFalse Negatives (Persons)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e86\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eFalse Negatives (Places)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eFalse Positives (Persons)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e57\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e26\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eFalse Positives (Places)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eAdditional tags\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003e6 (\u0026lt;anchor\u0026gt;, \u0026lt;bibl\u0026gt;, \u0026lt;hi\u0026gt;)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e43 (\u0026lt;app\u0026gt;, \u0026lt;bibl\u0026gt;, \u0026lt;date\u0026gt;, \u0026lt;hi\u0026gt;, \u0026lt;rdg\u0026gt;, \u0026lt;seg)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e2 (\u0026lt;seg\u0026gt;)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003e20 (\u0026lt;bibl\u0026gt;, \u0026lt;anchor\u0026gt;, \u0026nbsp;\u0026lt;seg\u0026gt;, \u0026lt;app\u0026gt;, \u0026lt;rdg\u0026gt;)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eValid TEI/XML\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 178px;\"\u003e\n \u003cp\u003eText lost/changed\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 102px;\"\u003e\n \u003cp\u003eyes, but very little (6 small adjustments)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003eyes, about 36 changes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003eyes, 9 small adjustments\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 118px;\"\u003e\n \u003cp\u003eyes, including corrections of mistakes (e.g. inserting missing whitespaces)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eRegarding Claude, out of 865 annotated French entities (761 persons and 104 places), a total of 22 references were not successfully transferred to the German version \u0026ndash; 16 person and 6 place annotations are missing in the annotated German output of the LLM. In contrast, the German annotated output text contains 7 additional entity annotations (6 persons and 1 place) not present in the French original. This corresponds to a transfer rate of over 97% for person and place annotations from the French source text to the German translation. A manual inspection confirmed that all transferred annotations were applied to the correct spans in the German text. In the few cases where discrepancies occurred, they typically involved minor variations such as the inclusion or omission of definite articles \u0026ndash; an inconsistency that also appears in the French source and does not significantly affect semantic interpretation.\u003c/p\u003e\n\u003cp\u003eAdditionally, all @ref\u0026ndash;attributes containing the project-specific identifiers for the entities have been transferred correctly. The following examples illustrate effective annotation projection from French source texts to their German translations, particularly in semantically and structurally complex cases. \u0026ldquo;P.xx\u0026rdquo; refers to the paragraph number or the corresponding @xml:id in the TEI\u0026ndash;encoded text:\u003c/p\u003e\n\u003cul class=\"decimal_type\"\u003e\n \u003cli\u003e\u003cstrong\u003eP. 102,\u0026nbsp;\u003c/strong\u003eFrench: la demoiselle de Wylich\u0026ndash;Lottum / German: Fr\u0026auml;ulein von W.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 14,\u0026nbsp;\u003c/strong\u003eFrench: ce gouverneur / German: diesen Hofmeister / Erzieher / Lehrer\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 16,\u0026nbsp;\u003c/strong\u003eFrench: un des premiers de ce lieu / German: einem der der Vornehmsten dort\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 18,\u0026nbsp;\u003c/strong\u003eFrench: un homme d\u0026rsquo;un plus grand caract\u0026egrave;re qu\u0026rsquo; / German: ein Mann mit einem h\u0026ouml;heren Rang\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 28,\u003c/strong\u003e French: une fille fort fid\u0026egrave;le qui m\u0026rsquo;avait \u0026eacute;lev\u0026eacute;e / German: sehr zuverl\u0026auml;ssigen M\u0026auml;dchen, das mich erzogen hatte\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 62,\u0026nbsp;\u003c/strong\u003eFrench: la plus belle terre dans le royaume / German: das sch\u0026ouml;nste Landgut im K\u0026ouml;nigreich\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eP. 97,\u003c/strong\u003e French: celle qui \u0026eacute;tait dans la maison / German: jener, die im elterlichen Hause verblieb\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe following observations concern instances where persons annotations were not projected from the French source text to the German translation, along with several cases of additional or spurious annotations. Five instances were not projected due to complex structures, particularly involving \u0026lt;app\u0026gt; and \u0026lt;rdg\u0026gt; elements in the French text, which are only partially reflected or simplified in the German translation (e.g., P.81). One instance involved a shift from a named entity (\u0026ldquo;\u003cem\u003ecomte de Schwerin\u0026rdquo;\u003c/em\u003e) to a pronominal reference (\u0026ldquo;\u003cem\u003edessen\u0026rdquo;\u003c/em\u003e) in German, impeding direct annotation transfer (curved brackets indicating annotations and bold highlighting indicating absent annotations):\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFrench annotated text:\u0026nbsp;\u003c/em\u003e\u003cbr\u003e\u0026nbsp;Il fallut lui donner toutes mes lettres du {comte de Schwerin} pour les porter \u0026agrave; {mon p\u0026egrave;re}, afin qu\u0026rsquo;il juge\u0026acirc;t de ce que le {comte de Schwerin} avait dit de {ceux qui m\u0026rsquo;\u0026eacute;levaient}, les pouvait toucher.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eGerman human translation for the edition, annotated with Claude:\u003c/em\u003e\u003cbr\u003eIch musste ihr alle meine Briefe des {Grafen von Schwerin} \u0026uuml;bergeben, damit sie sie {meinem Vater} bringen und dieser beurteilten konnte, ob \u003cstrong\u003edessen\u003c/strong\u003e \u0026Auml;u\u0026szlig;erungen \u0026uuml;ber {diejenigen, die mich aufgezogen hatten}, sie betreffen k\u0026ouml;nnten.\u003c/p\u003e\n\u003cp\u003eAnother case in P.76 featured the pronoun \u0026ldquo;er\u0026rdquo; in place of the explicit mention \u0026ldquo;der Graf von Schwerin\u0026rdquo;. In this case one annotation was missing and one incorrectly added \u0026ndash; \u0026ldquo;der Graf von Schwerin\u0026rdquo; was tagged even though the reference was to his father:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFrench annotated text:\u0026nbsp;\u003c/em\u003e\u003cbr\u003e\u0026nbsp; Apr\u0026egrave;s quoi {le comte de Schwerin le p\u0026egrave;re} vint me faire un grand compliment et m\u0026rsquo;embrassa comme sa ch\u0026egrave;re fille [...]\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u003cem\u003eGerman human translation for the edition, annotated with Claude:\u003c/em\u003e\u003cbr\u003e\u003cstrong\u003eDer Vater des\u0026nbsp;\u003c/strong\u003e{Grafen von Schwerin} kam herein, \u0026uuml;bersch\u0026uuml;ttete mich mit Komplimenten und umarmte mich, als w\u0026auml;re ich seine eigene Tochter.\u003c/p\u003e\n\u003cp\u003eIn one case in P.63 \u0026ldquo;la maison de mon p\u0026egrave;re\u0026rdquo; was rendered as \u0026ldquo;mein Elternhaus\u0026rdquo;, thus eliding the explicit person reference. In one case in P.83 the model correctly refrained from projection: the German text erroneously mentions \u0026ldquo;der Graf von Schwerin\u0026rdquo; instead of \u0026ldquo;die Gr\u0026auml;fin von Schwerin\u0026rdquo; (the translation has not undergone a final correction round yet). In another case in P.96 the phrase \u0026ldquo;ministres du Roi l\u0026rsquo;acc\u0026rdquo; (two entities) was only partially rendered in German as \u0026ldquo;k\u0026ouml;nigliche Beamte\u0026rdquo;, tagged as a single generic entity. In another case (P.98) \u0026ldquo;mon beau\u0026ndash;fr\u0026egrave;re\u0026rdquo; is omitted in the translation and in P.83 \u0026ldquo;der Graf von Schwerin\u0026rdquo; was not projected; it appears in square brackets in the German translation. In two cases the entity is not mentioned in the german text.\u003c/p\u003e\n\u003cp\u003eAmong the seven additional annotations identified in the German translation, four are attributable to duplicated sentences \u0026ndash; passages in which different versions of the translation coexist. In these cases, it remains to be determined which version will be retained as the final target segment, a decision that will directly impact the annotation projection. Only two annotations were incorrectly added in the LLM-generated TEI/XML output.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAn inspection of the six place annotations that were not projected from the French source text to the German translation reveals the following pattern: in four of the six cases, the respective place names (\u0026ldquo;Wesel\u0026rdquo;, P.22; \u0026ldquo;Berlin\u0026rdquo;, P.31; \u0026ldquo;Schlobitten\u0026rdquo;, P.89; \u0026ldquo;Berlin\u0026rdquo;, P.92) are simply absent in the German translation. A representative example is found in paragraph P.22, where \u0026ldquo;Wesel\u0026rdquo; appears twice in the French source but only once in the German version, preventing full projection:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFrench annotated text:\u0026nbsp;\u003c/em\u003e\u003cbr\u003e\u0026nbsp;Nous retourn\u0026acirc;mes \u0026agrave; Wesel peu de jours apr\u0026egrave;s \u0026agrave; Wesel et feu S.M. le Roi de Prusse y vint avec toute sa Cour.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eGerman human translation for the edition, annotated with Claude:\u003c/em\u003e\u003cbr\u003e\u0026nbsp;Wenige Tage sp\u0026auml;ter kehrten wir nach Wesel zur\u0026uuml;ck, und Seine verstorbene K\u0026ouml;nigliche Hoheit, der K\u0026ouml;nig von Preu\u0026szlig;en kam mit seinem gesamten Hof dorthin.\u003c/p\u003e\n\u003cp\u003eAnother one was a very complex case (\u0026ldquo;y\u0026rdquo; from the french was not projected to \u0026ldquo;wo\u0026rdquo;, engl. \u0026ldquo;where\u0026rdquo;) in the german translation:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFrench annotated text:\u0026nbsp;\u003c/em\u003e\u003cbr\u003eLe {p\u0026egrave;re} de {ma belle\u0026ndash;m\u0026egrave;re} {y} \u0026eacute;tait gouverneur.\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u003cem\u003eGerman human translation for the edition, annotated with Claude:\u003c/em\u003e\u003cbr\u003e[...] \u003cstrong\u003ewo\u003c/strong\u003e der {Vater} {meiner Stiefmutter} Gouverneur war.\u003c/p\u003e\n\u003cp\u003eThe last missing annotation was the following, where \u0026ldquo;Pologne\u0026rdquo; has been translated with \u0026ldquo;polish conditions\u0026rdquo;:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFrench annotated text:\u0026nbsp;\u003cbr\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/em\u003eDieu me pr\u0026eacute;serve des modes de Pologne.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eGerman human translation for the edition, annotated with Claude:\u003c/em\u003e\u003cbr\u003eGott bewahre mich vor solchen \u003cstrong\u003ep[olnischen]\u003c/strong\u003e Zust\u0026auml;nden.\u003c/p\u003e\n\u003cp\u003eThis section analyzed the annotation projection results achieved with \u003cem\u003eClaude Sonnet 4\u003c/em\u003e, which showed the highest performance among the compared models. Most discrepancies stemmed from complex alignment issues, such as structural mismatches, paraphrased expressions, or omitted entities in the translation.\u003c/p\u003e\n\u003cp\u003eCompared to \u003cem\u003eClaude\u003c/em\u003e, the annotation projection results with \u003cem\u003eLlama\u003c/em\u003e were notably lower. False Negatives included simple correspondences like \u0026quot;Mon p\u0026egrave;re perdit ma m\u0026egrave;re [...].\u0026quot; and \u0026ldquo;Mein Vater verlor meine Mutter [...]\u0026rdquo; (P.6). that were left unannotated in the german output from the LLM, but also cases where entities like \u0026ldquo;mon beau\u0026ndash;p\u0026egrave;re une des s\u0026oelig;urs\u0026rdquo; (two entities) in the source were only partially tagged in the target as one entity. False Positives included both plausible annotations that were just not considered in the French text, but also implausible annotations. Additionally, more inconsistencies in the tagging of the actual text spans can be observed, like in some cases only \u0026ldquo;Tante\u0026rdquo; instead of \u0026ldquo;meine Tante\u0026rdquo; has been annotated. Otherwise, the model struggled with the same cases as \u003cem\u003eClaude\u003c/em\u003e, but also did not hallucinate entities that do not appear in the translation because of rephrasing. Overall, \u003cem\u003eLlama\u003c/em\u003e\u0026rsquo;s projections were less consistent and accurate than those produced by Claude, but it also has to be noted that the model is smaller, and also open-source.\u003c/p\u003e\n\u003cp\u003eThe results of the annotation projection using \u003cem\u003eGPT-4.1\u003c/em\u003e positioned its performance between that of \u003cem\u003eClaude\u003c/em\u003e and \u003cem\u003eLlama.\u003c/em\u003e Overall, \u003cem\u003eGPT\u003c/em\u003e performed well, successfully projecting a substantial number of annotations and correctly applying them to the appropriate text spans \u0026ndash; often including articles, pronouns, or capturing longer and more complex entities. Regarding false positives, \u003cem\u003eGPT\u0026rsquo;\u003c/em\u003es output resembled that of \u003cem\u003eLlama\u003c/em\u003e, occasionally annotating terms such as \u0026ldquo;Hof\u0026rdquo; or \u0026ldquo;Landleben\u0026rdquo; (P.11), as well as religious references like \u0026ldquo;Gott\u0026rdquo; or \u0026ldquo;Jesus\u0026rdquo; (P.5) which were not annotated in the original French input text. Regarding false negatives, \u003cem\u003eGPT\u003c/em\u003e exhibited similar patterns as observed with the other models, particularly in cases involving rephrasings, missing entities in the translation, or merged entities \u0026ndash; such as annotating \u0026ldquo;mon mari le comte de Schwerin\u0026rdquo; as a single entity rather than two, like in the French annotated input.\u003c/p\u003e\n\u003cp\u003eThe annotations transferred from the French annotated source text to the German translation by the LLM \u003cem\u003eQwen\u003c/em\u003e were mostly correctly tagged (see e.g. figure 5). The model inserted 20 additional tags from the French text, such as \u0026lt;bibl\u0026gt; or \u0026lt;seg\u0026gt;. In comparison to the other models, it did not transfer a significantly higher number of person annotations (101), but it produced also markedly fewer false positives \u0026ndash; only one was recorded.\u003c/p\u003e\n\u003cp\u003eThe projections that were not transferred also included many simple instances, such as \u0026ldquo;mon p\u0026egrave;re\u0026rdquo;, \u0026ldquo;cette fille\u0026rdquo;, and \u0026ldquo;ma tante\u0026rdquo;. In some cases, the annotation was transferred to a completely incorrect text span\u0026mdash;for example, in paragraph P.34, the ID for the aunt was assigned to the phrase \u0026ldquo;mich verlobt h\u0026auml;tte\u0026rdquo; (engl. \u0026ldquo;would have engaged me\u0026rdquo;).\u003c/p\u003e\n\u003ch3\u003eDiscussion\u003c/h3\u003e\n\u003cp\u003eAs in many semi\u0026ndash;automatic annotation workflows, a human-in-the-loop remains essential to review, adjust, and validate projected annotations \u0026ndash; particularly in cases of missing content, complex textual variation, or ambiguous translation. Nonetheless, the evaluation demonstrates that annotation projection using a LLM can be highly effective. E.g., using Claude, over 97% of the annotated person and place entities in the French source text were successfully transferred to the German translation to the correct spans. These strong results are especially notable given that the translation is not literal but often paraphrased or adapted for fluency.\u003c/p\u003e\n\u003cp\u003eThree recurring patterns account for most of the remaining unprojected annotations:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eVariant text spans\u003c/strong\u003e: The French source text sometimes contains multiple textual variants encoded with parallel structures (e.g., \u0026lt;app\u0026gt; and \u0026lt;rdg\u0026gt;), while the German translation retains only one version. This structural mismatch makes the full projection of all annotated variants not necessary and impossible.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eSimplified renderings for readability\u003c/strong\u003e: In several cases, entities are not projected onto the German version due to minor rephrasings or deliberate omissions aimed at improving the translation\u0026rsquo;s naturalness or readability.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eParaphrased entities\u003c/strong\u003e: The German translation occasionally expresses the same referent using reworded or generalized phrases. While such paraphrases often maintain semantic equivalence, they do not support direct one\u0026ndash;to\u0026ndash;one annotation projection, and a few references were not captured.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBeyond these particular complexities, the models reliably followed the annotation schema and produced well-structured, valid TEI/XML output with minimal intervention. Only two annotations were incorrectly added, and all projected spans were aligned correctly, with only minor deviations (e.g., the inclusion or omission of articles). Importantly, all @ref attributes containing project\u0026ndash;specific identifiers were preserved during transfer.\u003c/p\u003e\n\u003cp\u003eIn sum, LLM-based annotation projection provides a powerful method for accelerating bilingual corpus annotation. While it is often noted that human involvement remains essential for many tasks related to digital editions, this may not necessarily apply in the present case, as even human input is prone to minor errors. Greater control and refinement are especially required when employing smaller or open-source models such as \u003cem\u003eLlama\u003c/em\u003e. The challenges observed for the LLMs point into a different future field of discussion: editorial self-reflection. What are our intentions in the translations and the annotations? Do we prefer readability and naturalness over text surface based semantic annotations? Should we replace annotation with information extraction?\u003c/p\u003e\n\u003ch2\u003eExtraction of interpersonal relationships\u003c/h2\u003e\n\u003cp\u003eIn this small trial, a prompt-based extraction strategy was employed to derive interpersonal relationships from a TEI/XML-encoded paragraph (P.86)\u003ca href=\"#_ftn4\" name=\"_ftnref4\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e10\u003c/sup\u003e of the French normalized text. The source material, enriched with \u0026lt;persName\u0026gt; elements containing @ref attributes, which refer to the @xml:id of the entries in the person index, was processed using \u003cem\u003eClaude Sonnet 4\u003c/em\u003e (the Chat\u0026ndash;Bot) to generate RDF triples.\u003ca href=\"#_ftn5\" name=\"_ftnref5\" title=\"\"\u003e\u003c/a\u003e\u003csup\u003e11\u003c/sup\u003e The prompt instructed the model to identify relationships between the narrator \u0026ndash; the Countess of Schwerin \u0026ndash; and the entities mentioned in the text. The aim was to structure the information contained in the text into interoperable semantic data. The specific prompt used for this task was as follows:\u003c/p\u003e\n\u003cp\u003ePlease extract person relationships from this TEI/XML paragraph in RDF format. Use the IDs from the \u0026lt;persName\u0026gt;-Elements and also store the names of the persons and the gender. The narrator is the countess of Schwerin. For relationships, which are true in both directions (like siblings) only state the triple once. For in-law relationships use mother-in-law or father-in-law. Please also provide descriptions of the relationships you used: {french paragraph}\u003c/p\u003e\n\u003cp\u003eThe results of the chat can be accessed in the GitHub repository. The extracted data comprises named person entities annotated with FOAF (foaf:Person) types, along with literal values for their names and gender, and descriptive comments offering biographical context. Relationships among these individuals are encoded using the Relationship Ontology and a custom property (ex:livesWithOf), capturing various interpersonal ties including marital (rel:spouseOf), sibling (rel:siblingOf), in-law (rel:childInLawOf), and cousin (rel:cousinOf) relations. A human inspection of the paragraph reveals the following relationships:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eP153 (the countess, the narrator) is the spouse of P162 (the Count of Schwerin, Friedrich Wilhelm)\u003c/li\u003e\n \u003cli\u003eP158 (Quadt Wickradt, Ermgardt Maria of; married name Countess of Schwerin) is the spouse of P169 (Count of Schwerin, Otto)\u003c/li\u003e\n \u003cli\u003eP158 (Quadt Wickradt, Ermgardt Maria of) is the mother\u0026ndash;in\u0026ndash;law of the countess\u003c/li\u003e\n \u003cli\u003eP169 (Count of Schwerin, Otto) is the father\u0026ndash;in\u0026ndash;law of the countess\u003c/li\u003e\n \u003cli\u003eP170 (Countess of Schwerin, Ottonette Wilhelmine)\u0026nbsp;is the sister of P162 (Count of Schwerin)\u003c/li\u003e\n \u003cli\u003eP155 (Countess of Schwerin, Dorothea Maria) is the sister of P162 (Count of Schwerin)\u003c/li\u003e\n \u003cli\u003eP399 (Wittenhorst-Sonsfeld, Hermine Alexandrine of) is the sibling of P205 (Wittenhorst-Sonsfeld, Dorothea Luise von)\u003c/li\u003e\n \u003cli\u003eP399 (Wittenhorst-Sonsfeld, Hermine Alexandrine of) is the cousin of P162 (Count of Schwerin)\u003c/li\u003e\n \u003cli\u003eP208 (Wylich-Lottum, Miss of) is the cousin of P162 (Count of Schwerin)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eA manual inspection of the RDF output generated by the LLM confirms the accuracy and completeness of the relationship extraction. All major interpersonal ties expressed in the TEI/XML paragraph have been correctly identified and semantically modeled in general (see figure 6):\u003c/p\u003e\n\u003cp\u003eHowever, it is important to underscore that some of the properties employed \u0026ndash; such as \u0026ldquo;cousinOf\u0026rdquo; or \u0026ldquo;childInLawOf\u0026rdquo; \u0026ndash; are not part of the official Relationship Ontology. Although the relationships have been modelled accurately, aligning them with an established ontology remains a task that requires human editorial intervention. Nevertheless, this process might be facilitated through the use of targeted prompting strategies or by enriching the modelling phase with domain-specific information on relationship ontologies.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis paper has shown that LLMs, though originally developed for text generation, can be effectively repurposed for tasks central to digital scholarly editing. Numerous studies have demonstrated that LLMs often outperform traditional state-of-the-art NLP methods in many of these tasks. Moreover, they tend to be significantly easier to use. In the context of digital editions, which frequently rely on TEI/XML encoding, applying NLP techniques presents additional challenges. TEI/XML is not a standard input format for most NLP tools, necessitating substantial preprocessing and the involvement of pipelines to reintegrate NLP outputs back into the structured XML data.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe application of LLMs to NER on 18th-century French TEI/XML texts has yielded promising results. In particular, the task of annotation projection was executed with remarkable results, only using zero-shot prompting, demonstrating the model\u0026apos;s ability to transfer complex, semantically rich annotations across languages while preserving structural and contextual integrity; even though the target text was not a literal translation. However, LLMs also offer other new challenges.\u003c/p\u003e\n\u003cp\u003eLLM output must be validated for tag integrity and text preservation, as subtle changes in wording, punctuation, or markup may occur or elements may be lost. Second, while LLMs can reduce the manual workload involved in annotating and projecting semantic content, the role of the human editor remains indispensable for verification and quality control. Strategies, how to make this efficient, are not yet available: Are we doing everything twice? Are we creating harder work for humans, similar to post-editing of machine translation (e.g., Tora et al. 2018)? Do we have to reconsider our own intentions in the editorial tasks? For instance, it is important to emphasize that, although considerable effort is devoted to textual annotation within digital editions, in the domain of ego-documents, the majority of editorial time is spent on interpretation, in particular identifying entities such as named persons (i.e., entity linking). Third, the use of LLMs in scholarly contexts raises broader technical and ethical considerations. These include the opacity of model decisions, the reliance on commercial APIs, the potential for output drift across model versions, and the carbon cost of large-scale inference.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis study has not addressed the challenge of entity linking, but it points to a productive direction for further research. Supplying models with curated contextual sources could improve their ability to suggest relevant associations, especially when tracing historical actors and their relationships.\u003c/p\u003e\n\u003cp\u003eIn conclusion, this paper has explored the potential of large language models (LLMs) as tools for semi-automated annotation in digital scholarly editions. Rather than assuming that such models \u0026quot;understand\u0026quot; historical texts in a human sense, we suggest that their usefulness lies elsewhere: in their capacity to generate plausible linguistic associations that can be productively harnessed within human-guided editorial workflows. When carefully integrated into such frameworks, LLMs may indeed support and extend editorial practice \u0026ndash; not by replacing interpretation, but by provoking it.\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch1\u003eAcknowledgements and Credits\u003c/h1\u003e\n\u003cp\u003eSelina Galka conceptualized and performed all experiments, did the background research, and wrote the text. Georg Vogeler contributed to the conceptualization and minor parts of the text. We would like to thank Suzana Sagadin for carefully reading the manuscript, providing valuable comments, and assisting with the setup of the LLMs \u003cem\u003eQwen\u003c/em\u003e and \u003cem\u003eOlmo\u003c/em\u003e. We are also grateful to Ines Peper for her insightful content-related feedback on the source material.\u003c/p\u003e\n\u003cp\u003eThis research is part of the FWF project \u0026ldquo;\u003cstrong\u003e\u0026lsquo;\u003c/strong\u003eTout Vienne me riait\u003cstrong\u003e\u0026rsquo;.\u0026nbsp;\u003c/strong\u003eFamily and court relations in the memoirs of Countess Louise Charlotte von Schwerin (1684\u0026minus;1732)\u0026rdquo; (PI Ines Peper), Grant-DOI: 10.55776/P34943. Project blog: https://memoiren.hypotheses.org/.\u0026nbsp;\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eData that support the findings of the this study can be find in this GitHub-Repository: URL: https://github.com/selinagalka1/dses_and_llms\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eAkbik, Alan, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. \u0026lsquo;FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP\u0026rsquo;. In \u003cem\u003eProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)\u003c/em\u003e, edited by Waleed Ammar, Annie Louis, and Nasrin Mostafazadeh. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-4010.\u003c/li\u003e\n \u003cli\u003eAkbik, Alan, Duncan Blythe, and Roland Vollgraf. 2018. \u0026lsquo;Contextual String Embeddings for Sequence Labeling\u0026rsquo;. In \u003cem\u003eProceedings of the 27th International Conference on Computational Linguistics\u003c/em\u003e, edited by Emily M. Bender, Leon Derczynski, and Pierre Isabelle. Association for Computational Linguistics. https://aclanthology.org/C18-1139/.\u003c/li\u003e\n \u003cli\u003eAkbik, Alan, and Roland Vollgraf. 2018. \u0026lsquo;ZAP: An Open-Source Multilingual Annotation Projection Framework\u0026rsquo;. In \u003cem\u003eProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)\u003c/em\u003e, edited by Nicoletta Calzolari, Khalid Choukri, Christopher Cieri, et al. European Language Resources Association (ELRA). https://aclanthology.org/L18-1344/.\u003c/li\u003e\n \u003cli\u003eBamman, David, Alison Babeu, and Gregory Crane. 2010. \u0026lsquo;Transferring Structural Markup across Translations Using Multilingual Alignment and Projection\u0026rsquo;. \u003cem\u003eProceedings of the 10th Annual Joint Conference on Digital Libraries\u003c/em\u003e, 11\u0026ndash;20. https://doi.org/10.1145/1816123.1816126.\u003c/li\u003e\n \u003cli\u003eBehzad, Shabnam, Seth Ebner, Marc Marone, Benjamin Van Durme, and Mahsa Yarmohammadi. 2023. \u0026lsquo;The Effect of Alignment Correction on Cross-Lingual Annotation Projection\u0026rsquo;. \u003cem\u003eProceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)\u003c/em\u003e, 244\u0026ndash;51. https://doi.org/10.18653/v1/2023.law-1.24.\u003c/li\u003e\n \u003cli\u003eChen, Yang, Chao Jiang, Alan Ritter, and Wei Xu. 2023. \u0026lsquo;Frustratingly Easy Label Projection for Cross-Lingual Transfer\u0026rsquo;. In \u003cem\u003eFindings of the Association for Computational Linguistics: ACL 2023\u003c/em\u003e, edited by Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-acl.357.\u003c/li\u003e\n \u003cli\u003eCOMUTE Project. Funded by the DFG. Running from 2024-2027. URL: https://www.comute-project.de/projekt/index.html\u003c/li\u003e\n \u003cli\u003eEhrmann, Maud, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, and Antoine Doucet. 2023. \u0026lsquo;Named Entity Recognition and Classification in Historical Documents: A Survey\u0026rsquo;. \u003cem\u003eACM Computing Surveys\u003c/em\u003e 56 (2): 27:1-27:47. https://doi.org/10.1145/3604931.\u003c/li\u003e\n \u003cli\u003eEhrmann, Maud, Marco Turchi, and R. Steinberger. 2011. \u0026lsquo;Building a Multilingual Named Entity-Annotated Corpus Using Annotation Projection\u0026rsquo;. Paper presented at Recent Advances in Natural Language Processing. https://www.semanticscholar.org/paper/Building-a-Multilingual-Named-Entity-Annotated-Ehrmann-Turchi/5f55aa811875e9789aad1c30b41ee819db4c4f15.\u003c/li\u003e\n \u003cli\u003eFang, Zhao, Liang-Chun Wu, Xuening Kong, and Spencer Dean Stewart. 2025. \u0026lsquo;A Comparative Analysis of Word Segmentation, Part-of-Speech Tagging, and Named Entity Recognition for Historical Chinese Sources, 1900-1950\u0026rsquo;. \u003cem\u003eProceedings of the 5th International Conference on Natural Language Processing for Digital Humanities\u003c/em\u003e, 1\u0026ndash;6.\u003c/li\u003e\n \u003cli\u003eGalassi, Andrea, Kasper Drazewski, Marco Lippi, and Paolo Torroni. 2020. \u0026lsquo;Cross-Lingual Annotation Projection in Legal Texts\u0026rsquo;. \u003cem\u003eProceedings of the 28th International Conference on Computational Linguistics\u003c/em\u003e, 915\u0026ndash;26. https://doi.org/10.18653/v1/2020.coling-main.79.\u003c/li\u003e\n \u003cli\u003eGalka, Selina. 2025. Blog Post: Named Entity Recognition using Large Language Models \u0026ndash; A comparison. Project Blog: \u003cem\u003eThe Memoirs of Countess Schwerin.\u0026nbsp;\u003c/em\u003e2025. Accessed 2025-07-09. https://doi.org/10.58079/146m.j\u003c/li\u003e\n \u003cli\u003eGarc\u0026iacute;a-Ferrero, Iker, Rodrigo Agerri, and German Rigau. 2022. \u0026lsquo;T-Projection: High Quality Annotation Projection for Sequence Labeling Tasks\u0026rsquo;. Version 2. https://doi.org/10.48550/ARXIV.2212.10548.\u003c/li\u003e\n \u003cli\u003eGitHub-Repository for this paper. URL: https://github.com/selinagalka1/dses_and_llms/tree/main\u003cu\u003e.\u0026nbsp;\u003c/u\u003e\u003c/li\u003e\n \u003cli\u003eGonz\u0026aacute;lez-Gallardo, Carlos-Emiliano, Emanuela Boros, Nancy Girdhar, Ahmed Hamdi, Jose G. Moreno, and Antoine Doucet. 2023. \u0026lsquo;Yes but.. Can ChatGPT Identify Entities in Historical Documents?\u0026rsquo; \u003cem\u003e2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL)\u003c/em\u003e, June, 184\u0026ndash;89. https://doi.org/10.1109/JCDL57899.2023.00034.\u003c/li\u003e\n \u003cli\u003eGonz\u0026aacute;lez-Gallardo, Carlos-Emiliano, Hanh Thi Hong Tran, Ahmed Hamdi, and Antoine Doucet. 2024. \u0026lsquo;Leveraging Open Large Language Models for Historical Named Entity Recognition\u0026rsquo;. In \u003cem\u003eLinking Theory and Practice of Digital Libraries\u003c/em\u003e, edited by Apostolos Antonacopoulos, Annika Hinze, Benjamin Piwowarski, et al., vol. 15177. Lecture Notes in Computer Science. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-72437-4_22.\u003c/li\u003e\n \u003cli\u003eGreif, Gavin, Niclas Griesshaber, and Robin Greif. 2025. \u003cem\u003eMultimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents\u003c/em\u003e. Version 1. https://doi.org/10.48550/arxiv.2504.00414.\u003c/li\u003e\n \u003cli\u003eHiltmann, Torsten, Martin Dr\u0026ouml;ge, Nicole Dresselhaus, et al. 2025. \u0026lsquo;NER4all or Context Is All You Need: Using LLMs for Low-Effort, High-Performance NER on Historical Texts. A Humanities Informed Approach\u0026rsquo;. arXiv:2502.04351. Preprint, arXiv, February 4. https://doi.org/10.48550/arXiv.2502.04351.\u003c/li\u003e\n \u003cli\u003eHumphries, Mark, Lianne C. Leddy, Quinn Downton, et al. 2024. \u0026lsquo;Unlocking the Archives: Large Language Models Achieve State-of-the-Art Performance on the Transcription of Handwritten Historical Documents\u0026rsquo;. SSRN Scholarly Paper 5006071. Social Science Research Network, October 24. https://doi.org/10.2139/ssrn.5006071.\u003c/li\u003e\n \u003cli\u003eKripke, Saul. 1982. \u003cem\u003eNaming and Necessity.\u003c/em\u003e Harvard University Press, Cambridge.\u003c/li\u003e\n \u003cli\u003eKuchmiichuk, Pavlo. 2023. \u0026lsquo;Silver Data for Coreference Resolution in Ukrainian: Translation, Alignment, and Projection\u0026rsquo;. \u003cem\u003eProceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP)\u003c/em\u003e, 62\u0026ndash;72. https://doi.org/10.18653/v1/2023.unlp-1.8.\u003c/li\u003e\n \u003cli\u003eLevchenko, Maria. 2024. \u0026lsquo;Automatic Translation Alignment Pipeline for Multilingual Digital Editions of Literary Works\u0026rsquo;. \u003cem\u003eCHR 2024: Computational Humanities Research Conference\u003c/em\u003e, December 4\u0026ndash;6 (Aarhus, Denmark), 1086\u0026ndash;104.\u003c/li\u003e\n \u003cli\u003eNadeau, David, and Satoshi Sekine. 2007. \u0026lsquo;A Survey of Named Entity Recognition and Classification\u0026rsquo;. \u003cem\u003eLingvistic\u0026aelig; Investigationes\u003c/em\u003e 30 (1): 3\u0026ndash;26. https://doi.org/10.1075/li.30.1.03nad.\u003c/li\u003e\n \u003cli\u003ener-french, french flair model. URL: https://huggingface.co/flair/ner\u0026ndash;french.\u003c/li\u003e\n \u003cli\u003ePangakis, Nick, and Sam Wolken. 2025. \u0026lsquo;Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI\u0026rsquo;. \u003cem\u003eProceedings of the International AAAI Conference on Web and Social Media\u003c/em\u003e 19 (June): 1471\u0026ndash;92. https://doi.org/10.1609/icwsm.v19i1.35883.\u003c/li\u003e\n \u003cli\u003ePollin, Christopher, Franz Fischer, Patrick Sahle, Martina Scholger, and Georg Vogeler. 2025. \u0026bdquo;When it was 2024 - Generative AI in the field of digital scholarly editions\u0026ldquo;. \u003cem\u003eZeitschrift f\u0026uuml;r digitale Geisteswissenschaften\u003c/em\u003e, 10. https://doi.org/10.17175/2025_008.\u003c/li\u003e\n \u003cli\u003eRai, Pooja, and Sanjay Chatterji. 2023. \u0026lsquo;Annotation Projection-Based Dependency Parser Development for Nepali\u0026rsquo;. \u003cem\u003eACM Transactions on Asian and Low-Resource Language Information Processing\u003c/em\u003e 22 (2): 1\u0026ndash;19. https://doi.org/10.1145/3542696.\u003c/li\u003e\n \u003cli\u003eRastinger, Nina. 2024. Named Entity Recognition with LLMs. Presentation at the workshop series of the \u0026ldquo;DHd\u0026ndash;AG Angewandte Generative KI in den Digitalen Geisteswissenschaften\u0026rdquo; (AGKI\u0026ndash;DH). URL: https://agki\u0026ndash;dh.github.io/pages/webinar/page\u0026ndash;7.html.\u003c/li\u003e\n \u003cli\u003eRelationship Ontology. URL: https://vocab.org/relationship/.\u003c/li\u003e\n \u003cli\u003eSantos, Joaquim, Helena Freire Cameron, F. Olival, F\u0026aacute;tima Farrica, and Renata Vieira. 2024. \u0026lsquo;Named Entity Recognition Specialised for Portuguese 18th-Century History Research\u0026rsquo;. Paper presented at International Conference on Computational Processing of the Portuguese Language. https://www.semanticscholar.org/paper/Named-entity-recognition-specialised-for-Portuguese-Santos-Cameron/b1efd28f5ae7faeaff6e01a640497ac15d9d8028.\u003c/li\u003e\n \u003cli\u003eScholger, Martina, Elisa Cugliana, Franz Fischer, Christopher Pollin, Patrick Sahle, and Georg Vogeler. 2025. \u0026bdquo;Bias Und Evaluation \u0026ndash; Beitr\u0026auml;ge Zur Kritik Der Digitalen Edition Mit Generativer KI\u0026ldquo;. Workshop: Digitale Quelleneditionen und KI \u0026ndash; Aktuelle Tendenzen, Herausforderungen und Probleme, Wolfenb\u0026uuml;ttel, 10. April 2025, accessed 2025-07-12. https://zenodo.org/records/15299432.\u003c/li\u003e\n \u003cli\u003eSenator, Ferial, Abdelaziz Lakhfif, Imene Zenbout, Hanane Boutouta, and Chahrazed Mediani. 2025. \u0026lsquo;Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection\u0026rsquo;. \u003cem\u003eIEEE Access\u003c/em\u003e 13: 3707\u0026ndash;25. https://doi.org/10.1109/ACCESS.2025.3525493.\u003c/li\u003e\n \u003cli\u003eseqeval. URL:\u003cu\u003e\u0026nbsp;\u003c/u\u003ehttps://pypi.org/project/seqeval/\u003cu\u003e.\u0026nbsp;\u003c/u\u003e\u003c/li\u003e\n \u003cli\u003espaCy. URL:\u003cu\u003e\u0026nbsp;\u003c/u\u003ehttps://spacy.io/\u003cu\u003e.\u0026nbsp;\u003c/u\u003e\u003c/li\u003e\n \u003cli\u003estandoffconverter. URL: https://github.com/standoff\u0026ndash;nlp/standoffconverter.\u003c/li\u003e\n \u003cli\u003eSteinbach, Uli, and Ines Rehbein. 2019. \u0026lsquo;Automatic Alignment and Annotation Projection for Literary Texts\u0026rsquo;. In \u003cem\u003eProceedings of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature\u003c/em\u003e, edited by Beatrice Alex, Stefania Degaetano-Ortlieb, Anna Kazantseva, Nils Reiter, and Stan Szpakowicz. Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-2505.\u003c/li\u003e\n \u003cli\u003eTang, Xuemei, Zekun Deng, Qi Su, Hao Yang, and Jun Wang. 2024. \u0026lsquo;CHisIEC: An Information Extraction Corpus for Ancient Chinese History\u0026rsquo;. In \u003cem\u003eProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)\u003c/em\u003e, edited by Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue. ELRA and ICCL. https://aclanthology.org/2024.lrec-main.283/.\u003c/li\u003e\n \u003cli\u003eTersch, Harald. 2004. \u0026lsquo;Fr\u0026uuml;hneuzeitliche Selbstzeugnisse\u0026rsquo;. In \u003cem\u003eQuellenkunde der Habsburgermonarchie (16.-18. Jahrhundert): Ein exemplarisches Handbuch\u003c/em\u003e, edited by Josef Pauser, Martin Scheutz, and Thomas Winkelbauer, vol. 44. MI\u0026Ouml;G Erg\u0026auml;nzungsband. B\u0026ouml;hlau Verlag.\u003c/li\u003e\n \u003cli\u003eThomas, Alan, Robert Gaizauskas, and Haiping Lu. 2024. \u0026lsquo;Leveraging LLMs for Post-OCR Correction of Historical Newspapers\u0026rsquo;. In \u003cem\u003eProceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-COLING-2024\u003c/em\u003e, edited by Rachele Sprugnoli and Marco Passarotti. ELRA and ICCL. https://aclanthology.org/2024.lt4hala-1.14/.\u003c/li\u003e\n \u003cli\u003eToral, Antonio, Martijn Wieling, and Andy Way. 2018. \u0026lsquo;Post-Editing Effort of a Novel With Statistical and Neural Machine Translation\u0026rsquo;. \u003cem\u003eFrontiers in Digital Humanities\u003c/em\u003e 5 (May). https://doi.org/10.3389/fdigh.2018.00009.\u003c/li\u003e\n \u003cli\u003eT\u0026ouml;rnberg, Petter. 2024. \u0026lsquo;Best Practices for Text Annotation with Large Language Models\u0026rsquo;. \u003cem\u003eSociologica\u003c/em\u003e 18 (2): 2. https://doi.org/10.6092/issn.1971-8853/19461.\u003c/li\u003e\n \u003cli\u003eTudor, Crina, Beata Megyesi, and Robert \u0026Ouml;stling. 2025. \u0026lsquo;Prompting the Past: Exploring Zero-Shot Learning for Named Entity Recognition in Historical Texts Using Prompt-Answering LLMs\u0026rsquo;. In \u003cem\u003eProceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2025)\u003c/em\u003e, edited by Anna Kazantseva, Stan Szpakowicz, Stefania Degaetano-Ortlieb, Yuri Bizzoni, and Janis Pagel. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.latechclfl-1.19.\u003c/li\u003e\n \u003cli\u003eWang, Shuhe, Xiaofei Sun, Xiaoya Li, et al. 2025. \u0026lsquo;GPT-NER: Named Entity Recognition via Large Language Models\u0026rsquo;. In \u003cem\u003eFindings of the Association for Computational Linguistics: NAACL 2025\u003c/em\u003e, edited by Luis Chiruzzo, Alan Ritter, and Lu Wang. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-naacl.239.\u003c/li\u003e\n \u003cli\u003eXu, Derong, Wei Chen, Wenjun Peng, et al. 2024. \u0026lsquo;Large Language Models for Generative Information Extraction: A Survey\u0026rsquo;. \u003cem\u003eFrontiers of Computer Science\u003c/em\u003e 18 (6): 186357. https://doi.org/10.1007/s11704-024-40555-y.\u003c/li\u003e\n \u003cli\u003eYarowsky, David, and Grace Ngai. 2001. \u0026lsquo;Inducing Multilingual POS Taggers and NP Bracketers via Robust Projection Across Aligned Corpora\u0026rsquo;. Paper presented at NAACL 2001. \u003cem\u003eSecond Meeting of the North American Chapter of the Association for Computational Linguistics\u003c/em\u003e. https://aclanthology.org/N01-1026/.\u003c/li\u003e\n \u003cli\u003eYousef, Tariq, Chiara Palladino, Gerhard Heyer, and Stefan J\u0026auml;nicke. 2023. \u0026lsquo;Named Entity Annotation Projection Applied to Classical Languages\u0026rsquo;. \u003cem\u003eProceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature\u003c/em\u003e, 175\u0026ndash;82. https://doi.org/10.18653/v1/2023.latechclfl-1.19.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Footnotes","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003e The digital edition is being developed within the framework of an FWF-funded research project (2022\u0026ndash;2026). The project is jointly based at the Institute for the Study of the Habsburg Monarchy and the Balkan Region of the Austrian Academy of Sciences in Vienna, and the Department of Digital Humanities at the University of Graz. Project blog: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://memoiren.hypotheses.org/\u003c/span\u003e\u003cspan address=\"https://memoiren.hypotheses.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e Exact model version: claude-sonnet-4-20250514.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e GPT-4, LLaMA 3.3 (70B) und Claude Sonnet 3.5.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e GitHub-Repository accompanying this paper, URL: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/selinagalka1/dses_and_llms/tree/main\u003c/span\u003e\u003cspan address=\"https://github.com/selinagalka1/dses_and_llms/tree/main\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e The annotation of the text was carried out by Ines Peper, Michael P\u0026ouml;lzl, and Chiara Petrolini.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e The BIO (Begin\u0026ndash;Inside\u0026ndash;Outside) format is a common tagging scheme used in sequence labeling tasks such as named entity recognition. Each token in the text is labeled as either the beginning of an entity (B), inside an entity (I), or outside of any entity (O).\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eIn this context, 'rough' refers solely to the presence of doubled sentences offering alternative readings, the inconsistent transfer of variant text from different manuscripts, and the fact that the text has not yet undergone final correction.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e See European Open Source AI Index. URL: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://osai-index.eu/the-index\u003c/span\u003e\u003cspan address=\"https://osai-index.eu/the-index\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. E.g. the model often returned multiple paragraphs instead of a one-to-one projection, produced variant versions, included lists of extracted entities instead of inline annotations, or even inserted French entities into the German translation. However, it should be noted that OLMo-7B is a relatively small model, and larger versions could be explored in future work to assess potential improvements.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e The prompts for GPT and Llama were slightly more structured, incorporating bullet points, but conveyed the same instructions.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e The paragraph can be found in the input document containing the french text of the memoirs for the Annotation Projection in the GitHub-Repository.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e At the current stage of the edition, the @ref attribute contains only the identifier (ID). During the final transformation and publication phases, it will be expanded to include the full URI referencing the corresponding person entries, which will also be utilized in the RDF representation.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-digital-humanities","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ijdh","sideBox":"Learn more about [International Journal of Digital Humanities](http://link.springer.com/journal/42803)","snPcode":"42803","submissionUrl":"https://submission.nature.com/new-submission/42803/3","title":"International Journal of Digital Humanities","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Digital Edition, Memoirs, Named Entity Recognition, Annotation Projection, Large Language Models","lastPublishedDoi":"10.21203/rs.3.rs-7175875/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7175875/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis paper explores the use of large language models (LLMs) to enhance semantic annotation and annotation projection in digital scholarly editions (DSEs), focusing on historical ego-documents. Using the TEI/XML-encoded French memoirs of Countess Luise Charlotte of Schwerin (1684\u0026ndash;1732) and their German translation as a case study, we evaluate LLMs for Named Entity Recognition (NER), annotation transfer across aligned bilingual texts, and the extraction of interpersonal relationships. A comparative analysis with a traditional NER framework shows that LLMs significantly outperform baseline models, particularly in recognizing complex person references, such as non-rigid designators and nested entities. For annotation projection, we demonstrate that LLMs can reliably transfer entity annotations between French and German texts without intermediate alignment layers, achieving over 97% of correct projected entities using zero-shot prompting. Additionally, a pilot experiment illustrates the potential of LLMs for structured relationship modeling. The analysis of the errors puts further emphasis on the question of our intentions as editors when translating and indexing texts.\u003c/p\u003e","manuscriptTitle":"Annotating, Projecting, and Interpreting Named Entities in Digital Scholarly Editions with LLMs","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-14 09:59:26","doi":"10.21203/rs.3.rs-7175875/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-08-08T04:59:37+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-08T04:12:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"304697046218852399288216780744781844428","date":"2025-08-08T03:26:48+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-08-08T03:24:58+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-08T03:24:13+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-23T12:51:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Digital Humanities","date":"2025-07-21T09:44:17+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-digital-humanities","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ijdh","sideBox":"Learn more about [International Journal of Digital Humanities](http://link.springer.com/journal/42803)","snPcode":"42803","submissionUrl":"https://submission.nature.com/new-submission/42803/3","title":"International Journal of Digital Humanities","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"0ec9bd9d-3027-4ece-b4ba-fd5e1747dc72","owner":[],"postedDate":"August 14th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-02-02T16:03:52+00:00","versionOfRecord":{"articleIdentity":"rs-7175875","link":"https://doi.org/10.1007/s42803-025-00114-8","journal":{"identity":"international-journal-of-digital-humanities","isVorOnly":false,"title":"International Journal of Digital Humanities"},"publishedOn":"2026-01-28 15:59:22","publishedOnDateReadable":"January 28th, 2026"},"versionCreatedAt":"2025-08-14 09:59:26","video":"","vorDoi":"10.1007/s42803-025-00114-8","vorDoiUrl":"https://doi.org/10.1007/s42803-025-00114-8","workflowStages":[]},"version":"v1","identity":"rs-7175875","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7175875","identity":"rs-7175875","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}