Abstract: In philosophy of science, the epistemic status of XAI/MechInterp is widely debated. My position here is something to the effect of XAI/MechInterp can be used to tighten a probabilistic story, but not give you hard evidence (beyond what the induction methods we are trying to explain, already provide us with). But what about the humanities? Definitions of the humanities come in many flavors, from definitions emphasizing the study of the human condition by analytic, critical or speculative means, to definitions that focus on the intellectual attempt to broaden the possibilities of human freedom. Here, XAI/MechInterp can quite possibly become an important tool.

Abstract: Time is a concept central to artificial intelligence (AI) and natural language processing (NLP). The ability to perform temporal reasoning is a prerequisite for understanding narratives and is a fundamental component of many more sophisticated reasoning tasks. Nevertheless, large language models (LLMs) --- despite having revolutionised the fields of AI and NLP, and showcased excellent reasoning capabilities across a wide range of tasks --- still struggled with temporal reasoning. The task still remains one of the few anomalies where fine-tuned small LMs outperform LLMs. Moreover, the internal workings of these LLMs remain enigmatic "black boxes," and we seem to have no clear idea how to improve them on temporal reasoning. Nevertheless, we are living in an exciting era of LLM interpretation research. And, we have gathered several novel insights into how LLMs process temporal information and reason about temporal relations. In this talk, I will enumerate those findings and discuss the implication on LLMs' general reasoning capability beyond time. We begin with an overview of the current state of LLMs’ ability to process temporal information. Then, we explore how LLMs organize temporal information into intuitive, human-understandable structures and reshape it into other formats based on the requirements of specific temporal reasoning questions. We also present methods for evaluating these structures and their transformations, and connect them to LLMs' overall reasoning capacity. We believe our work offers new insights into how LLMs incorporate real-world concepts with their internal vectorized representations, and how they use this fusion to reason about a wide range of entities and relations.

Abstract: With the growing popularity of LLMs, the research community in natural language processing has made many new connections to interdisciplinary fields in the humanities. However, many studies neglect to put the human user into the center, so there often is a disconnect between the perception of NLP systems by the domain experts and the tools and observations applied research provides. This talk provides an overview on best practices in human evaluation, potential pitfalls such as biases in training data, and the benefits of explainability and interpretability techniques.

Abstract: Deep learning techniques have recently brought major improvements to Computational Literary Studies (CLS). Examples include named entity recognition, anaphora and coreference, sentiment analysis, scene detection or genre classification. Apart from natural language processing, graph learning has recently shown great potential for data with complex relational structure that can be represented as graphs or networks. In CLS, this abstraction is frequently used to study character networks, i.e. graphs where nodes represent literary characters and links represent relationships such as, e.g., their co-occurrence in a sentence or scene, or dialogue interactions. Building on this abstraction, we discuss how we can apply network analysis and graph learning to analyse the narrative structure of literary works. We specifically show how we can combine recent advances in (i) natural language processing, e.g. to recognize entities, resolve coreferences, extract meaningful character networks, or generate word embeddings, and (ii) graph neural networks, e.g. to compute latent space representations of fictional characters or address node- and link-level learning tasks. Demonstrating our approach in Tolkien’s Legendarium, we address the question to what extent deep graph learning techniques can leverage the topology of automatically extracted character networks, and which advantages they provide over word embeddings that just consider word-context pairs. We finally show how recent advances in temporal graph learning can be used to analyse the temporal narrative structure of literary texts.

Abstract: The dynamic evolution of scientific knowledge presents a rich domain for modeling semantic relationships as temporally evolving graphs. In this contribution, we explore how Graph Neural Networks (GNNs) can be leveraged to analyze and explain the development of scientific discourse through structured representations of historical texts. Our contribution is two-fold. Firstly, we propose a framework to predict structure in scientific corpora in the complete absence of a priori structure. To this end, we disentangle temporal message passing structure from the actual semantic structure that we want to predict. Secondly, we introduce a process to deduct corpus-level interpretations from explanations of single link prediction samples. To provide transparency into the learned representations, we apply higher-order metadata attribution methods that reveal how multi-attribute signals are integrated by the model and cluster the emerged subgraphs, that explain single samples, to reoccurring patterns in the corpus. This opens up pathways for collaborative, explainable AI in the digital humanities: AI agents that not only assist in detecting and modeling historical knowledge patterns, but also explain their interpretive process. Our work contributes to the broader goal of reframing historical knowledge production as a hybrid effort between human historians and AI-driven models.

Abstract: This paper investigates the efficiency of small fine-tuned foundation Models as a potential replacement for a combination of various tailored, simple machine learning models and rule-based and/or probabilistic approaches in a particular DH workflow. Using the extensive knowledge graph developed within the GRACEFUL17 project as a basis, we demonstrate how a pipeline consisting of Named Entity Recognition, Classification, Fuzzy Entity Matching and Linking, and data querying can be implemented through the use of an open source (small and large) language model. Performance in the individual steps of the pipeline matches or outperforms that of the individual task-tailored tools (such as SpaCy, gradient-boosted decision trees, or SPARQL templates with visual query building) previously used by the project, while arguably putting less burden on the DH scholar working within what is essentially a prompt-based architecture. The core proposition of the foundation model lies in its ability to unify multiple tasks within a single multitask framework through instruction fine-tuning and retrieval-augmented generation (RAG) methods. At the center of our investigation are questions of feasibility, as well as meta-scientific concerns surrounding reproducibility in the context of a digital humanities research project. To this end, we evaluate both relevant scoring metrics and requirements related to implementation, transparency, and usability.

Abstract: In the digital humanities landscape, natural language processing (NLP) holds transformative potential for low-resource languages, such as those in Africa and other under-resourced regions. This keynote explores how NLP techniques have been adapted to settings with limited resources, focusing on sentiment analysis, hate speech detection, emotion analysis, speech-to-text, and machine translation, derived from our work in African languages. Using large language models (LLMs), our projects have advanced research in low-resource languages, offering unique insights into both challenges and opportunities. Drawing from these experiences, my talk will demonstrate how the collaboration between NLP and humanities research can be applied to broader humanities data in low-resource domains, particularly emphasizing the analysis of historical materials such as manuscripts and religious texts. I will explore how AI can enhance our understanding of cultural narratives and historical contexts. At the Hub of Computing and Data Science (HCDS) and the Language Technology Research Group at the University of Hamburg (UHH), we developed the Sense Clustering over Time (SCoT) tool, which illustrates how similar methodologies can extend to various humanities data. The SCoT project provides dynamic visualizations of word meanings as they evolve, enhancing diachronic semantic understanding. This keynote emphasizes the critical role of interdisciplinary collaboration and AI methodologies in advancing our understanding of humanities data in low-resource settings. By integrating technological innovation (mostly LLM) with cultural heritage studies, we can open new research avenues and gain insights through temporal and linguistic contexts.

Abstract: Language change is not random—it is driven by shifting communicative goals, social structures, and domain-specific conventions (Hamilton et al. 2016, Gries 2008, Blank 2013). While various methods exist to model change in language use, such as lexical semantic change through word meaning representations from pre-trained language models (PLMs), interpretability remains a difficult task (Periti & Montanelli 2024). This impedes domain experts' ability to qualitatively assess change and adapt models to their specific needs (Beck 2024). For instance, historians care less about new NLP benchmarks to detect when words shift in meaning, but want to understand which constellation of contextual factors—such as who is writing, where, and when—drive these changes. We address the detection of language change in the context of scientific revolutions, where shifts in language use reflect broader epistemic transitions. Specifically, we want to model changes according to interactions of contextual factors involved in change. Modeling these interactions poses the challenges of quantifying context and knowledge and predicting interactions between evolving units of knowledge. To capture these dynamics, we propose a graph-based approach that represents scientific texts as knowledge graphs. Our approach builds on models of knowledge development as the filling of conceptual gaps in networks (Ju et al. 2020), while acknowledging the limitations of unsupervised clustering in concept modeling (Held 2022). We would present work in progress where we aim, first, to identify key linguistic signals of change using Kullback-Leibler Divergence (KLD; Kullback & Leibler 1951), a metric proven effective for identifying key linguistic features of change (Degaetano-Ortlieb & Teich 2018). We then aim to generate context-sensitive embeddings and part-of-speech labels for these features with PLMs (e.g. BERT). Finally, we want to apply temporal graphs (static and dynamic) to model the interactions between linguistic features and contextual features. This integrated approach would reveal interpretable patterns in the emergence of concepts, shifts, or their disappearance—providing meaningful tools for a comprehensive context-aware modeling of texts in the Digital Humanities.

References: Beck, C. (2024). Review of Tahmasebi, Borin, Jatowt, Xu & Hengchen (2021): Computational Approaches to Semantic Change. Journal of Historical Linguistics, 14(2), 376–384. https://doi.org/10.1075/jhl.22063.bec Blank, A. (2013). Why do new meanings occur? A cognitive typology of the motivations for lexical semantic change. In A. Blank & P. Koch (Eds.), Historical Semantics and Cognition (pp. 61–90). De Gruyter Mouton. https://doi.org/10.1515/9783110804195.61 Degaetano-Ortlieb, S., & Teich, E. (2018). Using relative entropy for detection and analysis of periods of diachronic linguistic change. In B. Alex, S. Degaetano-Ortlieb, A. Feldman, A. Kazantseva, N. Reiter, & S. Szpakowicz (Eds.), Proceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (pp. 22–33). Association for Computational Linguistics. https://aclanthology.org/W18-4503 Gries, S. T. (2008). The identification of stages in diachronic data: Variability-based neighbor clustering. Hamilton, W. L., Leskovec, J., & Jurafsky, D. (2016). Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1489–1501. https://doi.org/10.18653/v1/P16-1141 Held, M. (2022). Know thy tools! Limits of popular algorithms used for topic reconstruction. Quantitative Science Studies, 3(4), 1054–1078. https://doi.org/10.1162/qss_a_00217 Ju, H., Zhou, D., Blevins, A. S., Lydon-Staley, D. M., Kaplan, J., Tuma, J. R., & Bassett, D. S. (2020). The network structure of scientific revolutions. SocArXiv. https://doi.org/10.31235/osf.io/tga9c Kullback, S., & Leibler, R. A. (1951). On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1), 79–86. https://doi.org/10.1214/aoms/1177729694 Periti, F., & Montanelli, S. (2024). Lexical Semantic Change through Large Language Models: A Survey. ACM Comput. Surv., 56(11), 282:1-282:38. https://doi.org/10.1145/3672393

Abstract: This contribution presents an AI-driven framework developed within the Closing the Gap in Non-Latin Script Data project, aimed at enhancing the sustainability, accessibility, and interpretability of humanities data embedded in non-Latin scripts (NLS). Addressing the unique challenges posed by under-resourced and linguistically diverse datasets, this two-step strategy merges interactive data visualization with contextual AI dialogue to support knowledge modelling, analysis, and long-term digital preservation. The first step focuses on visualizing a sustainability assessment of NLS-focused research projects using grouped bar charts. Our custom measurement system evaluates six key indicators: presence on GitHub, data accessibility, website availability, web hosting durability, related academic publications, and proportion of open access publications. These metrics are designed to estimate the likelihood of data survival beyond the project lifecycle, revealing gaps in infrastructural support, discoverability, and long-term usability. The second step introduces a LLaMA3-based chatbot fine-tuned on sustainability-related literature and NLS digital humanities practices. This chatbot engages in memory-preserving, context-aware dialogue—answering user queries, performing cross-metric correlations, and dynamically modifying the visualization (e.g., highlighting a specific bar or trend in response to a question). Through this integration, the system fosters active exploration of low-resource data and provides meaningful insights into the infrastructural and epistemic sustainability of NLS research. Our approach contributes to AI modelling strategies in low-resource settings by aligning machine learning with interpretive humanities work. It demonstrates how multimodal, multilingual AI tools can facilitate not only data interpretation but also institutional critique—empowering scholars working with marginalized scripts and languages to assess and advocate for the digital futures of their data.

Abstract: Foundation models are transforming AI research, yet their application in the humanities remains underexplored. These large-scale models, trained on diverse corpora, hold immense potential for analyzing historical texts, cultural artifacts, and long-term societal trends. However, their applicability to humanistic inquiry is limited by three key challenges: (1) the temporal drift of language and meaning across centuries, (2) the subjectivity of interpretation in disciplines where multiple perspectives coexist, and (3) the lack of explainability, making AI-generated insights difficult for scholars to trust or critique. This talk explores strategies to adapt foundation models for humanities research. We discuss approaches for handling chronologically distributed data and we examine how multi-annotator learning can embrace disagreement in interpretation—crucial for fields where diverse viewpoints enrich analysis rather than distort it. Additionally, the talk will highlight techniques for making AI-generated insights transparent and interpretable, such as explainability, confidence estimation, and human-in-the-loop validation, ensuring that AI remains a collaborative tool rather than a black box. By adapting foundation models to historical and cultural contexts, we can develop AI systems that are not only powerful but also responsible and aligned with humanistic research values. This talk aims to bridge the gap between state-of-the-art AI and humanities scholarship, demonstrating how interdisciplinary collaboration can unlock new insights into our past and present.

Abstract: A central focus in the humanities is the interpretation of texts (broadly defined). This talk discusses some works that explore the limitations and affordances of foundation models and other computational methods in measuring different aspects of cultural meaning. We survey a broad range of ways in which classification is used in cultural analytics, and how foundation models can be used for culturally-relevant classification tasks. Then, we examine how combining modalities can be useful in isolating different dimensions of meaning. Finally, we discuss how the humanities and social sciences can aid in computational meaning-making by providing frameworks and definitions for ambiguous cultural concepts.

Abstract: As the Call for Papers has illustrated, while foundation models that are trained with unprecedented amount of human-generated data for general purposes are transforming the application of AI research in all fields, their applicability to the humanities remains underexplored. Specifically, adapting these models for complex historical sources very different from the material the algorithms were trained on has proved non-trivial in past research. In this paper, we focus on using object detection foundation models such as YOLO and CLIP to identify illustrations in two different but equally challenging historical corpora: East Asian historical sources and alchemical laboratory objects. Humanities scholars have applied object detection foundation models to sources such as newspapers and travellogues to either extract illustrations or offer illustration search based on compuational similarity measures or keyword search for illustrations without pre-assigned metadata keywords.[1] Their experiments based on foundation models work quite well for their specific European sources, which might due to the fact that the styles of such drawings are more similar to photos on which foundation models were trained. However, when it turns to older and more diverse historical sources, such as East Asian sources or 17th century alchemical apparatus, we found that further refinement has to be done to improve the applicability of such models to recognize East Asian styles of drawings in manuscripts and prints or recognize specific alchemical apparatus to a level of granularity deep enough so that it actually becomes useful for scholars. In the alchemical subproject, we work with a dataset of 12 types of laboratory objects. This task has proven more difficult than image classification tasks due to issues of granularity and visual similarity between the classes concerned. Initial attempts using few-shot learning, providing only a small number of examples, showed that the algorithm struggled to distinguish between objects, often confusing them either with the background or with each other. To address this, we created a more robust ground truth with granular annotations to better train the model in recognising specific alchemical objects. The initial model failed to reliably differentiate the objects due to these challenges in classification and visual similarity. In this paper, on the one hand, we will demonstrate the refining process necessary for obtaining useful results when applying foundation models to challenging Humanities data of the early modern period. We will report on our datasets, our parameters, and our adoption of algorithms and our results. The research goal of the East Asian subproject is to be able to idenfiy similar illustrations in East Asian sources as a way to trace the transmission paths of visual presentations in historical China, Japan, and beyond. The research goal of the alchemy subproject is to trace how technical illustrations were used and re-used across handbooks. [1] Such as Thomas Smits orDoris Gruber.

Abstract: Computational extraction of structured information from unstructured historical sources represents a significant methodological advance for the historical sciences. Large Language Models (LLMs) extend traditional Named Entity Recognition (NER) tasks - such as extracting names, places, or organisations - by enabling higher-order reasoning tasks, including argument mining (AM) or the inference of latent structures like semantic triples. Additionally, multimodal LLMs, which integrate text, image and audio data, are showing growing capabilities for directly extracting information from scanned printed or handwritten documents, identifying visual or mathematical arguments without the need of traditional Automatic Text Recognition (ATR = OCR + HTR) technologies. This extraction paradigm not only significantly enhances accessibility, speed, scalability, and flexibility but also enables new approaches for various structural analyses within the same framework, i.e. from scan to a specified structured output. A central application of this paradigm is the construction of knowledge graphs (KG). These graphs integrate diverse entities, relations, and attributes (e.g. time) into structured formats that support advanced analytical tasks, including network analysis or information retrieval. This in turn enables researchers to ask a wide range of questions, whether highly specific or structural, of the same corpus. Although some capabilities remain experimental/inaccurate, the rapid development of domain-specific and specialized models underscores the potential of LLMs to extract information based on specialized ontologies. Despite these advances, significant challenges remain, particularly in handling specialized language or symbol systems, disambiguation, validating extracted or inferred relationships, or handling the opacity of LLM outputs. We address these challenges using a modular, two-stage pipeline incorporating multiple validation steps and human-in-the-loop supervision. An initial ontology-agnostic Open Information Extraction (OIE) stage informs the development of a research-question-driven ontology in the second stage, which then guides the final structuring. To illustrate the potential and current limitations of this approach, we apply this pipeline to two biographical datasets: Stanisław Zieliński’s A Little Dictionary of Polish Colonial and Maritime Pioneers (1933) and Who Was Who in the GDR? alongside the German Communists: Biographical Handbook, 1918–1945. With these datasets we demonstrate how unstructured text can serve as a basis for extracting relational data via LLMs for further use and analysis. Networks constructed by us from these extractions reveal social, material, and semantic patterns that provide insights into the structural development of these groups of historical actors within their respective contexts. By focusing on the practical and theoretical implications of LLM-based extraction, we aim to highlight how this paradigm enables historians to scale analyses, revisit siloed historical sources with new questions and ask structural questions of previously unconnected information. While promising, as we will illustrate, the application of LLMs for KG extraction requires rigorous validation, critical reflection on and estimation off their limitations, and ideally interdisciplinary collaboration to maximize usefulness for historical research.

Abstract: Throughout the history of philosophy different conceptualisations about morality were elaborated to identify, ascribe, and explain moral responsibility of human agents. Although the modern term of “moral responsibility” is not an actor category until recent centuries, scholars have identified such views starting from the ancient period (e.g., Aristotle’s virtue ethics). Our research focuses on classical sources in moral philosophy that build a conceptual history of moral responsibility using various terminologies (from the ancient times to the mid-twentieth century), as reflected in the entries of the Stanford Encyclopedia of Philosophy, which is taken as a reference for contemporary discussions of the topic. This choice is also meant to avoid problems of multilingualism, as our corpus consists in modern English translations of classical texts. Trying to uncover the development of such conceptualisations in the history of philosophy by querying a corpus of primary texts with digital humanities tools (e.g., Voyant) provides only a partial image, as word frequencies and word distances rely on the choice of terms. Our methodology is to implement a semantic vector analysis to the selected corpus. We outline a novel computational approach that leverages Retrieval-Augmented Generation (RAG) in large language models (LLMs) to track conceptual shifts across a curated corpus of historical texts in moral philosophy. Our method involves: indexing, retrieval, and generation, allowing LLMs to dynamically integrate, analyze, and extract complex phenomena in humanities. Our starting point is to embed texts from distinct periods—such as the corpus of Aristotle’s writings, Kant’s texts on moral philosophy, and Mill’s works—into a vector database using tools like Facebook AI Similarity Search (FAISS); creating thus a framework that can be used to investigate moral concepts. This approach enables us to investigate when certain ideas related to moral responsibility first appeared, how they have been reinterpreted in successive eras, and the ways in which they contribute to a coherent longue durée history of moral philosophy. Our case study concentrates on demonstrating how a vector database can illuminate the development of moral theory of selected authors over time. In this paper we showcase an AI-based research workflow in which by adding multiple authors and integrating historical context dynamically into LLM-driven analysis, we achieve deeper insights into the evolution of philosophical concepts describing moral responsibility, outlining the way for scalable applications that can extend this methodology to other periods and authors in the history of philosophy.

Abstract: The AIKON platforms represent a comprehensive technical solution for integrating computer vision capabilities into humanities research workflows. This ecosystem enables scholars to leverage artificial intelligence for analyzing historical visual materials without requiring specialized technical expertise. AIKON's ecosystem centers around a shared GPU-powered API that provides vision processing services to two purpose-built applications: • AIKON-platform: a full-featured research platform designed for research teams supporting comprehensive corpus management and analysis for large-scale projects ; • AIKON-demo: a streamlined web application for rapid inference on individual documents. The technical core of AIKON provides a unified framework for various automated tasks on historical documents (region extraction, similarity search, watermark detection, vectorization, etc.), built on principles of modular design that enable independent component operation while preserving coherent data flow throughout the system. This architecture allows models to be swapped and accessed through standardized interfaces designed to tackle recurrent analytic needs of humanities researchers. AIKON's technical approach rests on a collaborative, iterative methodology that begins with training models on synthetic heritage documents, which can then be deployed within the API. Humanities scholars evaluate these models' results through purpose-built interfaces, making necessary refinements. These expert-validated corrections serve as high-quality training data for subsequent model fine-tuning, establishing an improvement cycle that enhances performance while building domain-specific foundation models tailored specifically to historical visual materials. Through this approach, AIKON overcomes limitations of conventional vision models in heritage contexts, giving researchers transparency and oversight over the automatic analytical processes, while simultaneously addressing the challenge of scarce domain-specific training data.

Abstract: I will present early work-in-progress for the ERC Starting Grant project ‘VERITRACE’ or “Traces de la verité: The reappropriation of ancient wisdom in early modern natural philosophy” (2023-2028). Based at Vrije Universiteit Brussel and led by Professor Cornelis J. Schilt, VERITRACE anticipates significant breakthroughs in our understanding of the role of ancient wisdom in the development of early modern natural philosophy by using state-of-the-art digital techniques to analyse a large corpus of early modern texts. During the Italian Renaissance, ancient texts like the the Corpus Hermeticum (including the Asclepius), the Chaldean and Sibylline Oracles, and the Orphic Hymns were seen as sources of true knowledge, forming a perennial philosophy (prisca sapientia) that offered insights into God, humanity, and the cosmos (Schmitt, 1966; Walker, 1972; Garin, 1984; Schmidt-Biggemann, 2004; Leinkauf, 2017). Major scientific figures like Copernicus, Kepler, Bacon, and Newton embraced this tradition, though its precise influence has hardly been studied. The VERITRACE project integrates close and distant reading of Renaissance and early modern texts. We do this by creating, and analysing, both a Close Reading Corpus (CRC) – which includes all the relevant editions of the four main corpora under scrutiny that were published during the Renaissance and early modern period but also works that drew heavily on these ancient wisdom writings (c. 150 texts) – and a Distant Reading Corpus (DRC). The Distant Reading method, closely related to natural language processing, allows for the analysis of large text corpora, identifying patterns and uncovering both prominent and neglected works. The DRC consists of several hundred thousand works (c. 430,000 texts) from important European library digital collections in Latin, French, German, Dutch, English and Italian. VERITRACE has developed a Text Matching tool that can be used to identify matching passages and phrases between source and target texts in our corpus of c.430,000 multilingual texts (printed books). This tool, a kind of early modern plagiarism detector, is an effective, surface-level tool, focused on a text’s most important keywords and vocabulary. This approach is also known as ‘text similarity analysis’ or ‘text reuse analysis’ (see, e.g. the innovative work of the Helsinki Computational History Group at the University of Helsinki, especially Rosson et al, 2023 and Ryan, 2023). While the Text Matching Tool is good at picking up lexical, surface-level similarities between textual passages, it is nearly useless across languages. It detects similarities in the actual words used – the vocabulary itself – not their semantic meaning. Our Semantic Text Matching Tool (powered by sentence embeddings) therefore covers this deeper understanding of meaning, by providing a way to identify semantic similarity between textual passages, despite no overlapping lexical similarity. We are thus able to identify passages written in one language (e.g. Latin) that are semantically similar to passages written in another language (e.g. French), bridging the ‘multilingual gap’. But a major hurdle remains, even if we can accurately capture multi-linguistic semantics: our multilingual corpus was published over the course of almost two hundred years (1540-1728), and thus is subject to significant semantic shift over time. How can we use sentence or word embeddings, to capture significant changes in meaning over time in multiple languages? Will an existing embeddings model be sufficient? If not, how might we fine-tune an existing model to produce diachronic embeddings? The goal is to be able to compare the semantic similarity of a passage from, e.g. a Latin text from 1575 to a passage from, e.g. a German text printed in 1715 - using a single embeddings model.

Abstract: This project investigates a clustering-based approach with representations of pretrained Vision Transformers (ViTs) to explore how effectively they capture stylistic elements from decorated initials in early modern books. This unsupervised method targets low-resource data, offering a path for style analysis in contexts where labeled datasets are scarce. By comparing ViT network representations with the well-established GRAM style representations of Convolutional Neural Networks, it becomes evident that the class token from a middle layer in ViTs is promising for capturing style elements for our purposes. This is further investigated using Explainable AI techniques. Subsequently examples are presented to illustrate potential historical insights.

Abstract: In recent years, machine learning has exhibited powerful capabilities in data mining and analysis, and it has been widely applied across various academic domains. In particular, it has shown valuable in the humanities, especially in the history of science, by facilitating the extraction of hidden information from historical documents. As a result, historians can leverage machine learning to analyze historical texts more efficiently and accurately. However, low resource availability and heterogeneity have posed significant challenges for researchers. In this work, we aim to analyze the development of historical scientific texts from the Sphaera corpus, which encompasses astronomical and cosmological knowledge from the 15th to 17th centuries. The existing dataset consists of scanned images of historical scientific texts. Due to the wide temporal span and geographical diversity of the sources, there is considerable variation in fonts and image quality across the pages. Directly conducting scans analysis introduces significant noise and complexity. Therefore, we first performed text extraction from the scanned images, enabling the use of NLP techniques for textual similarity analysis more effectively and with minimal manual effort. A natural approach is to use TF-IDF for statistical similarity analysis, as it is both simple and efficient. However, a major challenge arises from the high diversity of characters and symbols in historical texts, which leads to a dimensionality explosion when applying TF-IDF directly. To address this, we introduce novel methods to control the dimensionality. In this work, we present two distinct approaches: one enhances traditional TF-IDF, and the other leverages self-supervised learning to generate fixed-dimensional text embeddings, informed by the statistics provided by TF-IDF. For the latter approach, our goal is to explore the potential effectiveness of foundation models in handling heterogeneous historical text data.