Transcribing the Cairo Genizah with Multimodal AI

What can medieval documents teach us about 21st century large language models and what can AI teach us about them

Example experiment using one of our fine-tuned models to transcribe T-S NS 219.40 a Cairo Genizah fragment of Babylonian Talmud Megillah 2b. Shown via LMS Studio.

The Cairo Genizah is one of the richest collections of right to left language documents from the medieval ages, shedding light on religious practices, trade routes, marriages/divorces and court proceedings. Yet despite more than a century of study, only an estimated 15% of the documents in the Cairo Genizah have been transcribed. Without accurate transcriptions, both semantic and keyword searches over the Genizah corpus are limited, and text-only large language models cannot reference the source text of fragments. Moreover, even looking at the raw images of fragments from the Cairo Genizah and discerning characters is often difficult for non-experts in the field. Having a clean and accurate transcription of text is critical for many areas of scholarship and study.

On a more fundamental level, from an AI perspective, the question of how to adapt multimodal models to novel or adjacent domains is fascinating. Using the Cairo Genizah as a test bed, we can observe how much training data is needed for models to generalize to likely never-before-seen languages/character styles, such as Judeo-Arabic and variants of Rashi script. We can also explore how…