‌AI for Cultural Preservation: Bridging Generative AI and Classical Methods to Decode East Asian Archives‌

This talk dives into the technical journey of transforming millions of unstructured historical East Asian records—from genealogies to Qing dynasty bureaucratic texts—into structured, machine-readable and human-readable formats required for modern AI applications. Faced with complex layouts (multi-column text, nested annotations) and degraded materials, we designed a pipeline that merges advanced layout extraction models, domain-tuned OCR for irregular scripts, and generative AI for context-aware text reconstruction. Statistical methods refine outputs to align with historical linguistics, mitigating AI hallucination risks, while custom NER models isolate key entities (names, dates, roles) to convert chaos into clean, searchable databases. By tackling the gap between unstructured archival content and the structured inputs LLMs demand, this work unlocks scalable analysis of cultural patterns—governance, lineage, migration—and offers a blueprint for turning fragile, analog archives into AI-ready datasets. Join us to explore how hybrid AI systems can bridge centuries-old texts with tomorrow’s generative tools, making humanity’s collective memory accessible in the age of machine intelligence.

We aim to foster a collaborative dialogue—getting feedback on our technical approaches and challenges while exchanging insights.