Introduction

• Balancing Regulation with Data Policies:
• Need for a Dual Approach: Effective AI governance requires more than just regulatory frameworks. It is crucial to also promote policies that treat high-quality data as a public good.
• Objective: This approach aims to enhance transparency, create equitable opportunities, and build public trust in AI technologies.

Data as the Lifeblood of AI

• Significance of Data in AI Development:
• Neural Scaling Laws: AI, particularly LLMs, thrives on large datasets. The volume and diversity of data directly influence model performance, with more data typically leading to better results.
• Comparison with Other Factors: While computing power and algorithmic innovations are important, data is often considered the most critical driver of progress in AI.
• Current State of Training Datasets:
• Massive Datasets: For example, Meta’s LLama 3 is trained on 15 trillion tokens, vastly exceeding the size of the British Library’s collection.
• Future Concerns: Studies suggest that data quality issues and potential ''peak data''—where data supply cannot meet growing demands—might arise before 2030. There''s also the risk of data contamination through feedback loops that exacerbate biases.

Ethical Concerns and Data Practices

• Ethics of Data Acquisition:
• Data Race: The intense competition for data can lead to ethical lapses. An example is the use of pirated books, known as ‘Books3,’ which some believe contribute to training leading LLMs.
• Legal and Ethical Debates: The legality and fairness of using such data are contested. The lack of clear ethical guidelines exacerbates these concerns.
• Quality and Bias Issues:
• Content Sources: LLMs are trained on a mix of licensed content, public data, and social media interactions. This training can reflect and reinforce existing biases, particularly in terms of language and cultural representation.
• Anglophone Bias: Current datasets are often skewed towards English and contemporary content, which limits the representativeness of AI outputs.

The Absence of Primary Sources

• Limitations of Existing LLM Training Data:
• Secondary vs. Primary Sources: LLMs primarily rely on secondary sources and may miss out on primary sources such as archival documents, oral traditions, and historical texts.
• Cultural Biases: The absence of diverse primary sources means that LLMs lack comprehensive coverage of global cultural and linguistic diversity.
• Potential of Untapped Data:
• Archival Riches: Examples include Italy’s State Archives, which hold extensive historical documents. Such archives, if digitized, could significantly enhance AI’s cultural and historical knowledge base.
• Volume of Data: The data contained in archives globally could rival or exceed current training datasets, offering a rich resource for improving AI models.

Potential Benefits of Harnessing Cultural Heritage

• Enrichment of AI:
• Cultural Understanding: Digitizing and utilizing primary sources would provide AI models with a deeper understanding of human culture and history.
• Preservation and Accessibility: Such data can help preserve cultural heritage and make it more accessible, safeguarding it from threats like war and climate change.
• Economic and Innovation Benefits:
• Opportunities for Smaller Entities: Publicly available data would enable start-ups and smaller companies to develop AI applications, fostering innovation and creating a more competitive market.
• Global Innovation: Free and transparent data access can drive global technological advancements and innovation by levelling the playing field against major tech companies.

Examples from Italy and Canada

• Italy’s Digital Library Project:
• Ambitious Goals: Italy initially invested €500 million in the ‘Digital Library’ project to digitize its cultural heritage and make it publicly accessible.
• Challenges: The project faced restructuring and reduced priority, reflecting a missed opportunity for leveraging AI and digital humanities.
• Canada’s Official Languages Act:
• Historical Context: Initially criticized as wasteful, Canada’s bilingual policy provided valuable datasets for translation technologies.
• Broader Implications: The success of this policy highlights the benefits of digitizing and promoting diverse languages, including those with fewer resources.

Conclusion

• Preserving and Democratizing Knowledge:
• Digital Transition: As the digital landscape evolves, prioritizing the digitization of cultural heritage is essential for preserving historical knowledge and making it accessible.
• Inclusive AI Innovation: Harnessing these resources can lead to more inclusive and equitable AI development, ensuring that benefits are distributed globally.