Releasing all datasets involved in the development of a Foundation Model, including training, fine-tuning, and evaluation data, can facilitate external scrutiny and support further research. Proper data governance practices, including respecting opt-out preference signals, pseudonymization, or PII redaction, are required at the curation and release stages. Data access control based on research needs and enabling data subjects to request removal from the hosted dataset are essential.
A paper detailing the data governance decisions undertaken during BigScience’s BLOOM project.
A paper that argues for the creation of a public data trust for collective input into the creation of AI systems and analyzes the feasibility of such a data trust.
A report outlining governance questions, approaches, and tooling in the BigCode project, with a focus on Data governance
A tool to let software developers check whether their code was included in TheStack dataset and opt out of inclusion in future versions
A model trained on a new dataset of PII in code used for pseudonymization of a dataset prior to training
A set of resource sheets focused on GDPR compliance covering legal basis for data collection, sharing, and best practices for handling personal data
An AI governance testing framework and software toolkit that help industries be more transparent about their AI to build trust. Foundation is fostering a community to contribute to the use and development of AI testing frameworks, code base, standards, and best practices.
Croissant is an open-source metadata format developed by MLCommons to standardise the description of machine learning (ML) datasets, enhancing their discoverability, portability, and interoperability across various tools and platforms. It builds upon the schema.org vocabulary, extending it to encapsulate ML-specific attributes, including dataset structure, resources, and semantics. Croissant is particularly relevant in scenarios requiring consistent dataset documentation to facilitate seamless integration into ML workflows.
