Model releases accompanied with claims on performance that are not reproducible, code that is unavailable, incomplete, or difficult to run costs the scientific community time and effort. The following resources are valuable to help others replicate and verify the claims.
7 Reproducibility Resources
- Home /
- Foundation Model Resources /
- Reproducibility Resources
Reproducibility
Text 7
Speech 6
Vision 6
LM Evaluation Harness
Orchestration framework for standardizing LM prompted evaluation, supporting hundreds of subtasks.
TextSemver
A widely used protcol for versioning to software, to ensure easy reproducibility.
Text Speech Vision