Foundation model development is often resource intensive. The following tools help one to measure energy consumption and estimate the carbon intensity of the energy source used. Decisions made during or prior to model training can have a significant effect on the upstream and downstream environmental impact of a given model.
Monitoring the environmental impact of training machine learning models on Azure
carbontracker is a tool for tracking and predicting the energy consumption and carbon footprint of training deep learning models as described in Anthony et al. (2020).
Estimate and track carbon emissions from your computer, quantify and analyze their impact.
A comprehensive account of the broader environmental impact of the BLOOM language model.
The experiment-impact-tracker is meant to be a simple drop-in method to track energy usage, carbon emissions, and compute utilization of your system. Currently, on Linux systems with Intel chips (that support the RAPL or powergadget interfaces) and NVIDIA GPUs, we record: power draw from CPU and GPU, hardware information, python package versions, estimated carbon emissions information, etc. In California we even support realtime carbon emission information by querying caiso.com!
Tracking the emissions of using Google’s cloud compute resources
Uncovering and Addressing the Secret Water Footprint of AI Models, and estimating water usage for training and deploying LLMs.
Provide scaling laws to determine the optimal allocation of a fixed compute budget.
Provides details on the optimal model size and number of tokens for training a transformer-based language model in a given computational budget.
