Cost Optimization for ML

ML cost problems are often allocation problems. Expensive hardware runs idle, small models are served on GPUs, batchable work is done synchronously, old checkpoints never expire, and experiments run without budgets.

Most cost optimization comes from matching each workload to the right hardware, freshness tier, and quality level, which usually leaves model quality untouched.