How to Cut Your AI Training Bill by 80%? Oxfords New Optimizer Delivers 7.5x Faster Training by Optimizing How a Model Learns

Researchers at the University of Oxford have developed a novel optimizer called Fisher-Orthogonal Projection (FOP) that significantly reduces the computational costs associated with AI model training, achieving up to an 87% reduction in GPU expenses. By rethinking the way gradients are handled during training, FOP effectively optimizes the learning process, enabling models such as vision transformers trained on ImageNet-1K to be trained 7.5 times faster and more efficiently. This innovation addresses a critical bottleneck in AI development, where the high cost of GPU compute limits experimentation and progress across startups, research labs, and