Unsupervised System 2 Thinking: The Next Leap in Machine Learning with Energy-Based Transformers

Energy-Based Transformers (EBTs) represent a significant advancement in AI by enabling unsupervised "System 2 Thinking," which involves slow, analytical, and multi-step reasoning akin to human cognition. Unlike traditional models that rely on domain-specific supervision, EBTs learn an energy function to evaluate the compatibility of input-prediction pairs, allowing machines to perform complex reasoning without restrictive training signals. This architectural innovation addresses the limitations of current AI systems that excel at fast, intuitive "System 1" tasks but struggle with deliberate reasoning, especially in out-of-distribution scenarios. By focusing on energy-based learning