Sakana AI's CTO says he's 'absolutely sick' of transformers, the tech that powers every major AI model

Ashish Vaswani, co-author of the groundbreaking 2017 paper "Attention Is All You Need" that introduced the transformer architecture foundational to modern AI, publicly criticized the field for becoming overly fixated on this single approach. Speaking at an AI conference in San Francisco, Vaswani highlighted how investor pressure and intense competition have narrowed research focus, prompting him to step away from transformers as CTO of Tokyo-based AI startup, instead seeking new paradigms beyond the dominant transformer model.