What is Universality in LLMs? How to Find Universal Neurons

Research indicates that independently trained transformer models develop similar neuron activation patterns, suggesting the presence of universal neurons that underpin core linguistic and cognitive functions across different instances of large language models (LLMs). This discovery highlights a potential intrinsic structure within transformer architectures, where certain neurons consistently encode specific features or concepts, regardless of training variations, thereby advancing our understanding of model interpretability and the fundamental principles of neural network universality.