Google DeepMind Finds a Fundamental Bug in RAG: Embedding Limits Break Retrieval at Scale

Google DeepMind has identified a fundamental architectural limitation in Retrieval-Augmented Generation (RAG) systems stemming from the fixed-dimensional nature of dense embeddings, which restricts their ability to scale effectively as document databases grow. The research reveals that the representational capacity of embeddingsdetermined by their dimensionalitylimits the number of documents that can be accurately retrieved: approximately 500,000 for 512-dimensional vectors, 4 million for 1024 dimensions, and up to 250 million for 4096 dimensions, based on theoretical bounds. This limitation persists despite improvements in model size or training techniques,