Meta AI Researchers Release MapAnything: An End-to-End Transformer Architecture that Directly Regresses Factored, Metric 3D Scene Geometry

Meta Reality Labs and Carnegie Mellon University have developed MapAnything, an innovative end-to-end transformer architecture capable of directly regressing factored metric 3D scene geometry from images and sensor inputs. Unlike traditional modular pipelines that require extensive task-specific tuning and post-processing, MapAnything supports over 12 distinct 3D vision tasks within a single feed-forward pass, significantly streamlining the 3D reconstruction process. This model advances the field by accepting up to 2,000 input images simultaneously and flexibly incorporating auxiliary data such as camera intrinsics, poses, and depth maps. It produces accurate metric