The foundation learning approach in robotics and AI centers on using large-scale, pre-trained foundation models as the core for learning tasks. These models are expansive neural networks trained on diverse datasets-such as images, videos, text, and sensor data-to capture broad knowledge about the world. For robotics like Optimus, FMPL represents a shift from narrowly focused, task-specific training (as in RL) to a generalized, predictive framework that reasons about actions and outcomes. Essentially, this gives Optimus a “brain” loaded with a wide understanding of physics, objects, and human behavior, which it can then fine-tune for specific tasks, like folding a shirt, using minimal additional data. Inspired by foundation models in natural language processing (like GPT-4) and vision (like CLIP), this approach extends to robotics by integrating multimodal inputs such as vision, tactile feedback, and proprioception.
