(20:51) Josh unpacked the OpenAI’s robotics system that was trained entirely in simulation and deployed on a physical robot, which can learn a new task after seeing it done once (Read the blog post “Robots That Learn” and watch the corresponding video).
(28:41) Josh talked about the paper "Domain Randomization and Generative Models for Robotic Grasping”, which (1) explores a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis; and (2) proposes an autoregressive grasp planning model that maps sensor inputs of a scene to a probability distribution over possible grasps.
(36:05) Josh investigated his most recent work called “Geometry-Aware Neural Rendering” - which tackles the neural rendering problem of understanding the 3D structure of the world implicitly.
(28:55) Josh summarized the mental decision tree to debug and improve the performance of neural networks, as a reference to his talk "Troubleshooting Deep Neural Networks” at Reinforce Conf 2019 in Budapest.
(41:25) Josh discussed the limitations of domain randomization and what the solutions could look like, as a reference to his talk "Beyond Domain Randomization” at the 2019 Sim2Real workshop in Freiburg.
(44:52) Josh emphasized the importance of working on the right problems and focusing on the core principles in machine learning for junior researchers who want to make a dent in the AI research community.
(48:30) Josh is a co-organizer of Full-Stack Deep Learning, a training program for engineers to learn about production-ready deep learning.
