Shows
ArgmaxMixture of ExpertsIn this episode we talk about the paper "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean.2024-10-0854 minArgmaxLoRAWe talk about Low Rank Approximation for fine tuning Transformers. We are also on YouTube now! Check out the video here: https://youtu.be/lLzHr0VFi3Y2023-09-021h 02Argmax15: InstructGPTIn this episode we discuss the paper "Training language models to follow instructions with human feedback" by Ouyang et al (2022). We discuss the RLHF paradigm and how important RL is to tuning GPT.2023-03-2857 minArgmax14: WhisperThis week we talk about Whisper. It is a weakly supervised speech recognition model.2023-03-1749 minArgmax13: AlphaTensorWe talk about AlphaTensor, and how researchers were able to find a new algorithm for matrix multiplication.2023-03-1149 minArgmax12: SIRENsIn this episode we talked about "Implicit Neural Representations with Periodic Activation Functions" and the strength of periodic non-linearities.2022-10-2554 minArgmax11: CVPR Workshop on Autonomous Driving Keynote by Ashok Elluswamy, a Tesla engineerIn this episode we discuss this video: https://youtu.be/jPCV4GKX9DwHow Tesla approaches collision detection with novel methods.2022-10-0148 minArgmax10: Outracing champion Gran Turismo drivers with deep reinforcement learningWe discuss Sony AI's accomplishment of creating a novel AI agent that can beat professional racers in Gran Turismo. Some topics include:- The crafting of rewards to make the agent behave nicely- What is QR-SAC?- How to deal with "rare" experiences in the replay bufferLink to paper: https://www.nature.com/articles/s41586-021-04357-72022-08-2354 minArgmax8: GATO (A Generalist Agent)Today we talk about GATO, a multi-modal, multi-task, multi-embodiment generalist agent.2022-07-3044 minArgmax9: Heads-Up Limit Hold'em Poker Is SolvedToday we talk about recent AI advances in Poker; specifically the use of counterfactual regret minimization to solve the game of 2-player Limit Texas Hold'em.2022-07-3047 minArgmax7: Deep Unsupervised Learning Using Nonequilibrium Thermodynamics (Diffusion Models)We start talking about diffusion models as a technique for generative deep learning.2022-06-1430 minArgmax6: Deep Reinforcement Learning at the Edge of the Statistical PrecipiceWe discuss NeurIPS outstanding paper award winning paper, talking about important topics surrounding metrics and reproducibility.2022-06-071h 01Argmax5: QMIXWe talk about QMIX https://arxiv.org/abs/1803.11485 as an example of Deep Multi-agent RL.2022-04-2642 minArgmax4: Can Neural Nets Learn the Same Model Twice?Todays paper: Can Neural Nets Learn the Same Model Twice? Investigating Reproducibilityand Double Descent from the Decision Boundary Perspective (https://arxiv.org/pdf/2203.08124.pdf)Summary:A discussion of reproducibility and double descent through visualizations of decision boundaries.Highlights of the discussion:Relationship between model performance and reproducibilityWhich models are robust and reproducibleHow they calculate the various scores2022-04-0655 minArgmax3: VICRegTodays paper: VICReg (https://arxiv.org/abs/2105.04906)Summary of the paperVICReg prevents representation collapse using a mixture of variance, invariance and covariance when calculating the loss. It does not require negative samples and achieves great performance on downstream tasks.Highlights of discussionThe VICReg architecture (Figure 1)Sensitivity to hyperparameters (Table 7)Top 5 metric usefulness2022-03-2244 minArgmax2: data2vecTodays paper: data2vec (https://arxiv.org/abs/2202.03555)Summary of the paperA multimodal SSL algorithm that predicts latent representation of different types of input.Highlights of discussionWhat are the motivations of SSL and multimodalHow does the student teacher learning work?What are similarities and differences between ViT, BYOL, and Reinforcement Learning algorithms.2022-03-0753 minArgmax1: Reward is EnoughThis is the first episode of Argmax! We talk about our motivations for doing a podcast, and what we hope listeners will get out of it.Todays paper: Reward is Enough Summary of the paperThe authors present the Reward is Enough hypothesis: Intelligence, and its associated abilities, can be understood as subserving the maximisation of reward by an agent acting in its environment.Highlights of discussionHigh level overview of Reinforcement LearningHow evolution can be encoded as a reward maximization problemWhat is the one reward signal...2022-02-2154 min