Shows
New Paradigm: AI Research SummariesHow Does AGORA BENCH Compare Language Models in Synthetic Data Generation?This episode analyzes the study "Evaluating Language Models as Synthetic Data Generators" by Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig, affiliated with institutions such as Carnegie Mellon University and KAIST AI. The discussion centers on the introduction of AGORA BENCH, a benchmark designed to assess the effectiveness of various language models in generating high-quality synthetic data.The episode delves into the comparative performance of six prominent language models, including GPT-4o and Claude-3.5-Sonnet, highlighting their distinct strengths in data generation tasks. It explores...
2024-12-1005 min
The Thesis Review[48] Tianqi Chen - Scalable and Intelligent Learning SystemsTianqi Chen is an Assistant Professor in the Machine Learning Department and Computer Science Department at Carnegie Mellon University and the Chief Technologist of OctoML. His research focuses on the intersection of machine learning and systems.
Tianqi's PhD thesis is titled "Scalable and Intelligent Learning Systems," which he completed in 2019 at the University of Washington. We discuss his influential work on machine learning systems, starting with the development of XGBoost,an optimized distributed gradient boosting library that has had an enormous impact in the field. We also cover his contributions to deep learning frameworks like MXNet and machine learning compilation...
2024-10-2846 min
The Thesis Review[47] Niloofar Mireshghallah - Auditing and Mitigating Safety Risks in Large Language ModelsNiloofar Mireshghallah is a postdoctoral scholar at the University of Washington. Her research focuses on privacy, natural language processing, and the societal implications of machine learning. Niloofar completed her PhD in 2023 at UC San Diego, where she was advised by Taylor Berg-Kirkpatrick.
Her PhD thesis is titled "Auditing and Mitigating Safety Risks in Large Language Models." We discuss her journey into research and her work on privacy and LLMs, including how privacy is defined, common attacks and mitigations, differential privacy, and the balance between memorization and generalization.
- Episode notes: www.wellecks.com/thesisreview/episode47.html
- Follow the Thesis...
2024-10-151h 17The Thesis Review[46] Yulia Tsvetkov - Linguistic Knowledge in Data-Driven NLPYulia Tsvetkov is a Professor in the Allen School of Computer Science & Engineering at the University of Washington. Her research focuses on multilingual NLP, NLP for social good, and language generation.
Yulia's PhD thesis is titled "Linguistic Knowledge in Data-Driven Natural Language Processing", which she completed in 2016 at CMU.
We discuss getting started in research, then move to Yulia's work in the thesis that combines ideas from linguistics and natural language processing. We discuss low-resource and multilingual NLP, large language models, and great advice about research and beyond.
- Episode notes: www.wellecks.com/thesisreview/episode46.html
- Follow the...
2023-08-1259 minThe Thesis Review[45] Luke Zettlemoyer - Learning to Map Sentences to Logical FormLuke Zettlemoyer is a Professor at the University of Washington and Research Scientist at Meta. His work spans machine learning and NLP, including foundational work in large-scale self-supervised pretraining of language models.
Luke's PhD thesis is titled "Learning to Map Sentences to Logical Form", which he completed in 2009 at MIT. We talk about his PhD work, the path to the foundational Elmo paper, and various topics related to large language models.
- Episode notes: www.wellecks.com/thesisreview/episode45.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about the show...
2023-07-2559 min
CS224USasha Rush on NLP research, engineering, and educationCoding puzzles, practices, and education, structured prediction, the culture of Hugging Face, large models, and the energy of New York.
Transcript: https://web.stanford.edu/class/cs224u/podcast/rush/
Sasha's website
Sasha on Twitter
Sasha on the Humans of AI podcast
Sasha on The Thesis Review Podcast with Sean Welleck
Sasha on the Talking Machines Podcast
Sasha interviewed by Sayak Paul
Hugging Face
PyTorch
The Annotated Transformer
The Annotated Alice
The Annotated S4
Sasha and Dan Oneață's declarative graphics library Chalk
Drawing Big Ben in Chalk
OpenNMT
Ken Shan
Blog post by Ken and Dy...
2022-10-041h 22
The Thesis Review[44] Hady Elsahar - NLG from Structured Knowledge Bases (& Controlling LMs)Hady Elsahar is a Research Scientist at Naver Labs Europe. His research focuses on Neural Language Generation under constrained and controlled conditions.
Hady's PhD was on interactions between Natural Language and Structured Knowledge bases for Data2Text Generation and Relation Extraction & Discovery, which he completed in 2019 at the Université de Lyon.
We talk about his phd work and how it led to interests in multilingual and low-resource in NLP, as well as controlled generation. We dive deeper in controlling language models, including his interesting work on distributional control and energy-based models.
- Episode notes: www.wellecks.com/thesisreview/episode44.html
...
2022-08-231h 05The Thesis Review[43] Swarat Chaudhuri - Logics and Algorithms for Software Model CheckingSwarat Chaudhuri is an Associate Professor at the University of Texas. His lab studies problems at the interface of programming languages, logic and formal methods, and machine learning.
Swarat's PhD thesis is titled "Logics and Algorithms for Software Model Checking", which he completed in 2007 at the University of Pennsylvania.
We discuss reasoning about programs, formal methods & safer machine learning systems, and the future of program synthesis & neurosymbolic programming.
- Episode notes: www.wellecks.com/thesisreview/episode43.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about the show at www.wellecks...
2022-06-281h 06The Thesis Review[42] Charles Sutton - Efficient Training Methods for Conditional Random FieldsCharles Sutton is a Research Scientist at Google Brain and an Associate Professor at the University of Edinburgh. His research focuses on deep learning for generating code and helping people write better programs.
Charles' PhD thesis is titled "Efficient Training Methods for Conditional Random Fields", which he completed in 2008 at UMass Amherst. We start with his work in the thesis on structured models for text, and compare/contrast with today's large language models. From there, we discuss machine learning for code & the future of language models in program synthesis.
- Episode notes: https://cs.nyu.edu/~welleck/episode42.html
...
2022-04-191h 18The Thesis Review[41] Talia Ringer - Proof RepairTalia Ringer is an Assistant Professor with the Programming Languages, Formal Methods, and Software Engineering group at University of Illinois Urbana-Champaign. Her research focuses on formal verification and proof engineering technologies.
Talia's PhD thesis is titled "Proof Repair", which she completed in 2021 at the University of Washington.
We discuss software verification and her PhD work on proof repair for maintaining verified systems, and discuss the intersection of machine learning with her work.
- Episode notes: https://cs.nyu.edu/~welleck/episode41.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about...
2022-03-301h 19The Thesis Review[40] Lisa Lee - Learning Embodied Agents with Scalably-Supervised RLLisa Lee is a Research Scientist at Google Brain. Her research focuses on building AI agents that can learn and adapt like humans and animals do.
Lisa's PhD thesis is titled "Learning Embodied Agents with Scalably-Supervised Reinforcement Learning", which she completed in 2021 at Carnegie Mellon University.
We talk about her work in the thesis on reinforcement learning, including exploration, learning with weak supervision, and embodied agents, and cover various topics related to trends in reinforcement learning.
- Episode notes: https://cs.nyu.edu/~welleck/episode40.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find...
2022-03-0946 minThe Thesis Review[39] Burr Settles - Curious Machines: Active Learning with Structured InstancesBurr Settles leads the research group at Duolingo, a language-learning website and mobile app whose mission is to make language education free and accessible to everyone.
Burr’s PhD thesis is titled "Curious Machines: Active Learning with Structured Instances", which he completed in 2008 at the University of Wisconsin-Madison. We talk about his work in the thesis on active learning, then chart the path to Burr’s role at DuoLingo. We discuss machine learning for education and language learning, including content, assessment, and the exciting possibilities opened by recent advancements.
- Episode notes: https://cs.nyu.edu/~welleck/episode39.html
- Foll...
2022-02-021h 06The Thesis Review[38] Andrew Lampinen - A Computational Framework for Learning and Transforming Task RepresentationsAndrew Lampinen is a research scientist at DeepMind. His research focuses on cognitive flexibility and generalization.
Andrew’s PhD thesis is titled "A Computational Framework for Learning and Transforming Task Representations", which he completed in 2020 at Stanford University.
We talk about cognitive flexibility in brains and machines, centered around his work in the thesis on meta-mapping. We cover a lot of interesting ground, including complementary learning systems and memory, compositionality and systematicity, and the role of symbols in machine learning.
- Episode notes: https://cs.nyu.edu/~welleck/episode38.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on...
2022-01-081h 04The Thesis Review[37] Joonkoo Park - Neural Substrates of Visual Word and Number ProcessingJoonkoo Park is an Associate Professor and Honors Faculty in the Department of Psychological and Brain Sciences at UMass Amherst.
He leads the Cognitive and Developmental Neuroscience Lab, focusing on understanding the developmental mechanisms and neurocognitive underpinnings of our knowledge about number and mathematics.
Joonkoo’s PhD thesis is titled "Experiential Effects on the Neural Substrates of Visual Word and Number Processing", which he completed in 2011 at the University of Michigan.
We talk about numerical processing in the brain, starting with nature vs. nurture, including the learned versus built-in aspects of neural architectures. We talk about the difference between word an...
2021-12-211h 09The Thesis Review[36] Dieuwke Hupkes - Hierarchy and Interpretability in Neural Models of Language ProcessingDieuwke Hupkes is a Research Scientist at Facebook AI Research and the scientific manager of the Amsterdam unit of ELLIS.
Dieuwke's PhD thesis is titled, "Hierarchy and Interpretability in Neural Models of Language Processing", which she completed in 2020 at the University of Amsterdam.
We discuss her work on which aspects of hierarchical compositionality and syntactic structure can be learned by recurrent neural networks, how these models can serve as explanatory models of human language processing, what compositionality actually means, and a lot more.
- Episode notes: https://cs.nyu.edu/~welleck/episode36.html
- Follow the Thesis Review (@thesisreview) and...
2021-11-301h 02
The Thesis Review[35] Armando Solar-Lezama - Program Synthesis by SketchingArmando Solar-Lezama is a Professor at MIT, and the Associate Director & COO of CSAIL. He leads the Computer Assisted Programming Group, focused on program synthesis.
Armando’s PhD thesis is titled, "Program Synthesis by Sketching", which he completed in 2008 at UC Berkeley.
We talk about program synthesis & his work on Sketch, how machine learning's role in program synthesis has evolved over time, and more.
- Episode notes: https://cs.nyu.edu/~welleck/episode35.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about the show at https://cs.nyu.edu/~welleck/po...
2021-11-061h 15
Yannic Kilcher Videos (Audio Only)Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (Explained)#gpt3 #knowledge #symbolic
Symbolic knowledge models are usually trained on human-generated corpora that are cumbersome and expensive to create. Such corpora consist of structured triples of symbolic knowledge. This paper takes a different approach and attempts to generate such a corpus by prompting GPT-3. Results show that clever prompting, combined with targeted small critic models trained on human ratings can outperform both human-generated data, as well as the teacher model (GPT-3) itself. The results of this paper give a general recipe for automatically building corpora for various NLP tasks by extracting samples from large...
2021-10-2545 minThe Thesis Review[34] Sasha Rush - Lagrangian Relaxation for Natural Language DecodingSasha Rush is an Associate Professor at Cornell Tech and researcher at Hugging Face. His research focuses on building NLP systems that are safe, fast, and controllable.
Sasha's PhD thesis is titled, "Lagrangian Relaxation for Natural Language Decoding", which he completed in 2014 at MIT.
We talk about his work in the thesis on decoding in NLP, how it connects with today, and many interesting topics along the way such as the role of engineering in machine learning, breadth vs. depth, and more.
- Episode notes: https://cs.nyu.edu/~welleck/episode34.html
- Follow the Thesis Review (@thesisreview) and Sean...
2021-10-201h 08The Thesis Review[33] Michael R. Douglas - G/H Conformal Field TheoryMichael R. Douglas is a theoretical physicist and Professor at Stony Brook University, and Visiting Scholar at Harvard University. His research focuses on string theory, theoretical physics and its relations to mathematics.
Michael's PhD thesis is titled, "G/H Conformal Field Theory", which he completed in 1988 at Caltech.
We talk about working with Feynman, Sussman, and Hopfield during his PhD days, the superstring revolutions and string theory, and machine learning's role in the future of science and mathematics.
- Episode notes: https://cs.nyu.edu/~welleck/episode33.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
...
2021-10-011h 12The Thesis Review[32] Andre Martins - The Geometry of Constrained Structured PredictionAndre Martins is an Associate Professor at IST and VP of AI Research at Unbabel in Lisbon, Portugal. His research focuses on natural language processing and machine learning.
Andre’s PhD thesis is titled, "The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax", which he completed in 2012 at Carnegie Mellon University and IST.
We talk about his work in the thesis on structured prediction in NLP, and discuss connections between his thesis work on later work on sparsity, sparse communication, and more.
- Episode notes: https://cs.nyu.edu/~welleck/episode32.html
- Follow th...
2021-09-161h 26The Thesis Review[31] Jay McClelland - Preliminary Letter Identification in the Perception of Words and NonwordsJay McClelland is a Professor in the Psychology Department and Director of the Center for Mind, Brain, Computation and Technology at Stanford. His research addresses a broad range of topics in cognitive science and cognitive neuroscience, including Parallel Distributed Processing (PDP).
Jay's PhD thesis is titled "Preliminary Letter Identification in the Perception of Words and Nonwords", which he completed in 1975 at University of Pennsylvania.
We discuss his work in the thesis on the word superiority effect, how it led to the Integrated Activation model, the path to Parallel Distributed Processing and the connectionist revolution, and distributed vs rule-based and symbolic...
2021-08-291h 33The Thesis Review[30] Dustin Tran - Probabilistic Programming for Deep LearningDustin Tran is a research scientist at Google Brain. His research focuses on advancing science and intelligence, including areas involving probability, programs, and neural networks.
Dustin’s PhD thesis is titled "Probabilistic Programming for Deep Learning", which he completed in 2020 at Columbia University.
We discuss the intersection of probabilistic modeling and deep learning, including the Edward library and the novel inference algorithms and models that he developed in the thesis.
- Episode notes: https://cs.nyu.edu/~welleck/episode30.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
- Find out more info about the show at...
2021-08-141h 02The Thesis Review[29] Tengyu Ma - Non-convex Optimization for Machine LearningTengyu Ma is an Assistant Professor at Stanford University. His research focuses on deep learning and its theory, as well as various topics in machine learning.
Tengyu's PhD thesis is titled "Non-convex Optimization for Machine Learning: Design, Analysis, and Understanding", which he completed in 2017 at Princeton University.
We discuss theory in machine learning and deep learning, including the 'all local minima are global minima' property, overparameterization, as well as perspectives that theory takes on understanding deep learning.
- Episode notes: https://cs.nyu.edu/~welleck/episode29.html
- Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter
...
2021-08-011h 17
The Thesis Review[28] Karen Ullrich - A Coding Perspective on Deep Latent Variable ModelsKaren Ullrich is a Research Scientist at FAIR. Her research focuses on the intersection of information theory and probabilistic machine learning and deep learning.
Karen's PhD thesis is titled "A coding perspective on deep latent variable models", which she completed in 2020 at The University of Amsterdam.
We discuss information theory & the minimum description length principle, along with her work in the thesis on compression and communication.
Episode notes: https://cs.nyu.edu/~welleck/episode28.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck...
2021-07-161h 06The Thesis Review[27] Danqi Chen - Neural Reading Comprehension and BeyondDanqi Chen is an assistant professor at Princeton University, co-leading the Princeton NLP Group. Her research focuses on fundamental methods for learning representations of language and knowledge, and practical systems including question answering, information extraction and conversational agents.
Danqi’s PhD thesis is titled "Neural Reading Comprehension and Beyond", which she completed in 2018 at Stanford University. We discuss her work on parsing, reading comprehension and question answering. Throughout we discuss progress in NLP, fundamental challenges, and what the future holds.
Episode notes: https://cs.nyu.edu/~welleck/episode27.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, an...
2021-07-0255 minThe Thesis Review[26] Kevin Ellis - Algorithms for Learning to Induce ProgramsKevin Ellis is an assistant professor at Cornell and currently a research scientist at Common Sense Machines. His research focuses on artificial intelligence, program synthesis, and neurosymbolic models.
Kevin's PhD thesis is titled "Algorithms for Learning to Induce Programs", which he completed in 2020 at MIT. We discuss Kevin’s work at the intersection of machine learning and program induction, including inferring graphics programs from images and drawings, DreamCoder, and more.
Episode notes: https://cs.nyu.edu/~welleck/episode26.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs...
2021-05-291h 17The Thesis Review[25] Tomas Mikolov - Statistical Language Models Based on Neural NetworksTomas Mikolov is a Senior Researcher at the Czech Institute of Informatics, Robotics, and Cybernetics. His research has covered topics in natural language understanding and representation learning, including Word2Vec, and complexity.
Tomas's PhD thesis is titles "Statistical Language Models Based on Neural Networks", which he completed in 2012 at the Brno University of Technology. We discuss compression and recurrent language models, the backstory behind Word2Vec, and his recent work on complexity & automata.
Episode notes: https://cs.nyu.edu/~welleck/episode25.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the...
2021-05-141h 19The Thesis Review[24] Martin Arjovsky - Out of Distribution Generalization in Machine LearningMartin Arjovsky is a postdoctoral researcher at INRIA. His research focuses on generative modeling, generalization, and exploration in RL.
Martin's PhD thesis is titled "Out of Distribution Generalization in Machine Learning", which he completed in 2019 at New York University. We discuss his work on the influential Wasserstein GAN early in his PhD, then discuss his thesis work on out-of-distribution generalization which focused on causal invariance and invariant risk minimization.
Episode notes: https://cs.nyu.edu/~welleck/episode24.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs...
2021-04-301h 02The Thesis Review[23] Simon Du - Gradient Descent for Non-convex Problems in Modern Machine LearningSimon Shaolei Du is an Assistant Professor at the University of Washington. His research focuses on theoretical foundations of deep learning, representation learning, and reinforcement learning.
Simon's PhD thesis is titled "Gradient Descent for Non-convex Problems in Modern Machine Learning", which he completed in 2019 at Carnegie Mellon University. We discuss his work related to the theory of
gradient descent for challenging non-convex problems that we encounter in deep learning. We cover various topics including connections with the Neural Tangent Kernel, theory vs. practice, and future research directions.
Episode notes: https://cs.nyu.edu/~welleck/episode23.html
Follow the Thesis Review...
2021-04-161h 06The Thesis Review[22] Graham Neubig - Unsupervised Learning of Lexical InformationGraham Neubig is an Associate Professor at Carnegie Mellon University. His research focuses on language and its role in human communication, with the goal of breaking down barriers in human-human or human-machine communication through the development of NLP technologies.
Graham’s PhD thesis is titled "Unsupervised Learning of Lexical Information for Language Processing Systems", which he completed in 2012 at Kyoto University. We discuss his PhD work related to the fundamental processing units that NLP systems use to process text, including non-parametric Bayesian models, segmentation, and alignment problems, and discuss how his perspective on machine translation has evolved over time.
Episode no...
2021-04-021h 02The Thesis Review[21] Michela Paganini - Machine Learning Solutions for High Energy PhysicsMichela Paganini is a Research Scientist at DeepMind. Her research focuses on investigating ways to compress and scale up neural networks.
Michela's PhD thesis is titled "Machine Learning Solutions for High Energy Physics", which she completed in 2019 at Yale University. We discuss her PhD work on deep learning for high energy physics, including jet tagging and fast simulation for the ATLAS experiment at the Large Hadron Collider, and the intersection of machine learning and physics.
Episode notes: https://cs.nyu.edu/~welleck/episode21.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info...
2021-03-191h 07The Thesis Review[20] Josef Urban - Deductive and Inductive Reasoning in Large Libraries of Formalized MathematicsJosef Urban is a Principal Researcher at the Czech Institute of Informatics, Robotics, and Cybernetics. His research focuses on artificial intelligence for large-scale computer-assisted reasoning.
Josef's PhD thesis is titled "Exploring and Combining Deductive and Inductive Reasoning in Large Libraries of Formalized Mathematics", which he completed in 2004 at Charles University in Prague. We discuss his PhD work on the Mizar Problems for Theorem Proving, machine learning for premise selection, and how it evolved into his recent research.
Episode notes: https://cs.nyu.edu/~welleck/episode20.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out...
2021-03-051h 25The Thesis Review[19] Dumitru Erhan - Understanding Deep Architectures and the Effect of Unsupervised PretrainingDumitru Erhan is a Research Scientist at Google Brain. His research focuses on understanding the world with neural networks.
Dumitru's PhD thesis is titled "Understanding Deep Architectures and the Effect of Unsupervised Pretraining", which he completed in 2010 at the University of Montreal. We discuss his work in the thesis on understanding deep networks and unsupervised pretraining, his perspective on deep learning's development, and the path of ideas to his recent research.
Episode notes: https://cs.nyu.edu/~welleck/episode19.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at...
2021-02-191h 20The Thesis Review[18] Eero Simoncelli - Distributed Representation and Analysis of Visual MotionEero Simoncelli is a Professor of Neural Science, Mathematics, Data Science, and Psychology at New York University. His research focuses on representation and analysis of visual information.
Eero's PhD thesis is titled "Distributed Representation & Analysis of Visual Motion", which he completed in 1993 at MIT. We discuss his PhD work which focused on optical flow, which ideas and methods have stayed with him throughout his career, making biological connections with machine learning models, and how Eero's perspective of vision has evolved.
Episode notes: https://cs.nyu.edu/~welleck/episode18.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter...
2021-02-051h 25The Thesis Review[17] Paul Middlebrooks - Neuronal Correlates of Meta-Cognition in Primate Frontal CortexPaul Middlebrooks is a neuroscientist and host of the Brain Inspired podcast, which explores the intersection of neuroscience and artificial intelligence.
Paul's PhD thesis is titled "Neuronal Correlates of Meta-Cognition in Primate Frontal Cortex", which he completed at the University of Pittsburgh in 2011. We discuss Paul's work on meta-cognition - informally, thinking about thinking - then discuss neuroscience for A.I. and A.I. for neuroscience.
Episode notes: https://cs.nyu.edu/~welleck/episode17.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck...
2021-01-221h 36The Thesis Review[16] Aaron Courville - A Latent Cause Theory of Classical ConditioningAaron Courville is a Professor at the University of Montreal. His research focuses on the development of deep learning models and methods.
Aaron's PhD thesis is titled "A Latent Cause Theory of Classical Conditioning", which he completed at Carnegie Mellon University in 2006. We discuss Aaron's work on the latent cause theory during his PhD, talk about how Aaron moved into machine learning and deep learning research, chart a path to today's deep learning methods, and discuss his recent work on systematic generalization in language.
Episode notes: cs.nyu.edu/~welleck/episode16.html
Follow the Thesis Review (@thesisreview) and Sean Welleck...
2021-01-081h 19The Thesis Review[15] Christian Szegedy - Some Applications of the Weighted Combinatorial LaplacianChristian Szegedy is a Research Scientist at Google. His research machine learning methods such as the inception architecture, batch normalization and adversarial examples, and he currently investigates machine learning for mathematical reasoning.
Christian’s PhD thesis is titled "Some Applications of the Weighted Combinatorial Laplacian" which he completed in 2005 at the University of Bonn. We discuss Christian’s background in mathematics, his PhD work on areas of both pure and applied mathematics, and his path into machine learning research. Finally, we discuss his recent work with using deep learning for mathematical reasoning and automatically formalizing mathematics.
Episode notes: https://cs.nyu...
2020-12-221h 06The Thesis Review[14] Been Kim - Interactive and Interpretable Machine Learning ModelsBeen Kim is a Research Scientist at Google Brain. Her research focuses on designing high-performance machine learning methods that make sense to humans.
Been's PhD thesis is titled "Interactive and Interpretable Machine Learning Models for Human Machine Collaboration", which she completed in 2015 at MIT. We discuss her work on interpretability, including her work in the thesis on the Bayesian Case Model and its interactive version, as well as connections with her subsequent work on black-box interpretability methods that are used in many real-world applications.
Episode notes: https://cs.nyu.edu/~welleck/episode14.html
Follow the Thesis Review (@thesisreview) and Sean...
2020-12-101h 04The Thesis Review[13] Adji Bousso Dieng - Deep Probabilistic Graphical ModelingAdji Bousso Dieng is currently a Research Scientist at Google AI, and will be starting as an assistant professor at Princeton University in 2021. Her research focuses on combining probabilistic graphical modeling and deep learning to design models for structured high-dimensional data.
Her PhD thesis is titled "Deep Probabilistic Graphical Modeling", which she completed in 2020 at Columbia University. We discuss her work on combining graphical models and deep learning, including models and algorithms, the value of interpretability and probabilistic models, as well as applications and making an impact through research.
Episode notes: https://cs.nyu.edu/~welleck/episode13.html
Follow the...
2020-11-261h 07The Thesis Review[12] Martha White - Regularized Factor ModelsMartha White is an Associate Professor at the University of Alberta. Her research focuses on developing reinforcement learning and representation learning techniques for adaptive, autonomous agents learning on streams of data.
Her PhD thesis is titled "Regularized Factor Models", which she completed in 2014 at the University of Alberta. We discuss the regularized factor model framework, which unifies many machine learning methods and led to new algorithms and applications. We talk about sparsity and how it also appears in her later work, as well as the common threads between her thesis work and her research in reinforcement learning.
Episode notes: https...
2020-11-131h 08The Thesis Review[11] Jacob Andreas - Learning from LanguageJacob Andreas is an Assistant Professor at MIT, where he leads the language and intelligence group, focusing on language as a communicative and computational tool.
His PhD thesis is titled "Learning from Language" which he completed in 2018 at UC Berkeley. We discuss compositionality and neural module networks, the intersection of RL and language, and translating a neural communication channel called 'neuralese', and how this can lead to more interpretable machine learning models.
Episode notes: https://cs.nyu.edu/~welleck/episode11.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show...
2020-10-301h 19The Thesis Review[10] Chelsea Finn - Learning to Learn with GradientsChelsea Finn is an Assistant Professor at Stanford University, where she leads the IRIS lab that studies intelligence through robotic interaction at scale.
Her PhD thesis is titled "Learning to Learn with Gradients", which she completed in 2018 at UC Berkeley. Chelsea received the prestigious ACM Doctoral Dissertation Award for her work in the thesis. We discuss machine learning for robotics, focusing on learning-to-learn - also known as meta-learning - and her work on the MAML algorithm during her PhD, as well as the future of robotics research.
Episode notes: https://cs.nyu.edu/~welleck/episode10.html
Follow the Thesis Review...
2020-10-1651 minThe Thesis Review[09] Kenneth Stanley - Efficient Evolution of Neural Networks through ComplexificationKenneth Stanley is a researcher at OpenAI, where he leads the team on Open-endedness. Previously he was a Professor Computer Science at the University of Central Florida, cofounder of Geometric Intelligence, and head of Core AI research at Uber AI labs.
His PhD thesis is titled "Efficient Evolution of Neural Networks through Complexification", which he completed on 2004 at the University of Texas. We talk about evolving increasingly complex structures and how this led to the NEAT algorithm that he developed during his PhD. We discuss his research directions related to open-endedness, how the field has changed over time, and how...
2020-10-021h 21The Thesis Review[08] He He - Sequential Decisions and Predictions in NLPHe He is an Assistant Professor at New York University. Her research focuses on enabling reliable communication in natural language between machine and humans, including topics in text generation, robust language understanding, and dialogue systems.
Her PhD thesis is titled "Sequential Decisions and Predictions in NLP", which she completed in 2016 at the University of Maryland. We talk about the intersection of language with imitation learning and reinforcement learning, her work in the thesis on opponent modeling and simultaneous translation, and how it relates to recent work on generation and robustness.
Episode notes: https://cs.nyu.edu/~welleck/episode8.html
Follow...
2020-09-251h 00The Thesis Review[07] John Schulman - Optimizing Expectations: From Deep RL to Stochastic Computation GraphsJohn Schulman is a Research Scientist and co-founder of Open AI. John co-leads the reinforcement learning team, researching algorithms that safely and efficiently learn by trial and error and by imitating humans.
His PhD thesis is titled "Optimizing Expectations: From Deep Reinforcement Learning to Stochastic Computation Graphs", which he completed in 2016 at Berkeley. We talk about his work on stochastic computation graphs and TRPO, how it evolved to PPO and how it's used in large-scale applications like Open AI Five, as well as his recent work on generalization in RL.
Episode notes: https://cs.nyu.edu/~welleck/episode7.html
Follow...
2020-09-111h 04The Thesis Review[06] Yoon Kim - Deep Latent Variable Models of Natural LanguageYoon Kim is currently a Research Scientist at the MIT-IBM AI Watson Lab, and will be joining MIT as an assistant professor in 2021.
Yoon’s research focuses on machine learning and natural language processing.
His PhD thesis is titled "Deep Latent Variable Models of Natural Language", which he completed in 2020 at Harvard University. We discuss his work on uncovering latent structure in natural language, including continuous vector representations, tree structures, and grammars. We cover learning and variational inference methods that he developed during his PhD, and he offers a look at where latent variable models will be heading in the fu...
2020-08-281h 05The Thesis Review[05] Julian Togelius - Computational Intelligence and GamesJulian Togelius is an Associate Professor at New York University, where he co-directs the NYU Game Innovation Lab. His research is at the intersection of computational intelligence and computer games.
His PhD thesis is titled "Optimization, Imitation, and Innovation: Computational Intelligence and Games", which he completed in 2007. We cover his work in the thesis on AI for games and games for AI, and how it connects to his recent work on procedural content generation.
Episode notes: https://cs.nyu.edu/~welleck/episode5.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about...
2020-08-141h 12The Thesis Review[04] Sebastian Nowozin - Learning with Structured Data: Applications to Computer VisionSebastian Nowozin is currently a Researcher at Microsoft Research Cambridge. His research focuses on probabilistic deep learning, consequences of model misspecification, understanding agent complexity in order to improve learning efficiency, and designing models for reasoning and planning.
His PhD thesis is titled "Learning with Structured Data: Applications to Computer Vision", which he completed in 2009. We discuss the work in his thesis on structured inputs and structured outputs, which involves beautiful ideas from polyhedral combinatorics and optimization. We talk about his recent work on Bayesian deep learning and the connections it has to ideas that he explored during his PhD.
Episode...
2020-07-311h 44The Thesis Review[03] Sebastian Ruder - Neural Transfer Learning for Natural Language ProcessingSebastian Ruder is currently a Research Scientist at Deepmind. His research focuses on transfer learning for natural language processing, and making machine learning and NLP more accessible.
His PhD thesis is titled "Neural Transfer Learning for Natural Language Processing", which he completed in 2019. We cover transfer learning from philosophical and technical perspectives, and talk about its societal
implications, focusing on his work on sequential transfer learning and cross-lingual learning.
Episode notes: https://cs.nyu.edu/~welleck/episode3.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs...
2020-07-171h 24The Thesis Review[02] Colin Raffel - Learning-Based Methods for Comparing SequencesColin Raffel is currently a Senior Research Scientist at Google Brain, and soon to be an assistant professor at the University of North Carolina. His recent work focuses on transfer learning and learning from limited labels.
His thesis is titled "Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching", which we discuss along with the connections to his later work, and plans for the future.
Episode notes: https://cs.nyu.edu/~welleck/episode2.html
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu...
2020-07-031h 15The Thesis Review[01] Gus Xia - Expressive Collaborative Music Performance via Machine LearningGus Xia is an assistant professor at New York University Shanghai. His research explores machine learning for music, with a goal of building intelligent systems that understand and extend musical creativity and expression.
His PhD thesis is titled Expressive Collaborative Music Performance via Machine Learning, which we discuss in depth along with his ongoing research at the NYU Shanghai Music X Lab.
- Gus Xia's homepage: https://www.cs.cmu.edu/~gxia/
- Thesis: http://reports-archive.adm.cs.cmu.edu/anon/ml2016/CMU-ML-16-103.pdf
Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more...
2020-06-1957 minThe Thesis Review[00] The Thesis Review Podcast - Introduction[00] The Thesis Review Podcast - Introduction by Sean Welleck
2020-06-1401 min