Shows
muckrAIkersAI, Reasoning or Rambling?In this episode, we redefine AI's "reasoning" as mere rambling, exposing the "illusion of thinking" and "Potemkin understanding" in current models. We contrast the classical definition of reasoning (requiring logic and consistency) with Big Tech's new version, which is a generic statement about information processing. We explain how Large Rambling Models generate extensive, often irrelevant, rambling traces that appear to improve benchmarks, largely due to best-of-N sampling and benchmark gaming.Words and definitions actually matter! Carelessness leads to misplaced investments and an overestimation of systems that are currently just surprisingly useful autocorrects.(00:00) - Intro
(00:40...2025-07-141h 11
Into AI SafetyGetting Into PauseAI w/ Will PetilloWill Petillo, onboarding team lead at PauseAI, joins me to discuss the grassroots movement advocating for a pause on frontier AI model development. We explore PauseAI's strategy, talk about common misconceptions Will hears, and dig into how diverse perspectives still converge on the need to slow down AI development.Will's LinksPersonal blog on AIHis mindmap of the AI x-risk debateGame demosAI focused YouTube channel(00:00) - Intro
(03:36) - What is PauseAI
(10:10) - Will Petillo's journey into AI safety advocacy
(21:13) - Understanding PauseAI
(31:35) - Pursuing a pause
(40:06) - Balancing advocacy in a complex world
(45:54...2025-06-231h 48muckrAIkersOne Big Bad BillIn this episode, we break down Trump's "One Big Beautiful Bill" and its dystopian AI provisions: automated fraud detection systems, centralized citizen databases, military AI integration, and a 10-year moratorium blocking all state AI regulation. We explore the historical parallels with authoritarian data consolidation and why this represents a fundamental shift away from limited government principles once held by US conservatives.(00:00) - Intro
(01:13) - Bill, general overview
(05:14) - Bill, AI overview
(07:54) - Medicaid fraud detection systems
(11:20) - Bias in AI Systems and Ethical Concerns
(17:58) - Centralization of data
(30:04) - Military integration of AI
(37:05) - Tax incentives...2025-06-2353 minmuckrAIkersBreaking Down the Economics of AIJacob and Igor tackle the wild claims about AI's economic impact by examining three main clusters of arguments: automating expensive tasks like programming, removing "cost centers" like call centers and corporate art, and claims of explosive growth. They dig into the actual data, debunk the hype, and explain why most productivity claims don't hold up in practice. Plus: MIT denounces a paper with fabricated data, and Grok randomly promotes white genocide myths.(00:00) - Recording date + intro
(00:52) - MIT denounces paper
(04:09) - Grok's white genocide
(06:23) - Butthole convergence
(07:13) - AI and the economy
(14:50) - Automating profit...2025-05-261h 06Into AI SafetyMaking Your Voice Heard w/ Tristan Williams & Felix de SimoneI am joined by Tristan Williams and Felix de Simone to discuss their work on the potential of constituent communication, specifically in the context of AI legislation. These two worked as part of an AI Safety Camp team to understand whether or not it would be useful for more people to be sharing their experiences, concerns, and opinions with their government representative (hint, it is).Check out the blogpost on their findings, "Talking to Congress: Can constituents contacting their legislator influence policy?" and the tool they created!(01:53) - Introductions
(04:04) - Starting the project
(13:30...2025-05-191h 33muckrAIkersDeepSeek: 2 Months OutDeepSeek has been out for over 2 months now, and things have begun to settle down. We take this opportunity to contextualize the developments that have occurred in its wake, both within the AI industry and the world economy. As systems get more "agentic" and users are willing to spend increasing amounts of time waiting for their outputs, the value of supposed "reasoning" models continues to be peddled by AI system developers, but does the data really back these claims?Check out our DeepSeek minisode for a snappier overview!EPISODE RECORDED 2025.03.30(00:40...2025-04-091h 31muckrAIkersDeepSeek MinisodeDeepSeek R1 has taken the world by storm, causing a stock market crash and prompting further calls for export controls within the US. Since this story is still very much in development, with follow-up investigations and calls for governance being released almost daily, we thought it best to hold of for a little while longer to be able to tell the whole story. Nonetheless, it's a big story, so we provide a brief overview of all that's out there so far.(00:00) - Recording date
(00:04) - Intro
(00:37) - DeepSeek drop and reactions
(04:27) - Export controls
(08:05) - Skepticism...2025-02-1015 minmuckrAIkersUnderstanding AI World Models w/ Chris CanalChris Canal, co-founder of EquiStamp, joins muckrAIkers as our first ever podcast guest! In this ~3.5 hour interview, we discuss intelligence vs. competencies, the importance of test-time compute, moving goalposts, the orthogonality thesis, and much more.A seasoned software developer, Chris started EquiStamp as a way to improve our current understanding of model failure modes and capabilities in late 2023. Now a key contractor for METR, EquiStamp evaluates the next generation of LLMs from frontier model developers like OpenAI and Anthropic.EquiStamp is hiring, so if you're a software developer interested in a fully remote opportunity with...2025-01-273h 19muckrAIkersNeurIPS 2024 Wrapped đŻWhat happens when you bring over 15,000 machine learning nerds to one city? If your guess didn't include racism, sabotage and scandal, belated epiphanies, a spicy SoLaR panel, and many fantastic research papers, you wouldn't have captured my experience. In this episode we discuss the drama and takeaways from NeurIPS 2024.Posters available at time of episode preparation can be found on the episode webpage.EPISODE RECORDED 2024.12.22(00:00) - Recording date
(00:05) - Intro
(00:44) - Obligatory mentions
(01:54) - SoLaR panel
(18:43) - Test of Time
(24:17) - And now: science!
(28:53) - Downsides of benchmarks
(41:39) - Improving the...2024-12-301h 26muckrAIkersOpenAI's o1 System Card, Literally Migraine InducingThe idea of model cards, which was introduced as a measure to increase transparency and understanding of LLMs, has been perverted into the marketing gimmick characterized by OpenAI's o1 system card. To demonstrate the adversarial stance we believe is necessary to draw meaning from these press-releases-in-disguise, we conduct a close read of the system card. Be warned, there's a lot of muck in this one.Note: All figures/tables discussed in the podcast can be found on the podcast website at https://kairos.fm/muckraikers/e009/(00:00) - Recorded 2024.12.08
(00:54) - Actual intro
(03:00) - System...2024-12-231h 16muckrAIkersHow to Safely Handle Your AGIWhile on the campaign trail, Trump made claims about repealing Biden's Executive Order on AI, but what will actually be changed when he gets into office? We take this opportunity to examine policies being discussed or implemented by leading governments around the world.(00:00) - Intro
(00:29) - Hot off the press
(02:59) - Repealing the AI executive order?
(11:16) - "Manhattan" for AI
(24:33) - EU
(30:47) - UK
(39:27) - Bengio
(44:39) - Comparing EU/UK to USA
(45:23) - China
(51:12) - Taxes
(55:29) - The muck
LinksSFChronicle article - US gathers allies to talk AI safety as Trump's vow to...2024-12-0258 minmuckrAIkersThe End of Scaling?Multiple news outlets, including The Information, Bloomberg, and Reuters [see sources] are reporting an "end of scaling" for the current AI paradigm. In this episode we look into these articles, as well as a wide variety of economic forecasting, empirical analysis, and technical papers to understand the validity, and impact of these reports. We also use this as an opportunity to contextualize the realized versus promised fruits of "AI".(00:23) - Hot off the press
(01:49) - The end of scaling
(10:50) - "Useful tools" and "agentic" "AI"
(17:19) - The end of quantization
(25:18) - Hedging
(29:41) - The end...2024-11-191h 07muckrAIkersUS National Security Memorandum on AI, Oct 2024October 2024 saw a National Security Memorandum and US framework for using AI in national security contexts. We go through the content so you don't have to, pull out the important bits, and summarize our main takeaways.(00:48) - The memorandum
(06:28) - What the press is saying
(10:39) - What's in the text
(13:48) - Potential harms
(17:32) - Miscellaneous notable stuff
(31:11) - What's the US governments take on AI?
(45:45) - The civil side - comments on reporting
(49:31) - The commenters
(01:07:33) - Our final hero
(01:10:46) - The muck
LinksUnited States National Security Memorandum on AIFact Sheet on the National...2024-11-061h 16muckrAIkersUnderstanding Claude 3.5 Sonnet (New)Frontier developers continue their war on sane versioning schema to bring us Claude 3.5 Sonnet (New), along with "computer use" capabilities. We discuss not only the new model, but also why Anthropic may have released this model and tool combination now.(00:00) - Intro
(00:22) - Hot off the press
(05:03) - Claude 3.5 Sonnet (New) Two 'o' 3000
(09:23) - Breaking down "computer use"
(13:16) - Our understanding
(16:03) - Diverging business models
(32:07) - Why has Anthropic chosen this strategy?
(43:14) - Changing the frame
(48:00) - Polishing the lily
LinksAnthropic press release - Introducing Claude 3.5 Sonnet (New)Model Card Addendum...2024-10-301h 00muckrAIkersWinter is Coming for OpenAIBrace yourselves, winter is coming for OpenAI - atleast, that's what we think. In this episode we look at OpenAI's recent massive funding round and ask "why would anyone want to fund a company that is set to lose net 5 billion USD for 2024?" We scrape through a whole lot of muck to find the meaningful signals in all this news, and there is a lot of it, so get ready!(00:00) - Intro
(00:28) - Hot off the press
(02:43) - Why listen?
(06:07) - Why might VCs invest?
(15:52) - What are people saying
(23:10) - How *is* OpenAI making...2024-10-221h 22muckrAIkersOpen Source AI and 2024 Nobel PrizesThe Open Source AI Definition is out after years of drafting, will it reestablish brand meaning for the âOpen Sourceâ term? Also, the 2024 Nobel Prizes in Physics and Chemistry are heavily tied to AI; we scrutinize not only this year's prizes, but also Nobel Prizes as a concept. (00:00) - Intro
(00:30) - Hot off the press
(03:45) - Open Source AI background
(10:30) - Definitions and changes in RC1
(18:36) - âBusiness sourceâ
(22:17) - Parallels with legislation
(26:22) - Impacts of the OSAID
(33:58) - 2024 Nobel Prize Context
(37:21) - Chemistry prize
(45:06) - Physics prize
(50:29) - Takeaways
(52:03) - Whatâs the real muck?
(01:00:27) - Outro
2024-10-161h 01muckrAIkersSB1047Why is Mark Ruffalo talking about SB1047, and what is it anyway? Tune in for our thoughts on the now vetoed California legislation that had Big Tech scared.(00:00) - Intro
(00:31) - Updates from a relatively slow week
(03:32) - Disclaimer: SB1047 vetoed during recording (still worth a listen)
(05:24) - What is SB1047
(12:30) - Definitions
(17:18) - Understanding the bill
(28:42) - What are the players saying about it?
(46:44) - Addressing critiques
(55:59) - Open Source
(01:02:36) - Takeaways
(01:15:40) - Clarification on impact to big tech
(01:18:51) - Outro
LinksSB1047 legislation pageSB1047 CalMatters pageNewsom vetoes SB1047CAIS newsletter on SB1047Prominent AI...2024-09-301h 19muckrAIkersOpenAI's o1, aka. StrawberryOpenAI's new model is out, and we are going to have to rake through a lot of muck to get the value out of this one!â Opt out of LinkedIn's GenAI scraping âĄď¸ https://lnkd.in/epziUeTi (00:00) - Intro
(00:25) - Other recent news
(02:57) - Hot off the press
(03:58) - Why might someone care?
(04:52) - What is it?
(06:49) - How is it being sold?
(10:45) - How do they explain it, technically?
(27:09) - Reflection AI Drama
(40:19) - Why do we care?
(46:39) - Scraping away the muck
Note: at around 32 minutes, Igor says the incorrect Llama model versio...2024-09-2350 min
The Third AngleCorPower Ocean: Making Waves In Renewable EnergyâWeâre trying to be the most environmentally friendly option for power production.âIn this episode we're catching the wave - literally - with CorPower Ocean, the trailblazers of wave power technology. Join us as we explore how this pioneering company is leading the way with a carbon-free solution that balances supply and demand more efficiently, requiring less generation capacity, storage, and grid infrastructure. Engineers, Jacob Ljungbäck and Antoine Bonel take us into their world of wave energy and tell us all about this forward thinking companyâs mission.Their innovative approach not only reduces th...2024-06-2518 minInto AI SafetyINTERVIEW: Scaling Democracy w/ (Dr.) Igor KrawczukThe almost Dr. Igor Krawczuk joins me for what is the equivalent of 4 of my previous episodes. We get into all the classics: eugenics, capitalism, philosophical toads... Need I say more?If you're interested in connecting with Igor, head on over to his website, or check out placeholder for thesis (it isn't published yet).Because the full show notes have a whopping 115 additional links, I'll highlight some that I think are particularly worthwhile here:The best article you'll ever read on Open Source AIThe best article you'll ever read on emergence in MLKate Crawford's...2024-06-032h 58Into AI SafetyINTERVIEW: StakeOut.AI w/ Dr. Peter Park (3)As always, the best things come in 3s: dimensions, musketeers, pyramids, and... 3 installments of my interview with Dr. Peter Park, an AI Existential Safety Post-doctoral Fellow working with Dr. Max Tegmark at MIT.As you may have ascertained from the previous two segments of the interview, Dr. Park cofounded StakeOut.AI along with Harry Luk and one other cofounder whose name has been removed due to requirements of her current position. The non-profit had a simple but important mission: make the adoption of AI technology go well, for humanity, but unfortunately, StakeOut.AI had to dissolve in...2024-03-251h 42Into AI SafetyINTERVIEW: StakeOut.AI w/ Dr. Peter Park (2)Join me for round 2 with Dr. Peter Park, an AI Existential Safety Postdoctoral Fellow working with Dr. Max Tegmark at MIT. Dr. Park was a cofounder of StakeOut.AI, a non-profit focused on making AI go well for humans, along with Harry Luk and one other individual, whose name has been removed due to requirements of her current position.In addition to the normal links, I wanted to include the links to the petitions that Dr. Park mentions during the podcast. Note that the nonprofit which began these petitions, StakeOut.AI, has been dissolved.Right AI...2024-03-181h 06Into AI SafetyMINISODE: Restructure Vol. 2UPDATE: Contrary to what I say in this episode, I won't be removing any episodes that are already published from the podcast RSS feed.After getting some advice and reflecting more on my own personal goals, I have decided to shift the direction of the podcast towards accessible content regarding "AI" instead of the show's original focus. I will still be releasing what I am calling research ride-along content to my Patreon, but the show's feed will consist only of content that I aim to make as accessible as possible.00:35 - TL;DL01:12...2024-03-1113 minInto AI SafetyINTERVIEW: StakeOut.AI w/ Dr. Peter Park (1)Dr. Peter Park is an AI Existential Safety Postdoctoral Fellow working with Dr. Max Tegmark at MIT. In conjunction with Harry Luk and one other cofounder, he founded â StakeOut.AI, a non-profit focused on making AI go well for humans.00:54 - Intro03:15 - Dr. Park, x-risk, and AGI08:55 - StakeOut.AI12:05 - Governance scorecard19:34 - Hollywood webinar22:02 - Regulations.gov comments23:48 - Open letters 26:15 - EU AI Act35:07 - Effective accelerationism40:50 - Divide and conquer dynamics45:40 - AI "art"53:09 - OutroLinks to all art...2024-03-0454 minInto AI SafetyMINISODE: "LLMs, a Survey"Take a trip with me through the paper Large Language Models, A Survey, published on February 9th of 2024. All figures and tables mentioned throughout the episode can be found on the Into AI Safety podcast website.00:36 - Intro and authors01:50 - My takes and paper structure04:40 - Getting to LLMs07:27 - Defining LLMs & emergence12:12 - Overview of PLMs15:00 - How LLMs are built18:52 - Limitations if LLMs23:06 - Uses of LLMs25:16 - Evaluations and Benchmarks28:11 - Challenges and future directions29:21 - Recap & outroLinks to...2024-02-2630 minInto AI SafetyFEEDBACK: Applying for Funding w/ Esben KranEsben reviews an application that I would soon submit for Open Philanthropy's Career Transitition Funding opportunity. Although I didn't end up receiving the funding, I do think that this episode can be a valuable resource for both others and myself when applying for funding in the future.Head over to Apart Research's website to check out their work, or the Alignment Jam website for information on upcoming hackathons.A doc-capsule of the application at the time of this recording can be found at this link.01:38 - Interview starts05:41 - Proposal11:00...2024-02-1945 minInto AI SafetyMINISODE: Reading a Research PaperBefore I begin with the paper-distillation based minisodes, I figured we would go over best practices for reading research papers. I go through the anatomy of typical papers, and some generally applicable advice.00:56 - Anatomy of a paper02:38 - Most common advice05:24 - Reading sparsity and path07:30 - Notes and motivationLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Ten simple rules for reading a scientific paperBest sources I foundLet's get critical: Reading academic articles#GradHacks: A guide to...2024-02-1209 minInto AI SafetyHACKATHON: Evals November 2023 (2)Join our hackathon group for the second episode in the Evals November 2023 Hackathon subseries. In this episode, we solidify our goals for the hackathon after some preliminary experimentation and ideation.Check out Stellaric's website, or follow them on Twitter.01:53 - Meeting starts05:05 - Pitch: extension of locked models23:23 - Pitch: retroactive holdout datasets34:04 - Preliminary results37:44 - Next steps42:55 - RecapLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Evalugator libraryPassword Locked Model blogpostTruthfulQA: Measuring...2024-02-0548 minInto AI SafetyMINISODE: PortfoliosI provide my thoughts and recommendations regarding personal professional portfolios.00:35 - Intro to portfolios01:42 - Modern portfolios02:27 - What to include04:38 - Importance of visual05:50 - The "About" page06:25 - Tools08:12 - Future of "Minisodes"Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.From Portafoglio to Eportfolio: The Evolution of Portfolio in Higher EducationGIMPAlternativeToJekyllGitHub PagesMinimal MistakesMy portfolio2024-01-2909 minInto AI SafetyINTERVIEW: Polysemanticity w/ Dr. Darryl WrightDarryl and I discuss his background, how he became interested in machine learning, and a project we are currently working on investigating the penalization of polysemanticity during the training of neural networks.Check out a diagram of the decoder task used for our research!01:46 - Interview begins02:14 - Supernovae classification08:58 - Penalizing polysemanticity20:58 - Our "toy model"30:06 - Task description32:47 - Addressing hurdles39:20 - Lessons learnedLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance....2024-01-2245 minInto AI SafetyMINISODE: Starting a PodcastA summary and reflections on the path I have taken to get this podcast started, including some resources recommendations for others who want to do something similar.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.LessWrongSpotify for PodcastersInto AI Safety podcast websiteEffective Altruism GlobalOpen Broadcaster Software (OBS)CraigRiverside2024-01-1510 minInto AI SafetyHACKATHON: Evals November 2023 (1)This episode kicks off our first subseries, which will consist of recordings taken during my team's meetings for the AlignmentJams Evals Hackathon in November of 2023. Our team won first place, so you'll be listening to the process which, at the end of the day, turned out to be pretty good.Check out Apart Research, the group that runs the AlignmentJamz Hackathons.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Generalization Analogies: A Testbed for Generalizing AI...2024-01-081h 08Into AI SafetyMINISODE: Staying Up-to-Date in AIIn this minisode I give some tips for staying up-to-date in the everchanging landscape of AI. I would like to point out that I am constantly iterating on these strategies, tools, and sources, so it is likely that I will make an update episode in the future.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.ToolsFeedlyarXiv Sanity LiteZoteroAlternativeToMy "Distilled AI" FolderAI Explained YouTube channelAI Safety newsletterData Machina newsletterImport AIMidwit AlignmentHonourable MentionsAI Alignment ForumLessWrongBounded Regret (Jacob Steinhart's blog)Cold Takes (Holden Karnofsky's...2024-01-0113 minInto AI SafetyINTERVIEW: Applications w/ Alice RiggAlice Rigg, a mechanistic interpretability researcher from Ottawa, Canada, joins me to discuss their path and the applications process for research/mentorship programs.Join the Mech Interp Discord server and attend reading groups at 11:00am on Wednesdays (Mountain Time)!Check out Alice's website.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. EleutherAI Join the public EleutherAI discord server DistillEffective Altruism (EA)MATS Retrospective Summer 2023 postAmbitious Mechanistic Interpretability AISC research plan by Alice RiggSPARStability AI During their most recent fund...2023-12-181h 10Into AI SafetyMINISODE: Program Applications (Winter 2024)We're back after a month-long hiatus with a podcast refactor and advice on the applications process for research/mentorship programs.Check out the About page on the Into AI Safety website for a summary of the logistics updates.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. MATSASTRA FellowshipARENAAI Safety CampBlueDot ImpactTech with TimFast.AI's Practical Deep Learning for CodersKaggleAlignmentJamsLessWrongAI Alignment Forum2023-12-1118 minInto AI SafetyMINISODE: EAG Takeaways (Boston 2023)This episode is a brief overview of the major takeaways I had from attending EAG Boston 2023, and an update on my plans for the podcast moving forward.TL;DLStarting in early December (2023), I will be uploading episodes on a biweekly basis (day TBD).I won't be releasing another episode until then, so that I can build a cache of episodes up.During this month (November 2023), I'll also try to get the podcast up on more platforms, set up comments on more platforms, and create an anonymous feedback form.Links Links to all articles/papers...2023-12-0409 minInto AI SafetyFEEDBACK: AISC Proposal w/ Remmelt EllenIn this episode I discuss my initial research proposal for the 2024 Winter AI Safety Camp with one of the individuals who helps facilitate the program, Remmelt Ellen.The proposal is titled The Effect of Machine Learning on Bioengineered Pandemic Risk. A doc-capsule of the proposal at the time of this recording can be found at this link.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. MegaSyn: Integrating Generative Molecule Design, Automated Analog Designer and Synthetic Viability PredictionDual use of artificial-intelligence-powered drug discoveryArtificial i...2023-11-2756 minInto AI SafetyMINISODE: Introduction and MotivationWelcome to the Into AI Safety podcast! In this episode I provide reasoning for why I am starting this podcast, what I am trying to accomplish with it, and a little bit of background on how I got here.Please email all inquiries and suggestions to intoaisafety@gmail.com.2023-11-1309 min