podcast
details
.com
Print
Share
Look for any podcast host, guest or anyone
Search
Showing episodes and shows of
Jacob Haimes
Shows
muckrAIkers
AI, Reasoning or Rambling?
In this episode, we redefine AI's "reasoning" as mere rambling, exposing the "illusion of thinking" and "Potemkin understanding" in current models. We contrast the classical definition of reasoning (requiring logic and consistency) with Big Tech's new version, which is a generic statement about information processing. We explain how Large Rambling Models generate extensive, often irrelevant, rambling traces that appear to improve benchmarks, largely due to best-of-N sampling and benchmark gaming.Words and definitions actually matter! Carelessness leads to misplaced investments and an overestimation of systems that are currently just surprisingly useful autocorrects.(00:00) - Intro (00:40...
2025-07-14
1h 11
Into AI Safety
Getting Into PauseAI w/ Will Petillo
Will Petillo, onboarding team lead at PauseAI, joins me to discuss the grassroots movement advocating for a pause on frontier AI model development. We explore PauseAI's strategy, talk about common misconceptions Will hears, and dig into how diverse perspectives still converge on the need to slow down AI development.Will's LinksPersonal blog on AIHis mindmap of the AI x-risk debateGame demosAI focused YouTube channel(00:00) - Intro (03:36) - What is PauseAI (10:10) - Will Petillo's journey into AI safety advocacy (21:13) - Understanding PauseAI (31:35) - Pursuing a pause (40:06) - Balancing advocacy in a complex world (45:54...
2025-06-23
1h 48
muckrAIkers
One Big Bad Bill
In this episode, we break down Trump's "One Big Beautiful Bill" and its dystopian AI provisions: automated fraud detection systems, centralized citizen databases, military AI integration, and a 10-year moratorium blocking all state AI regulation. We explore the historical parallels with authoritarian data consolidation and why this represents a fundamental shift away from limited government principles once held by US conservatives.(00:00) - Intro (01:13) - Bill, general overview (05:14) - Bill, AI overview (07:54) - Medicaid fraud detection systems (11:20) - Bias in AI Systems and Ethical Concerns (17:58) - Centralization of data (30:04) - Military integration of AI (37:05) - Tax incentives...
2025-06-23
53 min
muckrAIkers
Breaking Down the Economics of AI
Jacob and Igor tackle the wild claims about AI's economic impact by examining three main clusters of arguments: automating expensive tasks like programming, removing "cost centers" like call centers and corporate art, and claims of explosive growth. They dig into the actual data, debunk the hype, and explain why most productivity claims don't hold up in practice. Plus: MIT denounces a paper with fabricated data, and Grok randomly promotes white genocide myths.(00:00) - Recording date + intro (00:52) - MIT denounces paper (04:09) - Grok's white genocide (06:23) - Butthole convergence (07:13) - AI and the economy (14:50) - Automating profit...
2025-05-26
1h 06
Into AI Safety
Making Your Voice Heard w/ Tristan Williams & Felix de Simone
I am joined by Tristan Williams and Felix de Simone to discuss their work on the potential of constituent communication, specifically in the context of AI legislation. These two worked as part of an AI Safety Camp team to understand whether or not it would be useful for more people to be sharing their experiences, concerns, and opinions with their government representative (hint, it is).Check out the blogpost on their findings, "Talking to Congress: Can constituents contacting their legislator influence policy?" and the tool they created!(01:53) - Introductions (04:04) - Starting the project (13:30...
2025-05-19
1h 33
muckrAIkers
DeepSeek: 2 Months Out
DeepSeek has been out for over 2 months now, and things have begun to settle down. We take this opportunity to contextualize the developments that have occurred in its wake, both within the AI industry and the world economy. As systems get more "agentic" and users are willing to spend increasing amounts of time waiting for their outputs, the value of supposed "reasoning" models continues to be peddled by AI system developers, but does the data really back these claims?Check out our DeepSeek minisode for a snappier overview!EPISODE RECORDED 2025.03.30(00:40...
2025-04-09
1h 31
muckrAIkers
DeepSeek Minisode
DeepSeek R1 has taken the world by storm, causing a stock market crash and prompting further calls for export controls within the US. Since this story is still very much in development, with follow-up investigations and calls for governance being released almost daily, we thought it best to hold of for a little while longer to be able to tell the whole story. Nonetheless, it's a big story, so we provide a brief overview of all that's out there so far.(00:00) - Recording date (00:04) - Intro (00:37) - DeepSeek drop and reactions (04:27) - Export controls (08:05) - Skepticism...
2025-02-10
15 min
muckrAIkers
Understanding AI World Models w/ Chris Canal
Chris Canal, co-founder of EquiStamp, joins muckrAIkers as our first ever podcast guest! In this ~3.5 hour interview, we discuss intelligence vs. competencies, the importance of test-time compute, moving goalposts, the orthogonality thesis, and much more.A seasoned software developer, Chris started EquiStamp as a way to improve our current understanding of model failure modes and capabilities in late 2023. Now a key contractor for METR, EquiStamp evaluates the next generation of LLMs from frontier model developers like OpenAI and Anthropic.EquiStamp is hiring, so if you're a software developer interested in a fully remote opportunity with...
2025-01-27
3h 19
muckrAIkers
NeurIPS 2024 Wrapped đŻ
What happens when you bring over 15,000 machine learning nerds to one city? If your guess didn't include racism, sabotage and scandal, belated epiphanies, a spicy SoLaR panel, and many fantastic research papers, you wouldn't have captured my experience. In this episode we discuss the drama and takeaways from NeurIPS 2024.Posters available at time of episode preparation can be found on the episode webpage.EPISODE RECORDED 2024.12.22(00:00) - Recording date (00:05) - Intro (00:44) - Obligatory mentions (01:54) - SoLaR panel (18:43) - Test of Time (24:17) - And now: science! (28:53) - Downsides of benchmarks (41:39) - Improving the...
2024-12-30
1h 26
muckrAIkers
OpenAI's o1 System Card, Literally Migraine Inducing
The idea of model cards, which was introduced as a measure to increase transparency and understanding of LLMs, has been perverted into the marketing gimmick characterized by OpenAI's o1 system card. To demonstrate the adversarial stance we believe is necessary to draw meaning from these press-releases-in-disguise, we conduct a close read of the system card. Be warned, there's a lot of muck in this one.Note: All figures/tables discussed in the podcast can be found on the podcast website at https://kairos.fm/muckraikers/e009/(00:00) - Recorded 2024.12.08 (00:54) - Actual intro (03:00) - System...
2024-12-23
1h 16
muckrAIkers
How to Safely Handle Your AGI
While on the campaign trail, Trump made claims about repealing Biden's Executive Order on AI, but what will actually be changed when he gets into office? We take this opportunity to examine policies being discussed or implemented by leading governments around the world.(00:00) - Intro (00:29) - Hot off the press (02:59) - Repealing the AI executive order? (11:16) - "Manhattan" for AI (24:33) - EU (30:47) - UK (39:27) - Bengio (44:39) - Comparing EU/UK to USA (45:23) - China (51:12) - Taxes (55:29) - The muck LinksSFChronicle article - US gathers allies to talk AI safety as Trump's vow to...
2024-12-02
58 min
muckrAIkers
The End of Scaling?
Multiple news outlets, including The Information, Bloomberg, and Reuters [see sources] are reporting an "end of scaling" for the current AI paradigm. In this episode we look into these articles, as well as a wide variety of economic forecasting, empirical analysis, and technical papers to understand the validity, and impact of these reports. We also use this as an opportunity to contextualize the realized versus promised fruits of "AI".(00:23) - Hot off the press (01:49) - The end of scaling (10:50) - "Useful tools" and "agentic" "AI" (17:19) - The end of quantization (25:18) - Hedging (29:41) - The end...
2024-11-19
1h 07
muckrAIkers
US National Security Memorandum on AI, Oct 2024
October 2024 saw a National Security Memorandum and US framework for using AI in national security contexts. We go through the content so you don't have to, pull out the important bits, and summarize our main takeaways.(00:48) - The memorandum (06:28) - What the press is saying (10:39) - What's in the text (13:48) - Potential harms (17:32) - Miscellaneous notable stuff (31:11) - What's the US governments take on AI? (45:45) - The civil side - comments on reporting (49:31) - The commenters (01:07:33) - Our final hero (01:10:46) - The muck LinksUnited States National Security Memorandum on AIFact Sheet on the National...
2024-11-06
1h 16
muckrAIkers
Understanding Claude 3.5 Sonnet (New)
Frontier developers continue their war on sane versioning schema to bring us Claude 3.5 Sonnet (New), along with "computer use" capabilities. We discuss not only the new model, but also why Anthropic may have released this model and tool combination now.(00:00) - Intro (00:22) - Hot off the press (05:03) - Claude 3.5 Sonnet (New) Two 'o' 3000 (09:23) - Breaking down "computer use" (13:16) - Our understanding (16:03) - Diverging business models (32:07) - Why has Anthropic chosen this strategy? (43:14) - Changing the frame (48:00) - Polishing the lily LinksAnthropic press release - Introducing Claude 3.5 Sonnet (New)Model Card Addendum...
2024-10-30
1h 00
muckrAIkers
Winter is Coming for OpenAI
Brace yourselves, winter is coming for OpenAI - atleast, that's what we think. In this episode we look at OpenAI's recent massive funding round and ask "why would anyone want to fund a company that is set to lose net 5 billion USD for 2024?" We scrape through a whole lot of muck to find the meaningful signals in all this news, and there is a lot of it, so get ready!(00:00) - Intro (00:28) - Hot off the press (02:43) - Why listen? (06:07) - Why might VCs invest? (15:52) - What are people saying (23:10) - How *is* OpenAI making...
2024-10-22
1h 22
muckrAIkers
Open Source AI and 2024 Nobel Prizes
The Open Source AI Definition is out after years of drafting, will it reestablish brand meaning for the âOpen Sourceâ term? Also, the 2024 Nobel Prizes in Physics and Chemistry are heavily tied to AI; we scrutinize not only this year's prizes, but also Nobel Prizes as a concept. (00:00) - Intro (00:30) - Hot off the press (03:45) - Open Source AI background (10:30) - Definitions and changes in RC1 (18:36) - âBusiness sourceâ (22:17) - Parallels with legislation (26:22) - Impacts of the OSAID (33:58) - 2024 Nobel Prize Context (37:21) - Chemistry prize (45:06) - Physics prize (50:29) - Takeaways (52:03) - Whatâs the real muck? (01:00:27) - Outro
2024-10-16
1h 01
muckrAIkers
SB1047
Why is Mark Ruffalo talking about SB1047, and what is it anyway? Tune in for our thoughts on the now vetoed California legislation that had Big Tech scared.(00:00) - Intro (00:31) - Updates from a relatively slow week (03:32) - Disclaimer: SB1047 vetoed during recording (still worth a listen) (05:24) - What is SB1047 (12:30) - Definitions (17:18) - Understanding the bill (28:42) - What are the players saying about it? (46:44) - Addressing critiques (55:59) - Open Source (01:02:36) - Takeaways (01:15:40) - Clarification on impact to big tech (01:18:51) - Outro LinksSB1047 legislation pageSB1047 CalMatters pageNewsom vetoes SB1047CAIS newsletter on SB1047Prominent AI...
2024-09-30
1h 19
muckrAIkers
OpenAI's o1, aka. Strawberry
OpenAI's new model is out, and we are going to have to rake through a lot of muck to get the value out of this one!â Opt out of LinkedIn's GenAI scraping âĄď¸ https://lnkd.in/epziUeTi (00:00) - Intro (00:25) - Other recent news (02:57) - Hot off the press (03:58) - Why might someone care? (04:52) - What is it? (06:49) - How is it being sold? (10:45) - How do they explain it, technically? (27:09) - Reflection AI Drama (40:19) - Why do we care? (46:39) - Scraping away the muck Note: at around 32 minutes, Igor says the incorrect Llama model versio...
2024-09-23
50 min
The Third Angle
CorPower Ocean: Making Waves In Renewable Energy
âWeâre trying to be the most environmentally friendly option for power production.âIn this episode we're catching the wave - literally - with CorPower Ocean, the trailblazers of wave power technology. Join us as we explore how this pioneering company is leading the way with a carbon-free solution that balances supply and demand more efficiently, requiring less generation capacity, storage, and grid infrastructure. Engineers, Jacob Ljungbäck and Antoine Bonel take us into their world of wave energy and tell us all about this forward thinking companyâs mission.Their innovative approach not only reduces th...
2024-06-25
18 min
Into AI Safety
INTERVIEW: Scaling Democracy w/ (Dr.) Igor Krawczuk
The almost Dr. Igor Krawczuk joins me for what is the equivalent of 4 of my previous episodes. We get into all the classics: eugenics, capitalism, philosophical toads... Need I say more?If you're interested in connecting with Igor, head on over to his website, or check out placeholder for thesis (it isn't published yet).Because the full show notes have a whopping 115 additional links, I'll highlight some that I think are particularly worthwhile here:The best article you'll ever read on Open Source AIThe best article you'll ever read on emergence in MLKate Crawford's...
2024-06-03
2h 58
Into AI Safety
INTERVIEW: StakeOut.AI w/ Dr. Peter Park (3)
As always, the best things come in 3s: dimensions, musketeers, pyramids, and... 3 installments of my interview with Dr. Peter Park, an AI Existential Safety Post-doctoral Fellow working with Dr. Max Tegmark at MIT.As you may have ascertained from the previous two segments of the interview, Dr. Park cofounded StakeOut.AI along with Harry Luk and one other cofounder whose name has been removed due to requirements of her current position. The non-profit had a simple but important mission: make the adoption of AI technology go well, for humanity, but unfortunately, StakeOut.AI had to dissolve in...
2024-03-25
1h 42
Into AI Safety
INTERVIEW: StakeOut.AI w/ Dr. Peter Park (2)
Join me for round 2 with Dr. Peter Park, an AI Existential Safety Postdoctoral Fellow working with Dr. Max Tegmark at MIT. Dr. Park was a cofounder of StakeOut.AI, a non-profit focused on making AI go well for humans, along with Harry Luk and one other individual, whose name has been removed due to requirements of her current position.In addition to the normal links, I wanted to include the links to the petitions that Dr. Park mentions during the podcast. Note that the nonprofit which began these petitions, StakeOut.AI, has been dissolved.Right AI...
2024-03-18
1h 06
Into AI Safety
MINISODE: Restructure Vol. 2
UPDATE: Contrary to what I say in this episode, I won't be removing any episodes that are already published from the podcast RSS feed.After getting some advice and reflecting more on my own personal goals, I have decided to shift the direction of the podcast towards accessible content regarding "AI" instead of the show's original focus. I will still be releasing what I am calling research ride-along content to my Patreon, but the show's feed will consist only of content that I aim to make as accessible as possible.00:35 - TL;DL01:12...
2024-03-11
13 min
Into AI Safety
INTERVIEW: StakeOut.AI w/ Dr. Peter Park (1)
Dr. Peter Park is an AI Existential Safety Postdoctoral Fellow working with Dr. Max Tegmark at MIT. In conjunction with Harry Luk and one other cofounder, he founded â StakeOut.AI, a non-profit focused on making AI go well for humans.00:54 - Intro03:15 - Dr. Park, x-risk, and AGI08:55 - StakeOut.AI12:05 - Governance scorecard19:34 - Hollywood webinar22:02 - Regulations.gov comments23:48 - Open letters 26:15 - EU AI Act35:07 - Effective accelerationism40:50 - Divide and conquer dynamics45:40 - AI "art"53:09 - OutroLinks to all art...
2024-03-04
54 min
Into AI Safety
MINISODE: "LLMs, a Survey"
Take a trip with me through the paper Large Language Models, A Survey, published on February 9th of 2024. All figures and tables mentioned throughout the episode can be found on the Into AI Safety podcast website.00:36 - Intro and authors01:50 - My takes and paper structure04:40 - Getting to LLMs07:27 - Defining LLMs & emergence12:12 - Overview of PLMs15:00 - How LLMs are built18:52 - Limitations if LLMs23:06 - Uses of LLMs25:16 - Evaluations and Benchmarks28:11 - Challenges and future directions29:21 - Recap & outroLinks to...
2024-02-26
30 min
Into AI Safety
FEEDBACK: Applying for Funding w/ Esben Kran
Esben reviews an application that I would soon submit for Open Philanthropy's Career Transitition Funding opportunity. Although I didn't end up receiving the funding, I do think that this episode can be a valuable resource for both others and myself when applying for funding in the future.Head over to Apart Research's website to check out their work, or the Alignment Jam website for information on upcoming hackathons.A doc-capsule of the application at the time of this recording can be found at this link.01:38 - Interview starts05:41 - Proposal11:00...
2024-02-19
45 min
Into AI Safety
MINISODE: Reading a Research Paper
Before I begin with the paper-distillation based minisodes, I figured we would go over best practices for reading research papers. I go through the anatomy of typical papers, and some generally applicable advice.00:56 - Anatomy of a paper02:38 - Most common advice05:24 - Reading sparsity and path07:30 - Notes and motivationLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Ten simple rules for reading a scientific paperBest sources I foundLet's get critical: Reading academic articles#GradHacks: A guide to...
2024-02-12
09 min
Into AI Safety
HACKATHON: Evals November 2023 (2)
Join our hackathon group for the second episode in the Evals November 2023 Hackathon subseries. In this episode, we solidify our goals for the hackathon after some preliminary experimentation and ideation.Check out Stellaric's website, or follow them on Twitter.01:53 - Meeting starts05:05 - Pitch: extension of locked models23:23 - Pitch: retroactive holdout datasets34:04 - Preliminary results37:44 - Next steps42:55 - RecapLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Evalugator libraryPassword Locked Model blogpostTruthfulQA: Measuring...
2024-02-05
48 min
Into AI Safety
MINISODE: Portfolios
I provide my thoughts and recommendations regarding personal professional portfolios.00:35 - Intro to portfolios01:42 - Modern portfolios02:27 - What to include04:38 - Importance of visual05:50 - The "About" page06:25 - Tools08:12 - Future of "Minisodes"Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.From Portafoglio to Eportfolio: The Evolution of Portfolio in Higher EducationGIMPAlternativeToJekyllGitHub PagesMinimal MistakesMy portfolio
2024-01-29
09 min
Into AI Safety
INTERVIEW: Polysemanticity w/ Dr. Darryl Wright
Darryl and I discuss his background, how he became interested in machine learning, and a project we are currently working on investigating the penalization of polysemanticity during the training of neural networks.Check out a diagram of the decoder task used for our research!01:46 - Interview begins02:14 - Supernovae classification08:58 - Penalizing polysemanticity20:58 - Our "toy model"30:06 - Task description32:47 - Addressing hurdles39:20 - Lessons learnedLinks to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance....
2024-01-22
45 min
Into AI Safety
MINISODE: Starting a Podcast
A summary and reflections on the path I have taken to get this podcast started, including some resources recommendations for others who want to do something similar.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.LessWrongSpotify for PodcastersInto AI Safety podcast websiteEffective Altruism GlobalOpen Broadcaster Software (OBS)CraigRiverside
2024-01-15
10 min
Into AI Safety
HACKATHON: Evals November 2023 (1)
This episode kicks off our first subseries, which will consist of recordings taken during my team's meetings for the AlignmentJams Evals Hackathon in November of 2023. Our team won first place, so you'll be listening to the process which, at the end of the day, turned out to be pretty good.Check out Apart Research, the group that runs the AlignmentJamz Hackathons.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.Generalization Analogies: A Testbed for Generalizing AI...
2024-01-08
1h 08
Into AI Safety
MINISODE: Staying Up-to-Date in AI
In this minisode I give some tips for staying up-to-date in the everchanging landscape of AI. I would like to point out that I am constantly iterating on these strategies, tools, and sources, so it is likely that I will make an update episode in the future.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance.ToolsFeedlyarXiv Sanity LiteZoteroAlternativeToMy "Distilled AI" FolderAI Explained YouTube channelAI Safety newsletterData Machina newsletterImport AIMidwit AlignmentHonourable MentionsAI Alignment ForumLessWrongBounded Regret (Jacob Steinhart's blog)Cold Takes (Holden Karnofsky's...
2024-01-01
13 min
Into AI Safety
INTERVIEW: Applications w/ Alice Rigg
Alice Rigg, a mechanistic interpretability researcher from Ottawa, Canada, joins me to discuss their path and the applications process for research/mentorship programs.Join the Mech Interp Discord server and attend reading groups at 11:00am on Wednesdays (Mountain Time)!Check out Alice's website.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. EleutherAI Join the public EleutherAI discord server DistillEffective Altruism (EA)MATS Retrospective Summer 2023 postAmbitious Mechanistic Interpretability AISC research plan by Alice RiggSPARStability AI During their most recent fund...
2023-12-18
1h 10
Into AI Safety
MINISODE: Program Applications (Winter 2024)
We're back after a month-long hiatus with a podcast refactor and advice on the applications process for research/mentorship programs.Check out the About page on the Into AI Safety website for a summary of the logistics updates.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. MATSASTRA FellowshipARENAAI Safety CampBlueDot ImpactTech with TimFast.AI's Practical Deep Learning for CodersKaggleAlignmentJamsLessWrongAI Alignment Forum
2023-12-11
18 min
Into AI Safety
MINISODE: EAG Takeaways (Boston 2023)
This episode is a brief overview of the major takeaways I had from attending EAG Boston 2023, and an update on my plans for the podcast moving forward.TL;DLStarting in early December (2023), I will be uploading episodes on a biweekly basis (day TBD).I won't be releasing another episode until then, so that I can build a cache of episodes up.During this month (November 2023), I'll also try to get the podcast up on more platforms, set up comments on more platforms, and create an anonymous feedback form.Links Links to all articles/papers...
2023-12-04
09 min
Into AI Safety
FEEDBACK: AISC Proposal w/ Remmelt Ellen
In this episode I discuss my initial research proposal for the 2024 Winter AI Safety Camp with one of the individuals who helps facilitate the program, Remmelt Ellen.The proposal is titled The Effect of Machine Learning on Bioengineered Pandemic Risk. A doc-capsule of the proposal at the time of this recording can be found at this link.Links to all articles/papers which are mentioned throughout the episode can be found below, in order of their appearance. MegaSyn: Integrating Generative Molecule Design, Automated Analog Designer and Synthetic Viability PredictionDual use of artificial-intelligence-powered drug discoveryArtificial i...
2023-11-27
56 min
Into AI Safety
MINISODE: Introduction and Motivation
Welcome to the Into AI Safety podcast! In this episode I provide reasoning for why I am starting this podcast, what I am trying to accomplish with it, and a little bit of background on how I got here.Please email all inquiries and suggestions to intoaisafety@gmail.com.
2023-11-13
09 min