Shows
Zero SignalThe Importance of Purple Teaming in AI Security
- ft. Disesdi Susanna CoxEpisode SummaryIn this episode, AI architect and security researcher Disesdi Susanna Cox explains the vast and complex attack surface of AI systems, highlighting the need for new security approaches like purple teaming and MLSecOps. Her insights help security leaders understand the unique risks and ethical challenges of AI, making this a must-listen for anyone responsible for securing modern AI-driven organizations.SponsorsThank you to our sponsors who make this show possible.→ Hampton North. Hampton North is the premium US based cybersecurity search firm.→ Sysd...
2025-10-0836 min
Talent First‘Who’s Really Logging In? AI Agents, Access, and the CISO’s New Frontier’ with Diana KelleySEASON FIVE: In this weeks episode, I chat to Diana Kelley, CISO at Noma about a range of issues around AI integration, policy and governance and about what she is seeing when it comes to the impact of generative AI on recruitment and hiring. We also talk about the CISO role - is it becoming totally unmanageable and will we see a rise in the Deputy CISO position - or will CISOs need to become HR managers for AI Agents?About Our Guest:Diana Kelley is the Chief Information Security Officer (CISO) at Noma Security...
2025-08-2949 min
What's in the SOSS? An OpenSSF PodcastSecuring AI: A Conversation with Sarah Evans on OpenSSF's AI/ML InitiativesIn this episode of "What's in the SOSS," we welcome back Sarah Evans, Distinguished Engineer at Dell Technologies and a key figure in the OpenSSF's AI/ML working group. Sarah discusses the critical work being done to extend secure software development practices to the rapidly evolving field of AI. She dives into the AI Model Signing project, the groundbreaking MLOps white paper developed in partnership with Ericsson, and the crucial work of identifying and addressing new personas in AI/ML operations. Tune in to learn how OpenSSF is shaping the future of AI security and what challenges and opportunities...
2025-08-2614 min
Cloud Security Podcast by GoogleEP238 Google Lessons for Using AI Agents for Securing Our EnterpriseGuest: Dominik Swierad, Senior PM D&R AI and Sec-Gemini Topics: When introducing AI agents to security teams at Google, what was your initial strategy to build trust and overcome the natural skepticism? Can you walk us through the very first conversations and the key concerns that were raised? With a vast array of applications, how did you identify and prioritize the initial use cases for AI agents within Google's enterprise security? What specific criteria made a use case a good candidate for early evaluation? Were there any surprising 'no-go' areas you discovered?" Beyond simple efficiency ga...
2025-08-1131 min
Daily Security ReviewNvidia Triton Inference Server Vulnerabilities Expose AI Infrastructure to AttackA major warning has hit the AI community: Nvidia’s Triton Inference Server — one of the most widely used open-source platforms for deploying and scaling AI models — has been found to contain critical vulnerabilities that could allow attackers to take complete remote control of affected systems.The discovery, made by cloud security firm Wiz, revealed a chain of flaws that escalate from information disclosure to remote code execution (RCE), enabling attackers to not only steal valuable AI models but also access sensitive organizational data. Nvidia has since released urgent patches, but the incident highlights the growing security crisis...
2025-08-051h 02
The MLSecOps PodcastSeason 3 Finale: Top Insights, Hacks, and Lessons from the Frontlines of AI SecuritySend us a textTo close out Season 3, we’re revisiting the standout insights, wildest vulnerabilities, and most practical lessons shared by 20+ AI practitioners, researchers, and industry leaders shaping the future of AI security. If you're building, breaking, or defending AI/ML systems, this is your must-listen roundup.Full transcript, video, and links to episode resources available at https://mlsecops.com/podcast/season-3-finale-top-insights-hacks-and-lessons-from-the-frontlines-of-ai-securityThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and re...
2025-07-2124 minThe MLSecOps PodcastBreaking and Securing Real-World LLM AppsSend us a textFresh off their OWASP AppSec EU talk, Rico Komenda and Javan Rasokat join Charlie McCarthy to share real-world insights on breaking and securing LLM-integrated systems.Full transcript, video, and links to episode resources available at https://mlsecops.com/podcast/breaking-and-securing-real-world-llm-appsAsk ChatGPTThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at...
2025-07-1653 minThe MLSecOps PodcastHow Red Teamers Are Exposing Flaws in AI PipelinesSend us a textProlific bug bounty hunter and Offensive Security Lead at Toreon, Robbe Van Roey (PinkDraconian), joins the MLSecOps Podcast to break down how he discovered RCEs in BentoML and LangChain, the risks of unsafe model serialization, and his approach to red teaming AI systems. Full transcript, video, and links to episode resources available at https://mlsecops.com/podcast/how-red-teamers-are-exposing-flaws-in-ai-pipelinesThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:
2025-07-0941 minThe MLSecOps PodcastSecuring AI for Government: Inside the Leidos + Protect AI PartnershipSend us a textOn this episode of the MLSecOps Podcast, Rob Linger, Information Advantage Practice Lead at Leidos, join hosts Jessica Souder, Director of Government and Defense at Protect AI, and Charlie McCarthy to explore what it takes to deploy secure AI/ML systems in government environments.Full transcript, video, and links to episode resources available at https://mlsecops.com/podcast/securing-ai-for-government-inside-the-leidos-protect-ai-partnership.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. ...
2025-06-2534 minThe MLSecOps PodcastHolistic AI Pentesting PlaybookSend us a textJason Haddix, CEO of Arcanum Information Security, joins the MLSecOps Podcast to share his methods for assessing and defending AI systems.Full transcript, video, and links to episode resources available at https://mlsecops.com/podcast/holistic-ai-pentesting-playbook.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML...
2025-06-1349 minThe MLSecOps PodcastAI Agent Security: Threats & Defenses for Modern DeploymentsSend us a textResearchers Yifeng (Ethan) He and Peter Rong join host Madi Vorbrich to break down their paper "Security of AI Agents." They explore real-world AI agent threats, like session hijacks and tool-based jailbreaks, and share practical defenses, from sandboxing to agent-to-agent protocols.Full transcript with links to resources available at https://mlsecops.com/podcast/ai-agent-security-threats-defenses-for-modern-deploymentsThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:...
2025-05-2131 minThe MLSecOps PodcastAutonomous Agents Beyond the HypeSend us a textPart 2 with Gavin Klondike dives into autonomous AI agents—how they really work, the attack paths they open, and practical defenses like least-privilege APIs and out-of-band auth. A must-listen roadmap for anyone building—or defending—the next generation of AI applications.Full transcript with links to resources available at https://mlsecops.com/podcast/autonomous-agents-beyond-the-hypeThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protec...
2025-05-1424 minCloud Security Podcast by GoogleEP224 Protecting the Learning Machines: From AI Agents to Provenance in MLSecOpsGuest: Diana Kelley, CSO at Protect AI Topics: Can you explain the concept of "MLSecOps" as an analogy with DevSecOps, with 'Dev' replaced by 'ML'? This has nothing to do with SecOps, right? What are the most critical steps a CISO should prioritize when implementing MLSecOps within their organization? What gets better when you do it? How do we adapt traditional security testing, like vulnerability scanning, SAST, and DAST, to effectively assess the security of machine learning models? Can we? In the context of AI supply chain security, what is the essential role of third-party assessments, pa...
2025-05-1230 minThe MLSecOps PodcastBeyond Prompt Injection: AI’s Real Security GapsSend us a textIn Part 1 of this two-part MLSecOps Podcast, Principal Security Consultant Gavin Klondike joins Dan and Marcello to break down the real threats facing AI systems today. From prompt injection misconceptions to indirect exfiltration via markdown and the failures of ML Ops security practices, Gavin unpacks what the industry gets wrong—and how to fix it.Full transcript with links to resources available at https://mlsecops.com/podcast/beyond-prompt-injection-ais-real-security-gapsThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.co...
2025-04-3026 minThe MLSecOps PodcastWhat’s Hot in AI Security at RSA Conference 2025?Send us a textWhat’s really hot at RSA Conference 2025? MLSecOps Community Manager Madi Vorbrich sits down with Protect AI Co‑Founder Daryan “D” Dehghanpisheh for a rapid rundown of must‑see sessions, booth events, and emerging AI‑security trends—from GenAI agents to zero‑trust AI and million‑model scans. Use this episode to build a bullet‑proof RSA agenda before you land in San Francisco.Full transcript with links to resources available at https://mlsecops.com/podcast/whats-hot-in-ai-security-at-rsa-conference-2025Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find m...
2025-04-2124 minThe MLSecOps PodcastUnpacking the Cloud Security Alliance AI Controls MatrixSend us a textIn this episode of the MLSecOps Podcast, we sit down with three expert contributors from the Cloud Security Alliance’s AI Controls Matrix working group. They reveal how this newly released framework addresses emerging AI threats—like model poisoning and adversarial manipulation—through robust technical controls, detailed implementation guidelines, and clear auditing strategies.Full transcript with links to resources available at https://mlsecops.com/podcast/unpacking-the-cloud-security-alliance-ai-controls-matrix Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.Add...
2025-04-1635 minThe MLSecOps PodcastFrom Pickle Files to Polyglots: Hidden Risks in AI Supply ChainsSend us a textJoin Keith Hoodlet from Trail of Bits as he dives into AI/ML security, discussing everything from prompt injection and fuzzing techniques to bias testing and compliance challenges.Full transcript with links to resources available at https://mlsecops.com/podcast/from-pickle-files-to-polyglots-hidden-risks-in-ai-supply-chainsThanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI
2025-04-0241 minThe MLSecOps PodcastRethinking AI Red Teaming: Lessons in Zero Trust and Model ProtectionSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/rethinking-ai-red-teaming-lessons-in-zero-trust-and-model-protectionThis episode is a follow up to Part 1 of our conversation with returning guest Brian Pendleton, as he challenges the way we think about red teaming and security for AI. Continuing from last week’s exploration of enterprise AI adoption and high-level security considerations, the conversation now shifts to how red teaming, zero trust, and privacy concerns intertwine with AI’s unique risks.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and...
2025-03-2036 minThe MLSecOps PodcastAI Security: Map It, Manage It, Master ItSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/ai-security-map-it-manage-it-master-itIn part one of our two-part MLSecOps Podcast episode, security veteran Brian Pendleton takes us from his early hacker days to the forefront of AI security. Brian explains why mapping every AI integration is essential for uncovering vulnerabilities. He also dives into the benefits of using SBOMs over model cards for risk management and stresses the need to bridge the gap between ML and security teams to protect your enterprise AI ecosystem.Thanks for checking out...
2025-03-1341 minThe MLSecOps PodcastAgentic AI: Tackling Data, Security, and Compliance RisksSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/agentic-ai-tackling-data-security-and-compliance-risksJoin host Diana Kelley and CTO Dr. Gina Guillaume-Joseph as they explore how agentic AI, robust data practices, and zero trust principles drive secure, real-time video analytics at Camio. They discuss why clean data is essential, how continuous model validation can thwart adversarial threats, and the critical balance between autonomous AI and human oversight. Dive into the world of multimodal modeling, ethical safeguards, and what it takes to ensure AI remains both innovative and risk-aware.Thanks...
2025-03-0523 minThe MLSecOps PodcastAI Vulnerabilities: ML Supply Chains to LLM and Agent ExploitsSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/ai-vulnerabilities-ml-supply-chains-to-llm-and-agent-exploitsJoin host Dan McInerney and AI security expert Sierra Haex as they explore the evolving challenges of AI security. They discuss vulnerabilities in ML supply chains, the risks in tools like Ray and untested AI model files, and how traditional security measures intersect with emerging AI threats. The conversation also covers the rise of open-source models like DeepSeek and the security implications of deploying autonomous AI agents, offering critical insights for anyone looking to secure distributed AI systems.
2025-02-2424 minThe MLSecOps PodcastImplementing Enterprise AI Governance: Balancing Ethics, Innovation & Risk for Business SuccessSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/implementing-a-robust-ai-governance-framework-for-business-successIn this episode of the MLSecOps podcast, host Charlie McCarthy sits down with Chris McClean, Global Lead for Digital Ethics at Avanade, to explore the world of responsible AI governance. They discuss how ethical principles, risk management, and robust security practices can be integrated throughout the AI lifecycle—from design and development to deployment and oversight. Learn practical strategies for building resilient AI frameworks, understanding regulatory impacts, and driving innovation safely. Thanks for checking out the MLS...
2025-02-1438 minThe MLSecOps PodcastUnpacking Generative AI Red Teaming and Practical Security SolutionsSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/unpacking-generative-ai-red-teaming-and-practical-security-solutionsIn this episode, we explore LLM red teaming beyond simple “jailbreak” prompts with special guest Donato Capitella, from WithSecure Consulting. You’ll learn why vulnerabilities live in context—how LLMs interact with users, tools, and documents—and discover best practices for mitigating attacks like prompt injection. Our guest also previews an open-source tool for automating security tests on LLM applications.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources...
2025-02-0651 min
Tech Behind FintechIs MLSecOps the Missing Piece in AI Model Security?Join us on Tech Behind FinTech for an intriguing episode titled "Is MLSecOps the Missing Piece in AI Model Security?".Our guest, Daryan "D" Dehghanpisheh, President of Protect AI, brings unparalleled expertise in AI, machine learning, and cybersecurity. With a career spanning roles at AWS, NYSE, and Intel, D has been at the forefront of AI innovation, scaling some of the world’s largest AI businesses. In this episode, we’ll explore the emerging field of MLSecOps—a fusion of machine learning and cybersecurity. D will break down how Protect AI is pioneering secure environments for AI builders, addressing unique...
2025-01-0449 minThe MLSecOps PodcastAI Security: Vulnerability Detection and Hidden Model File RisksSend us a textIn this episode of the MLSecOps Podcast, the team dives into the transformative potential of Vulnhuntr: zero shot vulnerability discovery using LLMs. Madison Vorbrich hosts Dan McInerney and Marcello Salvati to discuss Vulnhuntr’s ability to autonomously identify vulnerabilities, including zero-days, using large language models (LLMs) like Claude. They explore the evolution of AI tools for security, the gap between traditional and AI-based static code analysis, and how Vulnhuntr enables both developers and security teams to proactively safeguard their projects. The conversation also highlights Protect AI’s bug bounty platform, huntr.com, and its...
2024-12-0938 minThe MLSecOps PodcastAI Governance Essentials: Empowering Procurement Teams to Navigate AI RiskSend us a textFull transcript with links to resources available at https://mlsecops.com/podcast/ai-governance-essentials-empowering-procurement-teams-to-navigate-ai-risk.In this episode of the MLSecOps Podcast, Charlie McCarthy from Protect AI sits down with Dr. Cari Miller to discuss the evolving landscapes of AI procurement and governance. Dr. Miller shares insights from her work with the AI Procurement Lab and ForHumanity, delving into the essential frameworks and strategies needed to mitigate risks in AI acquisitions. They cover the AI Procurement Risk Management Framework, practical ways to ensure transparency and accountability, and how the September 2024 OMB Memo M-24...
2024-11-0737 minThe MLSecOps PodcastCrossroads: AI, Cybersecurity, and How to Prepare for What's NextSend us a textIn this episode of the MLSecOps Podcast, Distinguished Engineer Nicole Nichols from Palo Alto Networks joins host and Machine Learning Scientist Mehrin Kiani to explore critical challenges in AI and cybersecurity. Nicole shares her unique journey from mechanical engineering to AI security, her thoughts on the importance of clear AI vocabularies, and the significance of bridging disciplines in securing complex systems. They dive into the nuanced definitions of AI fairness and safety, examine emerging threats like LLM backdoors, and discuss the rapidly evolving impact of autonomous AI agents on cybersecurity defense. Nicole’s in...
2024-10-3033 minThe MLSecOps PodcastAI Beyond the Hype: Lessons from Cloud on Risk and SecuritySend us a textOn this episode of the MLSecOps Podcast, we’re bringing together two cybersecurity legends. Our guest is the inimitable Caleb Sima, who joins us to discuss security considerations for building and using AI, drawing on his 25+ years of cybersecurity experience. Caleb's impressive journey includes co-founding two security startups acquired by HP and Lookout, serving as Chief Security Officer at Robinhood, and currently leading cybersecurity venture studio WhiteRabbit & chairing the Cloud Security Alliance AI Safety Initiative.Hosting this episode is Diana Kelley (CISO, Protect AI) an industry powerhouse with a long career de...
2024-10-0241 minThe MLSecOps PodcastGenerative AI Prompt Hacking and Its Impact on AI Security & SafetySend us a textWelcome to Season 3 of the MLSecOps Podcast, brought to you by Protect AI!In this episode, MLSecOps Community Manager Charlie McCarthy speaks with Sander Schulhoff, co-founder and CEO of Learn Prompting. Sander discusses his background in AI research, focusing on the rise of prompt engineering and its critical role in generative AI. He also shares insights into prompt security, the creation of LearnPrompting.org, and its mission to democratize prompt engineering knowledge. This episode also explores the intricacies of prompting techniques, "prompt hacking," and the impact of competitions like HackAPrompt on...
2024-09-1931 min
Trustworthy AI : De-risk business adoption of AISecuring GenAI: Secure our FutureThis episode of Trustworthy AI : De-risk business adoption of AI is brought to you by Trusted AI. Securing Generative AI, LLMs is a critical topic for adopting AI.Host Pamela Gupta, CEO Trusted AI talks with Steve Wilson leader of the LLM Governance & Cybersecurity OWASP , Product officer at Exabeam, author Developers Playbook for LLM security, O’ReillyWhy is Securing GenAI critical?Organizations are increasingly prioritizing value creation and demanding tangible results from their Generative AI initiatives. This requires them to scale up their Generative AI deployments— advancing beyond experimentation, pilots and proofs of co...
2024-09-1022 minThe MLSecOps PodcastThe MLSecOps Podcast Season 2 FinaleSend us a textThis compilation contains highlights from episodes throughout Season 2 of the MLSecOps Podcast, and it's a great one for community members who are new to the show. If there is a clip from this highlights reel that is especially interesting to you, you can note the name of the original episode that the clip came from and easily go check out that full length episode for a deeper dive.Extending enormous thanks to everyone who has supported this show, including our audience, Protect AI hosts, and stellar expert guests. Stay tuned for...
2024-09-0740 min
MLOps.communityVisualize - Bringing Structure to Unstructured Data // Markus Stoll // #258Markus Stoll is the Co-Founder of Renumics and the developer behind the open-source interactive ML dataset exploration tool, Spotlight. He shares insights on:AI in Engineering and ManufacturingInteractive ML Data VisualizationML Data ExplorationFollow Markus for hands-on articles about leveraging ML while keeping a strong focus on data.Visualize - Bringing Structure to Unstructured Data // MLOps Podcast #258 with Markus Stoll, CTO of Renumics.A huge thank you to SAS for their generous support! // Abstract...
2024-09-0350 min
MLOps.communityMLSecOps is Fundamental to Robust AISPM // Sean Morgan // #257Sean Morgan is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons. Learn more about the platform for end-to-end AI Security at https://protectai.com/.MLSecOps is Fundamental to Robust AI Security Posture Management (AISPM) // MLOps Podcast #257 with Sean Morgan, Chief Architect at Protect AI.// AbstractMLSecOps, which is the practice of integrating security practices into the AIML lifecycle (think infusing MLOps with DevSecOps practices), is a critical part of any team’s AI Security Posture Management. In this talk, we...
2024-08-3042 minThe MLSecOps PodcastExploring Generative AI Risk Assessment and Regulatory ComplianceSend us a textIn this episode of the MLSecOps Podcast we have the honor of talking with David Rosenthal, Partner at VISCHER (Swiss Law, Tax & Compliance). David is also an author & former software developer, and lectures at ETH Zürich & the University of Basel. He has more than 25 years of experience in data & technology law and kindly joined the show to discuss a variety of AI regulation topics, including the EU Artificial Intelligence Act, generative AI risk assessment, and challenges related to organizational compliance with upcoming AI regulations.Thanks for checking out th...
2024-07-2637 minThe MLSecOps PodcastMLSecOps Culture: Considerations for AI Development and Security TeamsSend us a textIn this episode, we had the pleasure of welcoming Co-Founder and CISO of Weights & Biases, Chris Van Pelt, to the MLSecOps Podcast. Chris discusses a range of topics with hosts Badar Ahmed and Diana Kelley, including the history of how W&B was formed, building a culture of security & knowledge sharing across teams in an organization, real-world ML and GenAI security concerns, data lineage and tracking, and upcoming features in the Weights & Biases platform for enhancing security.More about our guest speaker: Chris Van Pelt is a co-founder of W...
2024-07-0338 minThe MLSecOps PodcastPractical Offensive and Adversarial ML for Red TeamsSend us a textNext on the MLSecOps Podcast, we have the honor of highlighting one of our MLSecOps Community members and Dropbox™ Red Teamers, Adrian Wood. Adrian joined Protect AI threat researchers, Dan McInerney and Marcello Salvati, in the studio to share an array of insights, including what inspired him to create the Offensive ML (aka OffSec ML) Playbook, and diving into categories like adversarial machine learning (ML), offensive/defensive ML, and supply chain attacks.The group also discusses dual uses for "traditional" ML and LLMs in the realm of security, the ri...
2024-06-1735 min
Nexus: A Claroty PodcastDiana Kelley on Protecting the AI LifecycleProtect AI Chief Information Security Officer Diana Kelley joins the Claroty Nexus podcast to discuss the intricacies of securing machine learning and artificial intelligence use inside the enterprise. She also explains the concept of MLSecOps and how it compares and contrasts to DevOps used in application development. For more, visit nexusconnect.io/podcasts
2024-06-0326 minThe MLSecOps PodcastExpert Talk from RSA Conference: Securing Generative AISend us a textIn this episode, host Neal Swaelens (EMEA Director of Business Development, Protect AI) catches up with Ken Huang, CISSP at RSAC 2024 to talk about security for generative AI. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Sou...
2024-05-2025 minThe MLSecOps PodcastPractical Foundations for Securing AISend us a textIn this episode of the MLSecOps Podcast, we delve into the critical world of security for AI and machine learning with our guest Ron F. Del Rosario, Chief Security Architect and AI/ML Security Lead at SAP ISBN. The discussion highlights the contextual knowledge gap between ML practitioners and cybersecurity professionals, emphasizing the importance of cross-collaboration and foundational security practices. We explore the contrasts of security for AI to that for traditional software, along with the risk profiles of first-party vs. third-party ML models. Ron sheds light on the significance of understanding your...
2024-05-1338 minThe MLSecOps PodcastEvaluating RAG and the Future of LLM Security: Insights with LlamaIndexSend us a textIn this episode of the MLSecOps Podcast, host Neal Swaelens, along with co-host Oleksandr Yaremchuk, sit down with special guest Simon Suo, co-founder and CTO of LlamaIndex. Simon shares insights into the development of LlamaIndex, a leading data framework for orchestrating data in large language models (LLMs). Drawing from his background in the self-driving industry, Simon discusses the challenges and considerations of integrating LLMs into various applications, emphasizing the importance of contextualizing LLMs within specific environments.The conversation delves into the evolution of retrieval-augmented generation (RAG) techniques and the future trajectory...
2024-04-2331 minThe MLSecOps PodcastAI Threat Research: Spotlight on the Huntr CommunitySend us a textLearn about the world’s first bug bounty platform for AI & machine learning, huntr, including how to get involved!This week’s featured guests are leaders from the huntr community (brought to you by Protect AI): Dan McInerney, Lead AI Threat Researcher Marcello Salvati, Sr. Engineer & Researcher Madison Vorbrich, Community Manager Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:
2024-03-1331 minThe MLSecOps PodcastSecuring AI: The Role of People, Processes & Tools in MLSecOpsSend us a textIn this episode of The MLSecOps Podcast hosted by Daryan Dehghanpisheh (Protect AI) and special guest-host Martin Stanley, CISSP (Cybersecurity and Infrastructure Security Agency), we delve into critical aspects of AI security and operations. This episode features esteemed guests, Gary Givental (IBM) and Kaleb Walton (FICO).The group's discussion unfolds with insights into the evolving field of Machine Learning Security Operations, aka, MLSecOps. A recap of CISA's most recent Secure by Design and Secure AI initiatives sets the stage for the a dialogue that explores the parallels between MLSecOps and DevSecOps...
2024-02-2937 minThe MLSecOps PodcastReDoS Vulnerability Reports: Security Relevance vs. Noisy NuisanceSend us a textIn this episode, we delve into a hot topic in the bug bounty world: ReDoS (Regular Expression Denial of Service) reports. Inspired by reports submitted by the huntr AI/ML bug bounty community and an insightful blog piece by open source expert, William Woodruff (Engineering Director, Trail of Bits), this conversation explores: Are any ReDoS vulnerabilities worth fixing?Triaging and the impact of ReDoS reports on software maintainers.The challenges of addressing ReDoS vulnerabilities amidst developer fatigue and resource constraints.Analyzing the evolving trends and incentives shaping the rise of ReDoS reports i...
2024-02-2735 minThe MLSecOps PodcastFinding a Balance: LLMs, Innovation, and SecuritySend us a textIn this episode of The MLSecOps Podcast, special guest, Sandy Dunn, joins us to discuss the dynamic world of large language models (LLMs) and the equilibrium of innovation and security. Co-hosts, Daryan “D” Dehghanpisheh and Dan McInerney talk with Sandy about the nuanced challenges organizations face in managing LLMs while mitigating AI risks.Exploring the swift pace of innovation juxtaposed with the imperative of maintaining robust security measures, the trio examines the critical need for organizations to adapt their security posture management to include considerations for AI usage.
2024-02-1541 minThe MLSecOps PodcastSecure AI Implementation and GovernanceSend us a textIn this episode of The MLSecOps Podcast, Nick James, CEO of WhitegloveAI dives in with show host, Chris King, Head of Product at Protect AI, to offer enlightening insights surrounding:- AI Governance- ISO - International Organization for Standardization ISO/IEC 42001:2023-Information Technology, Artificial Intelligence Management System- Continuous improvement for AI securityThanks for listening! Find more episodes and transcripts at https://bit.ly/MLSecOpsPodcast.Additional MLSecOps and AI Security tools and resources to check out:Protect AI Radar (https://bit.ly...
2024-02-1338 minThe MLSecOps PodcastRisk Management and Enhanced Security Practices for AI SystemsSend us a textIn this episode of The MLSecOps Podcast, VP Security and Field CISO of Databricks, Omar Khawaja, joins the CISO of Protect AI, Diana Kelley. Together, Diana and Omar discuss a new framework for understanding AI risks, fostering a security-minded culture around AI, building the MLSecOps dream team, and some of the challenges that Chief Information Security Officers (CISOs) and other business leaders face when assessing the risk to their AI/ML systems.Get the scoop on Databricks’ new AI Security Framework on The MLSecOps Podcast. To learn more about the framework, co...
2024-02-0638 minThe MLSecOps PodcastEvaluating Real-World Adversarial ML Attack Risks and Effective Management: Robustness vs Non-ML MitigationsSend us a textIn this episode, co-hosts Badar Ahmed and Daryan Dehghanpisheh are joined by Drew Farris (Principal, Booz Allen Hamilton) and Edward Raff (Chief Scientist, Booz Allen Hamilton) to discuss themes from their paper, "You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks," co-authored with Michael Benaroch.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated...
2023-11-2841 minThe MLSecOps PodcastFrom Risk to Responsibility: Violet Teaming in AI; With Guest: Alexander TitusSend us a textIn this episode, the founder and CEO of The In Vivo Group, Alexander Titus, joins show hosts Diana Kelley and Daryan Dehghanpisheh to discuss themes from his forward-thinking paper, "The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forward," authored with Adam H. Russell.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models ...
2023-10-2443 minThe MLSecOps PodcastCybersecurity of Tomorrow: Exploring the Future of Security and Governance for AI Systems; With Guest: Martin Stanley, CISSPSend us a text*This episode is also available in video format! Click to watch the full YouTube video.*Welcome to Season 2 of The MLSecOps Podcast! In this episode, we joined Strategic Technology Branch Chief, Martin Stanley, CISSP, from the Cybersecurity and Infrastructure Security Agency (CISA), to celebrate 20 years of Cybersecurity Awareness Month, as well as hear his expert and thoughtful insights about CISA initiatives, partnering with the National Institute of Standards and Technology (NIST) to promote the adoption of their AI Risk Management Framework, AI security and governance, and much more. We ar...
2023-10-1839 minThe MLSecOps PodcastAI/ML Security in Retrospect: Insights from Season 1 of The MLSecOps Podcast (Part 2)Send us a text*This episode is also available in video format! Click to watch the full YouTube video.*Welcome back, everyone, to The MLSecOps Podcast. We’re thrilled to have you with us for Part 2 of our two-part season finale, as we wrap up Season 1 and look forward to an exciting and revamped Season 2. In this two-part season recap, we’ve been revisiting some of the most captivating discussions from our first season, offering an overview on essential topics related to AI and machine learning security. Part 1 of this serie...
2023-09-2242 minThe MLSecOps PodcastAI/ML Security in Retrospect: Insights from Season 1 of The MLSecOps Podcast (Part 1)Send us a text*This episode is also available in video format! Click to watch the full YouTube video.*Welcome to the final episode of the first season of The MLSecOps Podcast, brought to you by the team at Protect AI.In this two-part episode, we’ll be taking a look back at some favorite highlights from the season where we dove deep into machine learning security operations. In this first part, we’ll be revisiting clips related to things like adversarial machine learning; how malicious actors can use AI to fool machine lear...
2023-09-1937 minThe MLSecOps PodcastA Holistic Approach to Understanding the AI Lifecycle and Securing ML Systems: Protecting AI Through People, Processes & Technology; With Guest: Rob van der VeerSend us a textJoining us for the first time as a guest host is Protect AI’s CEO and founder, Ian Swanson. Ian is joined this week by Rob van der Veer, a pioneer in AI and security. Rob gave a presentation at Global AppSec Dublin earlier this year called “Attacking and Protecting Artificial Intelligence” which was a large inspiration for this episode. In it, Rob talks about the lack of security considerations and processes in AI production systems compared to traditional software development, and the unique challenges and particularities of building security into AI and machin...
2023-09-0529 minThe MLSecOps PodcastML Model Fairness: Measuring and Mitigating Algorithmic Disparities; With Guest: Nick SchmidtSend us a textThis week we’re talking about the role of fairness in AI/ML. It is becoming increasingly apparent that incorporating fairness into our AI systems and machine learning models while mitigating bias and potential harms is a critical challenge. Not only that, it’s a challenge that demands a collective effort to ensure the responsible, secure, and equitable development of AI and machine learning systems.But what does this actually mean in practice? To find out, we spoke with Nick Schmidt, the Chief Technology and Innovation Officer at SolasAI. In this week...
2023-08-1835 minThe MLSecOps PodcastExploring AI/ML Security Risks: At Black Hat USA 2023 with Protect AISend us a textWatch the video for this episode at: https://mlsecops.com/podcast/exploring-ai/ml-security-risks-at-black-hat-usa-2023 This episode of The MLSecOps Podcast features expert security leaders who sat down at Black Hat USA 2023 last week with team members from Protect AI to talk about various facets of AI and machine learning security:- What is the overall state of the AI/ML security realm at this time?- What is currently the largest threat to AI and machine learning security?- What is the most important thing we need to...
2023-08-1735 min
Software Snack Bites#17 - Ian Swanson (CEO of Protect AI) & Emilio Escobar (CISO of Datadog) - Deep Dive into ML & AI Security, Emerging Attack Surfaces, MLSecOps, and LLMs Use in Adversarial and Defensive SecurityIan Swanson is the Co-Founder & CEO of Protect AI which secures AI and ML systems. He was previously the Co-Founder & CEO of Datascience.com which sold to Oracle, where he was VP AI & ML and then was Worldwide Leader of AI & ML at AWS before starting Protect AI. I’m also joined by Emilio Escobar, CISO of Datadog as a special co-host! In this episode, we dive into what is ML & AI Security, the different types of attacks that hackers are using, and how teams can best secure their systems. We also talk about MLSecOps and why an ML BO...
2023-08-1659 minThe MLSecOps PodcastEverything You Need to Know About Hacker Summer Camp 2023Send us a textWelcome back to The MLSecOps Podcast for this week's episode, “Everything You Need to Know About Hacker Summer Camp 2023.” This week, our show is hosted by Protect AI's Chief Information Security Officer, Diana Kelley, and Diana talks with two more longtime security experts, Chloé Messdaghi and Dan McInerney, about all things related to what the security research community fondly refers to as Hacker Summer Camp. The group discusses various events held throughout the course of this exciting week in Las Vegas, including what to expect at Black Hat [US...
2023-08-0338 minThe MLSecOps PodcastPrivacy Engineering: Safeguarding AI & ML Systems in a Data-Driven Era; With Guest Katharine JarmulSend us a textWelcome to The MLSecOps Podcast, where we dive deep into the world of machine learning security operations. In this episode, we talk with the renowned Katharine Jarmul. Katharine is a Principal Data Scientist at Thoughtworks, and the author of the popular new book, Practical Data Privacy. Katharine also writes a blog titled, Probably Private, where she writes about data privacy, data security, and the intersection of data science and machine learning. We cover a lot of ground in this conversation; from the more general data privacy and security ri...
2023-07-1246 minThe MLSecOps PodcastThe Intersection of MLSecOps and DataPrepOps; With Guest: Jennifer Prendki, PhDSend us a textOn this week’s episode from The MLSecOps Podcast, we have the pleasure of hearing from Dr. Jennifer Prendki, founder and CEO of Alectio - The DataPrepOps Company. Alectio’s name comes from a blend of the acronym “AL,” standing for Active Learning, and the Latin term for the word “selection,” which is “lectio.”In this episode, Dr. Prendki defines DataPrepOps for us and describes its contrasts to MLOps, along with how DataPrepOps intersects with MLSecOps best practices. She also discusses data quality, security risks in data science, and the role that data curati...
2023-06-2134 minThe MLSecOps PodcastThe Evolved Adversarial ML Landscape; With Guest: Apostol Vassilev, NISTSend us a textIn this episode, we explore the National Institute of Standards and Technology (NIST) white paper, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. The report is co-authored by our guest for this conversation; Apostol Vassilev, NIST Research Team Supervisor. Apostol provides insights into the motivations behind this initiative and the collaborative research methodology employed by the NIST team.Apostol shares with us that this taxonomy and terminology report is part of the Trustworthy & Responsible AI Resource Center that NIST is developing. Additional tools in t...
2023-06-1430 minThe MLSecOps PodcastNavigating the Challenges of LLMs: Guardrails AI to the Rescue; With Guest: Shreya RajpalSend us a textIn “Navigating the Challenges of LLMs: Guardrails to the Rescue,” Protect AI Co-Founders, Daryan Dehghanpisheh and Badar Ahmed, interview the creator of Guardrails AI, Shreya Rajpal.Guardrails AI is an open source package that allows users to add structure, type, and quality guarantees to the outputs of large language models (LLMs). In this highly technical discussion, the group digs into Shreya’s inspiration for starting the Guardrails project, the challenges of building a deterministic “guardrail” system on top of probabilistic large language models, and the challenges in general (both technical a...
2023-06-0739 min
The Secure DeveloperThe Five Pillars Of MLSecOps With Ian SwansonAt the rate at which AI is infiltrating operations around the globe, AI regulation and security is becoming an increasingly pressing topic. As external regulations are put in place, it’s important to ensure that your internal compliance measures are up to scratch and your systems are safe. Joining us today to discuss the security of ML systems and AI applications is Ian Swanson, the Co-Founder and CEO of Protect AI. In this episode, Ian breaks down the five pillars of ML SecOps: supply chain vulnerabilities, model provenance, GRC (governance, risk, and compliance), trusted AI, and adversarial machine learning. We...
2023-06-051h 00The MLSecOps PodcastIndirect Prompt Injections and Threat Modeling of LLM Applications; With Guest: Kai GreshakeSend us a textThis talk makes it increasingly clear. The time for machine learning security operations - MLSecOps - is now. In “Indirect Prompt Injections and Threat Modeling of LLM Applications,” (transcript here -> https://bit.ly/45DYMAG) we dive deep into the world of large language model (LLM) attacks and security. Our conversation with esteemed cyber security engineer and researcher, Kai Greshake, centers around the concept of indirect prompt injections, a novel adversarial attack and vulnerability in LLM-integrated applications, which Kai has explored extensively. Our host, Daryan Dehghanpisheh, is joined by specia...
2023-05-2436 minThe MLSecOps PodcastResponsible AI: Defining, Implementing, and Navigating the Future; With Guest: Diya WynnSend us a textIn this episode of The MLSecOps Podcast, Diya Wynn, Sr. Practice Manager in Responsible AI in the Machine Learning Solutions Lab at Amazon Web Services shares her background and the motivations that led her to pursue a career in Responsible AI. Diya shares her passion for work related to diversity, equity, and inclusion (DEI), and how Responsible AI offers a unique opportunity to merge her passion for DEI with what her core focus has always been: technology. She explores the definition of Responsible AI as an operating approach focused on minimizing un...
2023-05-1733 minThe MLSecOps PodcastML Security: AI Incident Response Plans and Enterprise Risk Culture; With Guest: Patrick HallSend us a textIn this episode of The MLSecOps Podcast, Patrick Hall, co-founder of BNH.AI and author of "Machine Learning for High-Risk Applications," discusses the importance of “responsible AI” implementation and risk management. He also shares real-world examples of incidents resulting from the lack of proper AI and machine learning risk management; supporting the need for governance, security, and auditability from an MLSecOps perspective.This episode also touches on the culture items and capabilities organizations need to build to have a more responsible AI implementation, the key technical components of AI risk management, and...
2023-05-1038 minThe MLSecOps PodcastAI Audits: Uncovering Risks in ML Systems; With Guest: Shea Brown, PhDSend us a textShea Brown, PhD explores with us the “W’s” and security practices related to AI and algorithm audits. What is included in an AI audit? Who is requesting AI audits and, conversely, who isn’t requesting them but should be? When should organizations request a third party audit of their AI/ML systems and machine learning algorithms?Why should they do so? What are some organizational risks and potential public harms that could result from not auditing AI/ML systems? What are some next steps t...
2023-05-0341 minThe MLSecOps PodcastMLSecOps: Red Teaming, Threat Modeling, and Attack Methods of AI Apps; With Guest: Johann RehbergerSend us a textJohann Rehberger is an entrepreneur and Red Team Director at Electronic Arts. His career experience includes time with Microsoft and Uber, and he is the author of “Cybersecurity Attacks – Red Team Strategies: A practical guide to building a penetration testing program having homefield advantage” and the popular blog, EmbraceTheRed.com. In this episode, Johann offers insights about how to apply a traditional security engineering mindset and red team approach to analyzing the AI/ML attack surface. We also discuss ways that organizations can adapt their traditional security postures to address the unique ch...
2023-04-2640 minThe MLSecOps PodcastMITRE ATLAS: Defining the ML System Attack Chain and Need for MLSecOps; With Guest: Christina Liaghati, PhDSend us a textThis week The MLSecOps Podcast talks with Dr. Christina Liaghati, AI Strategy Execution & Operations Manager of the AI & Autonomy Innovation Center at MITRE.Chris King, Head of Product at Protect AI, guest-hosts with regular co-host D Dehghanpisheh this week. D and Chris discuss various AI and machine learning security topics with Dr. Liaghati, including the contrasts between the MITRE ATT&CK matrices focused on traditional cybersecurity, and the newer AI-focused MITRE ATLAS matrix. The group also dives into consideration of new classifications of ML attacks re...
2023-04-1939 minThe MLSecOps PodcastUnpacking AI Bias: Impact, Detection, Prevention, and Policy; With Guest: Dr. Cari Miller, MBA, FHCASend us a textWhat is AI bias and how does it impact both organizations and individual members of society? How does one detect if they’ve been impacted by AI bias? What can be done to prevent or mitigate it? Can AI/ML systems be audited for bias and, if so, how?The MLSecOps Podcast explores these questions and more with guest Cari Miller, Founder of the Center for Inclusive Change and member of the For Humanity Board of Directors.This week’s episode delves into the controversial topics of Trusted and Ethi...
2023-04-1139 minThe MLSecOps PodcastJust How Practical Are Data Poisoning Attacks? With Guest: Dr. Florian TramèrSend us a textETH Zürich's Assistant Professor of Computer Science, Dr. Florian Tramèr, joins us to talk about data poisoning attacks and the intersection of Adversarial ML and MLSecOps (machine learning security operations).Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LL...
2023-03-2947 minThe MLSecOps PodcastA Closer Look at "Adversarial Robustness for Machine Learning" With Guest: Pin-Yu ChenSend us a textIn this episode of The MLSecOps podcast, the co-hosts interview Pin-Yu Chen, Principal Research Scientist at IBM Research, about his book co-authored with Cho-Jui Hsieh, "Adversarial Robustness for Machine Learning." Chen explores the vulnerabilities of machine learning (ML) models to adversarial attacks and provides examples of how to enhance their robustness. The discussion delves into the difference between Trustworthy AI and Trustworthy ML, as well as the concept of LLM practical attacks, which take into account the practical constraints of an attacker. Chen also discusses security measures that can be taken to protect...
2023-03-2938 minThe MLSecOps PodcastA Closer Look at "Securing AIML Systems in the Age of Information Warfare" With Guest: Disesdi Susanna CoxSend us a textSecurity researcher, AI/ML architect, & former political operative, Disesdi Susanna Cox, talks with us about her research, some of which can be accessed via her website: anglesofattack.io.Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com. Additional tools and resources to check out:Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open So...
2023-03-2930 min
MLOps.communityEnterprise Security and Governance MLOps // Diego Oppenheimer // MLOps Coffee Sessions #45Coffee Sessions #45 with Diego Oppenheimer of Algorithmia, Enterprise Security and Governance MLOps.Join the Community: https://go.mlops.community/YTJoinInGet the newsletter: https://go.mlops.community/YTNewsletter// AbstractMLOps in the enterprise is difficult due to security and compliance. In this MLOps Coffee Session, the CEO of Algorithmia, Diego, talks to us about how we can better approach MLOps within the enterprise. This is an introduction to essential principles of security in MLOps and why it is crucial to be aware of security best practices as an ML professional.// BioDiego Oppenheimer is co-founder and CEO of Algorithmia. Previously, he designed, managed, and shipped some of Microsoft’s most used data analysis products, including Excel, Power Pivot, SQL Server, and Power BI. He holds a Bachelor’s degree in Information Systems and a Master’s degree in Business Intelligence and Data Analytics from Carnegie Mellon University.--------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup...
2021-07-0253 min