AI Evaluation Methods

Description

How do you know if an AI system is trustworthy, compliant, ethical, and fit for purpose?

In this episode of the FIT4Privacy Podcast, Punit Bhatia is joined by Stella Liu, an AI evaluation expert and founder of AI Evals & Analytics, to unpack one of the most practical—anud overlooked—challenges in AI today: how to evaluate AI systems before and after deployment.

KEY MOMENTS

02:09 —AI Definition

03:02 —AI Evaluations

10:31 — Why AI Testing Is Hard

14:06 — Evals Plus Analytics
18:15 —Synthetic Data

23:47 —Protecting Privacy Ethical
29:05 — AI Evals as a Company

29:52 —How to reach Stella Liu

Stella explains why AI behaves differently from traditional software, why testing code alone is no longer enough, and how AI evaluations (AI evals) help organizations assess real‑world behavior, risk, and performance. From evaluation‑driven development to continuous monitoring in production, the conversation explores how teams can move beyond guesswork and hype toward repeatable, measurable AI governance.

⸻

ABOUT THE GUEST

Stella Liu is the Co-founder of AI Evals & Analytics (Maven), where she created the AI Evals & Analytics Playbook and teaches top-rated courses on LLM evaluation, monitoring, and product alignment. She’s also the Head of AI Applied Science at ASU, leading evals and analytics across university-wide AI products and building higher-ed’s first formal AI evaluation framework, and she previously led data science at Shopify and Carvana with 12+ years shipping large-scale ML systems.

ABOUT THE HOST

Punit Bhatia is one of the leading privacy experts who works independently and has worked with professionals in over 30 countries. Punit works with business and privacy leaders to create an organization culture with high privacy awareness and compliance as a business priority. Selectively, Punit is open to mentor and coach privacy professionals.

⸻

Resources & Links

Guest Links
Stella Lui

• Website: https://maven.com/
• LinkedIn: https://www.linkedin.com/in/wenxingl/

Grow Skills (Privacy Courses & Insights)

• Courses: https://growskills.store/courses/

• Insights: https://growskills.store/insights/

• Website: https://growskills.store/

FIT4Privacy

• Website: https://www.fit4privacy.com

• Podcast: https://www.fit4privacy.com/podcast

• Blog: https://www.fit4privacy.com/blog