Episode Summary
In this episode, Maribel Lopez interviews Kate Soule, Director of Technical Product Management for IBM's Granite products. They discuss IBM's third-generation AI models, their focus on efficiency and enterprise readiness, and the latest advancements including vision capabilities and reasoning features.
Guest
Kate Soule - Director of Technical Product Management for IBM's Granite products
Key Topics & Timestamps
00:04 - Introduction
- Maribel introduces the show and Kate Soule
- Brief overview of IBM Granite as fit-for-purpose, open-source enterprise AI models
00:48 - What is IBM Granite?
- Designed as core building blocks for enterprises building with generative AI
- Focus on efficiency with smaller model sizes
- Monthly innovation updates to keep pace with rapidly evolving field
02:19 - Understanding AI Reasoning
- Explanation of reasoning capabilities in AI models
- How allowing models to generate more text at inference time can improve performance
- Cost/benefit tradeoffs of reasoning features
03:13 - Enterprise AI Model Selection Criteria
- Moving beyond "one model to rule them all" thinking
- Importance of fit-for-purpose models
- Why smaller models can be customized more easily
- Trust and transparency considerations
05:38 - AI Governance and Safety
- How to evaluate models for governance requirements
- Safety evaluations and benchmarks as table stakes
- Systems-based approach to safety with guardrails
- IBM's Granite Guardian and protection mechanisms
08:55 - Benefits of Smaller Models
- Why size matters: cost, latency, and customization advantages
- Smaller models are easier to customize and require less computing power
- IBM's transparent approach to training data
10:13 - Future of AI Evaluation
- Performance per cost becoming the key evaluation metric
- The growing importance of flexibility in model selection
- How the "efficient frontier" between cost and performance will differentiate providers
12:41 - IBM's Vision Models
- IBM's pragmatic enterprise focus for multimodal capabilities
- Vision understanding (image in, text out) for practical business use cases
- Specialization for documents, charts, and dashboards
- Delivering powerful capabilities in only 2 billion parameters
15:25 - Understanding Model Size Context
- Evolution from millions to billions of parameters
- Practical considerations of deploying different-sized models
- Finding the right cost-benefit trade-off for specific use cases
