LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian.
// Abstract
Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event!
// MLOps Jobs board
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related LinksLLM in Production Survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform
Virtual LLMs in Production Conference registration:
https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13
Chinchilla papers:
https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556
--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/
Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/
Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/
Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/
Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/
Timestamps:
[00:00] Round table success to Virtual LLM in Production Conference on April 13th!
[00:18] Register for the Virtual LLM in Production Conference now!
[00:44] LLM in Production survey
[01:40] Lightning round of introduction of speakers
[04:34] Large Language Models definition
[09:17] What do we consider large?
[10:35] Thought process in use cases production
[14:30] LLM open source huge movements
[16:50] Problems with qualifications
[19:25] Production use cases frameworks directions
[25:25] Open-source language models tokenizer
[26:25] Language models democratization
[29:25] Three categories for LLMs in Production
[31:22] Latency at 2 levels
[33:27] Defining production
[34:57] Hitting the latency problems
[38:20] Fundamental latency barrier
[40:39] Latency use case requirement
[44:25] Costs and the use cases
[48:12] Product management involvement in costing
[49:38] LLMs Hallucination definition
[52:05] Building deterministic systems trust
[55:21] Wrap up
