Generative AI is everywhere, but how do we monitor and observe it? OpenTelemetry has been a prominent tool and standard for observability, and recently the OTel community has been aiming to expand its scope and cover GenAI workloads with semantic conventions and tools.
In this episode, Horovits is joined by Nir Gazit, creator of the OpenLLMetry project, and member of the OpenTelemetry Generative AI SIG. We discuss new semantic conventions, tracing prompts and model behavior, the OpenLLMetry project’s journey, and what observability even means for modern AI systems.
Nir Gazit is the CEO and co-founder of Traceloop, and brings a wealth of data and AI experience, with previous experience leading AI teams at Google and serving as the Chief Architect at Fiverr.
You can read the recap post: https://medium.com/p/81b9cea6a771/
Show Notes:
00:00 - intro
04:09 - what is observability for AI
18:07 - AI observability differences from traditional observability
25:22 - OpenLLMetry intro
41:21 - OpenLLMetry latest updates and roadmap
47:00 - OpenTelemetry GenAI Semantic Conventions SIG
56:03 - KubeCon updates: CrossPlane, Knative, Dragonfly, in-toto reached CNCF graduation
1:00:08 - outro
Resources:
OpenTelemetry Generative AI Observability SIG: https://github.com/open-telemetry/community/blob/1c71595874e5d125ca92ec3b0e948c4325161c8a/projects/llm-semconv.md
https://github.com/traceloop/openllmetry
https://github.com/traceloop/hub
https://github.com/traceloop/opentelemetry-mcp-server
Socials:
BlueSky: https://bsky.app/profile/openobservability.bsky.social
Twitter: https://twitter.com/OpenObserv
LinkedIn: https://www.linkedin.com/company/openobservability/
YouTube: https://www.youtube.com/@openobservabilitytalks
Dotan Horovits
============
Twitter:@horovits
LinkedIn:www.linkedin.com/in/horovits
Mastodon: @horovits@fosstodon
BlueSky: @horovits.bsky.social
Nir Gazit
========
Twitter: https://x.com/nir_ga
LinkedIn: https://www.linkedin.com/in/nirga/
OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.
