Shows
Prodity: Product by DesignLessons in Data Engineering: Scaling, AI, and Open Source with Sandy RyzaIn this episode of Product by Design, Kyle chats with Sandy Ryza, lead engineer on the Dagster project, author, and thought leader in data engineering. Sandy shares his journey through the world of data—from building big data tools at Cloudera to working as a data scientist, product manager, and engineer—and how those experiences led him to help create Dagster, an open-source data orchestration platform.We discuss:The evolution of data engineering and the growing complexity of modern data pipelines.The role of AI and unstructured data in shaping the future of data platforms.How orga...
2025-02-0746 min
Adventures in Machine LearningMaintaining Backward Compatibility in Software Projects: Strategies from Industry Experts - ML 164Today, host Michael Berk and Ben Wilson dive deep into the multifaceted world of software engineering and data science with their insightful guest, Sandy Ryza a lead engineer from Dagster Labs. In this episode, they explore a range of intriguing topics, from the impact of the broken windows theory on code quality to the delicate balance of maintaining backward compatibility in evolving software projects. Sandy talks about the challenges and learnings in transitioning from data science back to software engineering, including dependency management and designing for diverse use cases. They touch on the importance of clear naming conventions, tooling, a...
2024-08-2959 min
The Developers' BakeryDagster with Sandy RyzaToday is time to talk about Data Pipelines and Data Engineering. I’m really excited to have on stage Sandy Ryza, Lead Engineer of Dagster.
If you’re a software engineer and you’re afraid of dealing with data pipelines, fear no more! Sandy is on a mission to make data pipelines easier to handle for software engineers. Join us in this episode to learn more about Dagster, and how it can make it easier for you to build and manage your data assets.
Enjoy the show 👨🍳
Show Notes
00.00 Intro
00.46 Episode Start
01.09 Sandy’s Introd...
2024-07-3140 min
Podcast Ruined by a Software EngineerBuilding Data Tooling with Sandy Ryza | Ep. 43Sandy Ryza is the lead engineer at Dagster Labs, the group behind the popular developer tool Dagster which allows data practitioners to see all their data assets across their data pipelines.Dive into topics such as going from software engineering to data science / engineering (and back), what performance issues do data applications encounter, how to maintain an open-source project with thousands of developers and much more. Hosted by Perry Tiu.Episode Links:• Dagster: https://dagster.io• Sandy's Twitter: https://twitter.com/s_ryz• Sandy's LinkedIn: https://www.linkedin.com/in/sandyryza—
2024-06-211h 12
Value Driven Data Science: Boost your impact. Earn what you’re worth. Rewrite your career algorithm.Episode 39: The Impact of Data Science on Data OrchestrationOne of the big promises of data science is its ability to combine multiple disparate datasets to produce value-creating insights. But this is only possible if you can get all those disparate datasets together, in the one location, to begin with. The has led to the rise of the data engineer and the data orchestration platform.In this episode, Sandy Ryza joins Dr Genevieve Hayes to discuss the impact of the data scientist on the creation of the next generation of data orchestration tools.Guest BioSandy Ryza is a data...
2024-06-1939 min
SaaS FuelSandy Ryza - Data Fusion: Minds on Orchestration and AI InnovationsIn today's SaaS Fuel Expert Series episode, we have the privilege of diving deep into the realms of data fusion, where minds meet to orchestrate and innovate with AI at the helm. Join Jeff Mains, alongside our esteemed guest, Sandy Ryza, the ingenious lead engineer at Dagster Labs.Together, we embark on an exhilarating journey, exploring the intricate dance of data orchestration within modern organizations. From reconciling the myriad versions of truth to sculpting flawless data pipelines, Sandy shares his wealth of insights, unveiling the challenges, solutions, and the compelling future of data management.Tune...
2024-04-1148 min
The Data Stack Show171: Machine Learning Pipelines Are Still Data Pipelines with Sandy Ryza of DagsterHighlights from this week’s conversation include:The role of an orchestrator in the lifecycle of data (1:34)Relevance of orchestration in data pipelines (00:02:45)Changes around data ops and MLOps (3:37)Data Cleaning (11:42)Overview of Dagster (13:50)Assets vs Tasks in Data Pipeline (19:15)Building a Data Pipeline with Dexter (25:40)Difference between Data Asset and Materialized Dataset (28:28)Defining Lineage and Data Assets in Dagster (29:32)The boundaries of software and organizational structures (37:25)The benefits of a unified orchestration framework (39:56)Orchestration in the development phase (45:29)The emergence of analytics engineer role (51:53)Fluidity in data pipeline and infrastructure roles (52:40)The Data Stack Show is a...
2024-01-0355 min
The Data Stack ShowThe PRQL: Does Machine Learning Need Its Own Orchestrator? Featuring Sandy Ryza of DagsterIn this bonus episode, Eric and Kostas preview their upcoming conversation with Sandy Ryza of Dagster. Hosted by Simplecast, an AdsWizz company. See https://pcm.adswizz.com
for information about our collection and use of personal data for
advertising.
2024-01-0203 min
AI Engineering PodcastBuild More Reliable Machine Learning Systems With The Dagster Orchestration EngineSummaryBuilding a machine learning model one time can be done in an ad-hoc manner, but if you ever want to update it and serve it in production you need a way of repeating a complex sequence of operations. Dagster is an orchestration engine that understands the data that it is manipulating so that you can move beyond coarse task-based representations of your dependencies. In this episode Sandy Ryza explains how his background in machine learning has informed his work on the Dagster project and the foundational principles that it is built on to allow for collaboration across data...
2022-12-0245 min