When a company gets big enough, there is so much data to be processed that an entire data engineering team becomes responsible for managing this data and making it available to other teams. Airbnb is one such company.
Max Beauchemin works on the data engineering team at Airbnb, where he creates infrastructure and tooling for managing data. In this episode of Software Engineering Daily, we talk about AIrflow, a workflow scheduler that assists in job processing. If you don’t know what a workflow is, or a job, we will explain that in this episode. Max and I also talk about Panoramix, a data slicing and visualization tool that helps data scientists and business analysts understand large volumes of data. Max will also be presenting at Strata + Hadoop World in San Jose. We’re partnering with O’Reilly to support this conference – if you want to go to Strata, you can save 20% off a ticket with our code PCSED.
