Stream processing engines are often the component of data engineering stacks with the most variety, with the big data ecosystem offering several compelling options. Assessing the differences between projects like Flink, Storm, and Spark Streaming is difficult without an agreed upon set of metrics to compare them on. Fortunately, the Yahoo engineering team created a set of benchmarks to do exactly this. On today’s episode, Jeff and Bobby compare streaming frameworks through the performance on several of Yahoo’s benchmarking tests.
Bobby Evans is an architect at Yahoo working on streaming frameworks, primarily on Apache Storm. He is also the Apache Storm Incubating PMC at The Apache Software Foundation.
