Shows
Postgres FMMulti-tenant optionsNikolay and Michael are joined by Gwen Shapira to discuss multi-tenant architectures — the high level options, the pros and cons of each, and how they're trying to help with Nile. Here are some links to things they mentioned:Gwen Shapira https://postgres.fm/people/gwen-shapiraNile https://www.thenile.devSaaS Tenant Isolation Strategies (AWS whitepaper) https://docs.aws.amazon.com/whitepapers/latest/saas-tenant-isolation-strategies/saas-tenant-isolation-strategies.html Row Level Security https://www.postgresql.org/docs/current/ddl-rowsecurity.htmlCitus https://github.com/citusdata/citusPostgres.AI Bot https://postgres.ai/blog/20240127-postgres-ai-bot RLS Performance and Best Practices https://supabase.com/docs/guide...2025-06-2050 min
The Joe Reis ShowGwen Shapira - Multi-Tenant Databases Done RightMulti-tenancy in databases is very difficult to pull off at scale. Gwen Shapira and I chat about multi-tenant databases at Nile (and elsewhere), AI, RAG, and much more.2024-12-0451 min
The GeekNarratorDatabase Internals - NileDB Postgres re-engineered for multitenant appsDatabase Internals - NileDB: Postgres Re-engineered for Multitenant Apps with Gwen Shapira
Join us in this episode as we dive deep into the intricacies of NileDB, a groundbreaking database re-engineered for multi-tenant applications. Our special guest, Gwen Shapira, co-founder of NileDB and a notable figure in the database field, shares her insights and technical know-how on solving the common challenges faced by multitenant SaaS applications. From the benefits of using Postgres as the underlying database to the unique tenant isolation features of NileDB, we cover it all. Don't miss out on learning about AI native capabilities, handling schema migrations...2024-11-071h 06
What's New In DataFrom Apache Kafka to PostgreSQL, PostgreSQL maturity and extensions, and building on PostgreSQL with Gwen Shapira (CPO at Nile)What does it take to go from leading Kafka development at Confluent to becoming a key figure in the PostgreSQL world? Join us as we talk with Gwen Shapira, co-founder and chief product officer at Nile, about her transition from cloud-native technologies to the vibrant PostgreSQL community. Gwen shares her journey, including the shift from conferences like O'Reilly Strata to PostgresConf and JavaScript events, and how the Postgres community is evolving with tools like Discord that keep it both grounded and dynamic.We dive into the latest developments in PostgreSQL, like hypothetical indexes that enable performance tuning...2024-10-2550 min
TractableNile Co-Founder Gwen Shapira: The Tenet of Multi-TenancyIn this episode of Tractable, Orb CTO and Co-founder Kshitij Grover and Gwen Shapira, Co-founder of Nile, explored the cutting-edge world of multi-tenancy and serverless Postgres offerings for modern SaaS applications. This podcast episode delved into Nile's innovative approach to virtual tenant databases, the complexities of managing mixed data blocks, and the challenges of building a database offering tailored to diverse customer needs. Gwen uncovered insights into the decision-making process of building on Postgres and the relentless focus on reliability for ongoing operations. 2024-09-3041 min
Cloud CommuteHow to Build a Serverless Postgres? ft Gwen ShapiraIn this episode, our guest Gwen Shapira talks about her co-founded database startup Nile, a serverless Postgres database. She explains how they implemented the multi-tenant features, made sure that customers are probably isolated, how scalability works, and what the future holds. Gwen has years and years of experience using technologies as Oracle databases, PostgreSQL, Kafka, and many more.In this episode of Cloud Commute, Chris and Gwen discuss:Building Nile: A serverless Postgres for SaaS applicationsChallenges of scaling multi-tenant architecture in databasesKubernetes migration and infrastructure automationThe future of AI and vector databases in enhancing performance...2024-08-1628 min
Breaking ChangesHow to Lead Your Team to Success When Failure Seems Inevitable—with Gwen Shapira I PostmanIn this episode of Breaking Changes, Postman Head of Product-Observability Jean Yang interviews Gwen Shapira, co-founder and CPO of Nile, for an insightful discussion on engineering leadership and navigating transformations. Learn about her strategies for keeping up with trends and balancing long-term visions with user feedback, as well as the rewards of finding the right people for the right challenges. Discover Gwen's advice on offshore teams, cultural bridges, serverless tech, AI's impact on content, and Nile's exciting developments!
For more on Gwen Shapira, check out the following:
LinkedIn: https://www.linkedin.com/in/g...2024-07-2452 min
Happy Path Programming#95 21st Century Databases with Gwen ShapiraWe chat with Gwen Shapira, co-founder of Nile, about her journey to creating a virtualized, serverless Postgres database service. We also dive into the challenges with traditional data architectures and approaches like ORMs.
Discuss this episode: discord.gg/XVKD2uPKyF2024-03-101h 00
Kubernetes for Humans#005 - Kubernetes For Humans Podcast with Gwen Shapira (Nile)Gwen Shapira is the Co-Founding CPO at Nile. Previously a system architect at Confluent (employee no. 12!) helping customers achieve success with their Apache Kafka implementation. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. She specializes in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, an author of "Hadoop Application Architectures", and a frequent presenter at data-driven conferences. Gwen is also a committer on the Apache Kafka and Apache Sqoop projects.
Buy Gwen's books on Amazon: https://www...2023-09-2031 min
GOTO - The Brightest Minds in TechUnlocking the Power of Real-Time Analytics • Tim Berglund & Adi PolakThis interview was recorded for GOTO Unscripted.gotopia.techRead the full transcription of this interview hereTim Berglund - VP DevRel at StarTree & Author of "Gradle Beyond the Basics"Adi Polak - VP of Developer Experience at Treeverse & Contributing to lakeFS OSSRESOURCESTimtimberglund.comtwitter.com/tlberglundlinkedin.com/in/tlberglundAditwitter.com/AdiPolakinstagram.com/polak.codelinkedin.com/in/polak-adiTools & companiespinot.apache.orgtwitter.com/startreedatalinkedin...2023-06-0244 min
Tales at ScaleWho Really Needs Real-Time Data? with Gwen ShapiraThe term “real time” gets thrown around a lot, especially when it comes to data. Gwen Shapira, co-founder and CPO of Nile joins us to help define real-time data, discuss who needs it (and who probably doesn't) and how to not build yourself into a corner with your architecture. When you're able to harness the power of real-time insights, you can do incredible things from simplifying daily decision making to building trust with customers. Gwen discusses her approach to modern SaaS platforms and how to reach the ultimate goal of bringing the right data to the right person at the...2023-01-3137 min
SaaS for DevelopersReverse ETL - Why is it a big deal?Reverse ETL is a bit of a buzzword these days. But why?
In this video Gwen Shapira discusses the new capabilities that reverse ETL introduces to businesses, and especially B2B SaaS with product-lead growth strategy. She also discusses why reverse ETL is more challenging to implement than you may imagine and why stream processing platforms may be a key part of solving these challenges.
Reverse ETL is not for everyone - but after watching this video, you'll know why everyone is talking about it.2022-05-1021 min
SaaS for DevelopersIt is an API Economy and SaaS products can joinAPI Economy is booming and provides many services that make SaaS development easier. Adding great public APIs to your product is also easier than ever.
In this episode, Gwen Shapira, usually the host of the channel, shared everything she learned about APIs, API Economy, and building great public APIs in the last week.
Interested in more SaaS content? Join our Slack: https://saas-community.github.io/
Willing to help my product discovery efforts? https://forms.gle/k3Rai3af1BN6D8c2A
Great links I found in my API research:
Very...2022-01-1417 min
Streaming Audio: Apache Kafka® & Real-Time DataUsing Apache Kafka as Cloud-Native Data System ft. Gwen ShapiraWhat does cloud native mean, and what are some design considerations when implementing cloud-native data services? Gwen Shapira (Apache Kafka® Committer and Principal Engineer II, Confluent) addresses these questions in today’s episode. She shares her learnings by discussing a series of technical papers published by her team, which explains what they’ve done to expand Kafka’s cloud-native capabilities on Confluent Cloud. Gwen leads the Cloud-Native Kafka team, which focuses on developing new features to evolve Kafka to its next stage as a fully managed cloud data platform. Turning Kafka into a self-service platform is not entirely...2021-12-0733 min
The InfoQ PodcastEvent Driven Architectures of ScaleWes Reisz, Matthew Clark, Gwen Shapira, and Ian Thomas discuss the evolution of event-driven architectures over the decades, the advantages that EDA offers, and thoughts for the future.
Read a transcript of this interview: https://bit.ly/2Z3b1t5
Subscribe to our newsletters:
- The InfoQ weekly newsletter: www.infoq.com/news/InfoQ-Newsletter/
- The Software Architects’ Newsletter [monthly]: www.infoq.com/software-architects-newsletter/
Upcoming Virtual Events - events.infoq.com/
QCon London:
https://qconlondon.com/
- April 4-6, 2022 / London, UK
QCon Plus:
https://plus.qconferences.com/
- May 10-20. 2022
InfoQ Live:
https://live.infoq.com/
- Feb 22, 2022
- June 21, 2022
...2021-11-1932 min
Streaming Audio: Apache Kafka® & Real-Time DataEvent Streaming Trends and Predictions for 2021 ft. Gwen Shapira, Ben Stopford, and Michael NollComing out of a whirlwind year for the event streaming world, Tim Berglund sits down with Gwen Shapira (Engineering Leader, Confluent), Ben Stopford (Senior Director, Office of the CTO, Confluent), and Michael Noll (Principal Technologist, Office of the CTO, Confluent) to take a guess at what 2021 will bring. The experts share what they believe will happen for analytics, frameworks, multi-cloud services, stream processing, and other topics important to the event streaming space. These Apache Kafka® related predictions include the future of the Kafka cluster partitions and removing restrictions that users have found in the past, such as to...2021-01-0644 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #18: The Toughest Questions ft. Anna McDonaldIt’s the first work-from-home episode of Ask Confluent, where Gwen Shapira (Core Kafka Engineering Leader, Confluent) virtually sits down with Apache Kafka® expert Anna McDonald (Staff Technical Account Manager, Confluent) to answer questions from Twitter. Find out Anna’s favorite Kafka Improvement Proposal (KIP), which will start to use racially neutral terms in the Kafka community and in our code base, as well as answers to the following questions: If you could pick any one KIP from the backlog that hasn't yet been implemented and have it immediately available, which one would you pick?Are we able...2020-10-2233 min
Streaming Audio: Apache Kafka® & Real-Time DataBuilding an Apache Kafka Center of Excellence Within Your Organization ft. Neil BuesingNeil Buesing, an Apache Kafka® community stalwart at Object Partners, spends his days building things out of Kafka and helping others do the same. Today, he discusses the concept of a CoE (center of excellence), and how a CoE is integral to attain and sustain world-class performance, business value, and success in a business. Neil talks us through how to make a CoE successful, the importance of event streaming, how to better understand streaming technologies, and how to best utilize CoE for your needs. This includes evangelizing Kafka, building a Proof of Value (PoV) with team members, defining deliverables a...2020-10-1446 min
Streaming Audio: Apache Kafka® & Real-Time DataTop 6 Things to Know About Apache Kafka ft. Gwen ShapiraThis year, Confluent turns six! In honor of this milestone, we are taking a very special moment to celebrate with Gwen Shapira by highlighting the top six things everyone should know about Apache Kafka®:Clients have metricsBug fix releases/Kafka Improvement Proposals (KIPs)Idempotent producers and how they workKafka Connect is part of Kafka and Single Message Transforms (SMTs) are worth not missing out onCooperative rebalancing Generating sequence numbers and how Kafka changes the way you thinkListen as Tim and Gwen talk through the importance of Kafka Connect, cooperative rebalancing protocols, and the promise (and warning) th...2020-09-1547 min
Streaming Audio: Apache Kafka® & Real-Time Data5 Years of Event Streaming and Counting ft. Gwen Shapira, Ben Stopford, and Michael NollWith the explosion of real-time data, Apache Kafka and event stream processing (ESP) have grown in proliferation, with event streaming technology becoming the de facto technology transforming businesses across numerous verticals. Gwen Shapira (Engineering Leader, Confluent), Ben Stopford (Senior Director, OCTO, Confluent), and Michael Noll (Principal Technologist, Confluent) meet up to talk all about their last five years at Confluent and the changes they’ve seen in event streaming. They discuss what they were doing with Apache Kafka® before they arrived at Confluent, challenges in event streaming challenges that have arisen, and their favorite use cases. They then talk thr...2020-08-3148 min
Streaming Audio: Apache Kafka® & Real-Time DataBenchmarking Apache Kafka Latency at the 99th Percentile ft. Anna PovznerReal-time stock trades, GPS location, and website click tracking are just a few industries that heavily rely on Apache Kafka®'s real-time messaging and data delivery functions. As such, Kafka's latency is incredibly important.Anna Povzner (Software Engineer, Confluent) gives you the breakdown and everything you need to know when it comes to measuring latency. The five components of latency are produce time, publish time, commit time, catch-up time, and fetch time. When consumer pulling adds to latency, Anna shares some best practices to keep in mind for how to think about partitioning in conjunction wi...2020-07-0846 min
Streaming Audio: Apache Kafka® & Real-Time DataDistributed Systems Engineering with Apache Kafka ft. Gwen ShapiraAs an engineering leader managing a team, Gwen Shapira talks through the steps she took to get to Confluent and how she got started working with Apache Kafka®. She shares about what it's like being on the Project Management Committee (PMC) for the Apache Software Foundation as well as some of the responsibilities involved, such as choosing Kafka Improvement Proposals (KIPs), monitoring releases, and making contributions to the community. For Gwen, part of finding Kafka was her willingness to take risks, learn all types of code bases, and leave companies for a new technology that showed promise an...2020-03-0448 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #16: ksqlDB EditionVinoth Chandar has led various infrastructure projects at Uber and is one of the main drivers behind the ksqlDB project. In this episode hosted by Gwen Shapira (Engineering Manager, Cloud-Native Apache Kafka®), Vinoth and Gwen discuss what ksqlDB is, the kinds of applications that you can build with it, vulnerabilities, and various ksqlDB use cases. They also talk about what's currently the best version of Apache Kafka version for performance improvements that don’t cause breaking changes to existing Kafka configuration and functionality. EPISODE LINKSRead about ksqlDB on the blogLearn more about ksqlDBksqlDB Demo | The Even...2019-12-1230 min
Streaming Audio: Apache Kafka® & Real-Time DataImproving Fairness Through Connection Throttling in the Cloud with KIP-402 ft. Gwen ShapiraThe focus of KIP-402 is to improve fairness in how Apache Kafka® processes connections and how network threads pick up requests and new data. Gwen Shapira (Engineering Manager for Cloud-Native Kafka, Confluent) outlines the details of this KIP and her team’s efforts to make user-facing Kafka improvements. Halfway through the episode, Gwen shares how to send metadata and produce client messages. EPISODE LINKSKIP-402: Improve fairness in SocksetServer processorsJoin the Confluent Community SlackFully managed Apache Kafka as a service! Try free.2019-10-0948 min
Streaming Audio: Apache Kafka® & Real-Time DataScaling Apache Kafka with Todd PalinoTodd Palino, a senior SRE at LinkedIn, talks about the start of Apache Kafka® at LinkedIn, what learning to use Kafka was like, how Kafka has changed, and what he and others in the community hope for in the future of Kafka. If you’re curious about life as an SRE, Todd shares the details on that too, and goes into how Kafka is used at LinkedIn, as well as several wins and challenges over the years with the product. EPISODE LINKSKafka: The Definitive Guide by Neha Narkhede, Gwen Shapira & Todd PalinoURP? Excuse You! The Thre...2019-09-2546 min
Streaming Audio: Apache Kafka® & Real-Time DataUnderstand What’s Flying Above You with Kafka Streams ft. Neil BuesingNeil Buesing (Director of Real-Time Data, Object Partners) discusses what a day in his life looks like and how Kafka Streams helps analyze flight data.EPISODE LINKSUsing Location Data to Showcase Keys, Windows, and Joins in Kafka Streams DSL and KSQLKafka: The Definitive Guide by Neha Narkhede, Gwen Shapira & Todd PalinoRead the Confluent blogJoin the Confluent Community Slack2019-09-2313 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #15: Attack of the Zombie ControllerGwen Shapira (Core Kafka Software Engineer, Confluent) sits down to answer the questions you've had about event streaming, Apache Kafka®, Confluent, and everything in between. This includes creating tables in nested JSON topics, how to balance ordering, latency and reliability, building event-based systems, and how to navigate the tricky endOffsets API. She talks about the hardships of fencing Zombie requests, some of the talks given at previous Kafka Summits, and an important question from Ask Confluent #3. EPISODE LINKSKIP-91: Provide Intuitive User Timeouts in The ProducerKIP-79: ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the ne...2019-09-0422 min
Streaming Audio: Apache Kafka® & Real-Time DataRolling Kafka Upgrades and Confluent Cloud ft. Gwen ShapiraIf you operate a Kafka cluster, hopefully you upgrade your brokers occasionally. Each release of Apache Kafka® includes detailed documentation that describes a tested procedure for doing a rolling upgrade of your cluster. Couldn’t be easier, right? Well, what if you have to do it with hundreds or thousands of brokers, such as you’d have to do if you were running Confluent Cloud? Today, Gwen Shapira shares some of the lessons she’s learned doing just that.EPISODE LINKSFully managed Apache Kafka as a service! Try free.2019-06-2542 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #10: Cooperative Rebalances for Kafka Connect ft. Konstantine KarantasisWant to know how Kafka Connect distributes tasks to workers? Always thought Connect rebalances could be improved? In this episode of Ask Confluent, Gwen Shapira speaks with Konstantine Karantasis, software engineer at Confluent, about the latest improvements to Kafka Connect and how to run the Confluent CLI on Windows.EPISODE LINKSImproved rebalancing for Kafka ConnectImproved rebalancing for Kafka StreamsThe "what would Kafka do?" scenario from Mark PapadakisThe future of retail at NordstromWatch the video version of this podcast 2019-02-2021 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #9: With and Without ZooKeeperGwen asks: What happens when garbage collection causes Kafka to pause? And how do we run a Schema Registry cluster? We’ll find out in this episode of Ask Confluent.In "Ask Confluent," Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSZooKeeper connection timeout configuration: zookeeper.connection.timeout.ms, as we said, this defaults to 6,000Schema Registry failover instructionsWatch the video version of this podcast2019-01-0815 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #8: Guozhang Wang on Kafka Streams Standby TasksGwen is joined in studio by special guest Guozhang Wang, Kafka Streams pioneer and engineering lead at Confluent. He’ll talk to us about standby tasks and how one deserializes message headers. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSDocumentation of standby tasks, including configsEvents with different schema in same topicHow to populate a database from Kafka and solve the parent-child relation problemWatch the video version of this podcast2018-12-1822 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #7: Kafka Consumers and Streams Failover Explained ft. Matthias SaxGwen is joined in studio by special guest Matthias J. Sax, a software engineer at Confluent. He’ll talk to us about Kafka consumers and Kafka Streams failover. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSWatch the video version of this podcast2018-12-0323 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #6: Kafka, Partitions, and Exactly Once ft. Jason GustafsonGwen is joined in studio by special guest Jason Gustafson, a Kafka PMC member and engineer at Confluent. He’ll talk to us about the big questions on Kafka architecture— number of partitions and exactly once. In "Ask Confluent," Gwen Shapira (Data Architect, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSHardening Kafka ReplicationKafka open issuesWatch the video version of this podcast2018-11-0522 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #4: The GitHub EditionWant to see a feature implemented in KSQL or other Kafka-related project? Gwen answers your questions from YouTube and walks through how to use GitHub issues to request features. This is the episode #4 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSWatch the video version of this podcast2018-08-1613 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #3: Kafka Upgrades, Cloud APIs and Data DurabilityTim Berglund and Gwen Shapira have a discussion with Koelli Mungee (Customer Operations Lead, Confluent) and cover the latest Apache Kafka upgrades, cloud APIs, and data durability. This is episode #3 of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube, and elsewhere.EPISODE LINKSWatch the video version of this podcastFully managed Apache Kafka as a service! Try free.2018-07-2022 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #2: Consumers, Culture and SupportGwen Shapira answers your questions and interviews Sam Hecht (Head of Support, Confluent). This is the second episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSWatch the video version of this podcast2018-07-0224 min
Streaming Audio: Apache Kafka® & Real-Time DataAsk Confluent #1: Kubernetes, Confluent Operator, Kafka and KSQLTim Berglund and Gwen Shapira discuss Kubernetes, Confluent Operator, Kafka, KSQL, and more. This is the first episode of "Ask Confluent," a segment in which Gwen Shapira and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere.EPISODE LINKSWatch the video version of this podcast2018-06-2022 min
Drill to DetailDrill to Detail Ep.52 'Lyft, Analytics + ETL Developer Productivity' With Special Guest Mark GroverMark Grover LinkedIn Profile and Github Profile"Hadoop Application Architectures" "Drill to Detail Ep. 7 'Apache Spark and Hadoop Application Architectures'Lyft Engineering Blog"Software Engineer to Product Manager" blog by Gwen Shapira"Introduction to the Oracle Data Integrator Topology" from the Oracle Data Integrator docs siteApache Airflow and Amazon Kinesis homepages "Experimentation in a Ridesharing Marketplace" by Nicholas Chamandy, Head of Data Science at Lyft"How Uber Eats Works with Restaurants""Deliveroo has built a bunch of tiny kitchens to fe...2018-04-0948 minDrill to DetailDrill to Detail Ep.27 'Apache Kafka, Streaming Data Integration and Schema Registry' with Special Guest Gwen ShapiraMark Rittman is joined by Gwen Shapira from Confluent to talk about Apache Kafka, streaming data integration and how it differs from batch-based, GUI-developed ETL development, the problem with architects, exactly-once processing and how data governance is coming to Kafka development with Confluent's new schema registry server.2017-05-2342 minDrill to DetailDrill to Detail Ep.7 'Apache Spark, and Hadoop Application Architectures' with Special Guest Mark GroverMark is joined by Cloudera's Mark Grover to talk about his work with Apache Spark and Apache Spot, and the book "Hadoop Application Architectures" co-authored with Gwen Shapira, Jonathan Seidman and Ted Malaska2016-11-0140 min