In this talk, Juan, Analytics Engineer and author of Fundamentals of Analytics Engineering share his professional journey from studying psychological research in Colombia to becoming one of the first analytics engineers in the Netherlands. We explore the evolution of the role, the shift toward engineering rigor in data modeling, and how the landscape of tools like dbt and Databricks is changing the way teams work.You’ll learn about:- The fundamental differences between traditional BI engineering and modern analytics engineering.- How to bridge the gap between business stakeholders and technical data infrastructure.- The technical "glue" that connects Python and SQL for robust data pipelines.- The importance of automated testing (generic vs. singular tests) to prevent "silent" data failures.- Strategies for modeling messy, fragmented source data into a unified "business reality."- The current state of the "Lakehouse" paradigm and how it impacts storage and compute costs.- Expert advice on navigating the dbt ecosystem and its emerging competitors.Links:- DE Course: https://github.com/DataTalksClub/data-engineering-zoomcamp- Luma: https://luma.com/0uf7mmupTIMECODES:0:00 Juan’s psychological research and transition to data4:36 Riding the wave: The early days of analytics engineering7:56 Breaking down the gap between analysts and engineers11:03 The art of turning business reality into clean data16:25 Why data engineering is about safety, not just speed20:53 Reimagining data modeling in the modern era26:53 To split or not to split: Finding the right team roles30:35 Python, SQL, and the technical toolkit for success38:41 How to stop manually testing your data dashboards46:34 Bringing software engineering rigor to data workflows49:50 Must-read books and resources for mastering the craft55:42 The future of dbt and the shifting tool landscape1:00:29 Deciphering the lakehouse: Warehousing in the cloud1:11:16 Pro-tips for starting your data engineering journey1:14:40 The big debate: Databricks vs. Snowflake1:18:28 Why every data professional needs a local communityThis talk is designed for data analysts looking to level up their engineering skills, data engineers interested in the business-logic layer, and data leaders trying to structure their teams more effectively. It is particularly valuable for those preparing for the Data Engineering Zoomcamp or anyone looking to transition into an Analytics Engineering role.Connect with Juan- Linkedin - https://www.linkedin.com/in/jmperafan/ - Website - https://juanalytics.com/Connect with DataTalks.Club:- Join the community - https://datatalks.club/slack.html- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ- Check other upcoming events - https://lu.ma/dtc-events- GitHub: https://github.com/DataTalksClub- LinkedIn - https://www.linkedin.com/company/datatalks-club/ - Twitter - https://twitter.com/DataTalksClub - Website - https://datatalks.club/
