Data Engineering Podcast


This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.

Support the show!

Rewind 10 seconds
1X
Skip 30 seconds ahead
0:00/0:00

Listen in your favorite app:



More options

Here are shows you might like

See show recommendations
AI Engineering Podcast
Tobias Macey
The Python Podcast.__init__
Tobias Macey

454 Episodes

Neon: A Serverless And Developer Friendly Postgres - E433

Summary Postgres is one of the most widely respected and liked database engines ever. To make it even easier to use for developers to use, Nikita Shamgunov decided to makee it serverless, so that it can scale from zero to infinity. In this episode he explains the engineering involved to make that possible, as well as the numerous details that he…

Summary Postgres is one of the most widely respected and liked database engines ever. To make it…

08 July 2024 | 00:57:43


Improve Data Quality Through Engineering Rigor And Business Engagement With Synq - E432

Summary This episode features an insightful conversation with Petr Janda, the CEO and founder of Synq. Petr shares his journey from being an engineer to founding Synq, emphasizing the importance of treating data systems with the same rigor as engineering systems. He discusses the challenges and solutions in data reliability, including the need for…

Summary This episode features an insightful conversation with Petr Janda, the CEO and founder of…

30 June 2024 | 00:59:48


Stitching Together Enterprise Analytics With Microsoft Fabric - E431

Summary Data lakehouse architectures have been gaining significant adoption. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture. In this episode Dipti Borkar shares her experiences working on the product team at Fabric and explains the various use cases for the Fabric…

Summary Data lakehouse architectures have been gaining significant adoption. To accelerate adoption…

23 June 2024 | 00:53:23


Being Data Driven At Stripe With Trino And Iceberg - E430

Summary Stripe is a company that relies on data to power their products and business. To support that functionality they have invested in Trino and Iceberg for their analytical workloads. In this episode Kevin Liu shares some of the interesting features that they have built by combining those technologies, as well as the challenges that they face…

Summary Stripe is a company that relies on data to power their products and business. To support…

16 June 2024 | 00:53:20


X-Ray Vision For Your Flink Stream Processing With Datorios - E429

Summary Streaming data processing enables new categories of data products and analytics. Unfortunately, reasoning about stream processing engines is complex and lacks sufficient tooling. To address this shortcoming Datorios created an observability platform for Flink that brings visibility to the internals of this popular stream processing system.…

Summary Streaming data processing enables new categories of data products and analytics.…

09 June 2024 | 00:42:22


Practical First Steps In Data Governance For Long Term Success - E428

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the challenge of building data systems to support that goal. Data governance is the binding force between these two parts of the organization. Nicola Askham found her way into data governance by accident, and stayed because of the benefit that she was able…

Summary Modern businesses aspire to be data driven, and technologists enjoy working through the…

02 June 2024 | 01:00:41


Data Migration Strategies For Large Scale Systems - E427

Summary Any software system that survives long enough will require some form of migration or evolution. When that system is responsible for the data layer the process becomes more challenging. Sriram Panyam has been involved in several projects that required migration of large volumes of data in high traffic environments. In this episode he shares…

Summary Any software system that survives long enough will require some form of migration or…

27 May 2024 | 01:00:00


Zenlytic Is Building You A Better Coworker With AI Agents - E426

Summary The purpose of business intelligence systems is to allow anyone in the business to access and decode data to help them make informed decisions. Unfortunately this often turns into an exercise in frustration for everyone involved due to complex workflows and hard-to-understand dashboards. The team at Zenlytic have leaned on the promise of…

Summary The purpose of business intelligence systems is to allow anyone in the business to access…

19 May 2024 | 00:54:19


Release Management For Data Platform Services And Logic - E425

Summary Building a data platform is a substrantial engineering endeavor. Once it is running, the next challenge is figuring out how to address release management for all of the different component parts. The services and systems need to be kept up to date, but so does the code that controls their behavior. In this episode your host Tobias Macey…

Summary Building a data platform is a substrantial engineering endeavor. Once it is running, the…

12 May 2024 | 00:20:09


Barking Up The Wrong GPTree: Building Better AI With A Cognitive Approach - E424

Summary Artificial intelligence has dominated the headlines for several months due to the successes of large language models. This has prompted numerous debates about the possibility of, and timeline for, artificial general intelligence (AGI). Peter Voss has dedicated decades of his life to the pursuit of truly intelligent software through the…

Summary Artificial intelligence has dominated the headlines for several months due to the successes…

05 May 2024 | 00:54:17