Data Engineering Podcast


This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.

Support the show!

Rewind 10 seconds
1X
Skip 30 seconds ahead
0:00/0:00

Listen in your favorite app:



More options

Here are shows you might like

See show recommendations
AI Engineering Podcast
Tobias Macey
The Python Podcast.__init__
Tobias Macey

444 Episodes

The View Below The Waterline Of Apache Iceberg And How It Fits In Your Data Lakehouse - E363

Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data applications, but they are still inherently limiting. Because of their complete ownership of your data they constrain the possibilities of what data you can store and how it can be used. Projects like Apache Iceberg provide a viable alternative in the…

Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data…

19 February 2023 | 00:55:07


Let The Whole Team Participate In Data With The Quilt Versioned Data Hub - E362

Summary Data is a team sport, but it's often difficult for everyone on the team to participate. For a long time the mantra of data tools has been "by developers, for developers", which automatically excludes a large portion of the business members who play a crucial role in the success of any data project. Quilt Data was created as an answer to…

Summary Data is a team sport, but it's often difficult for everyone on the team to participate. For…

11 February 2023 | 00:52:02


Reflecting On The Past 6 Years Of Data Engineering - E361

Summary This podcast started almost exactly six years ago, and the technology landscape was much different than it is now. In that time there have been a number of generational shifts in how data engineering is done. In this episode I reflect on some of the major themes and take a brief look forward at some of the upcoming…

Summary This podcast started almost exactly six years ago, and the technology landscape was much…

06 February 2023 | 00:32:21


Let Your Business Intelligence Platform Build The Models Automatically With Omni Analytics - E360

Summary Business intelligence has gone through many generational shifts, but each generation has largely maintained the same workflow. Data analysts create reports that are used by the business to understand and direct the business, but the process is very labor and time intensive. The team at Omni have taken a new approach by automatically…

Summary Business intelligence has gone through many generational shifts, but each generation has…

30 January 2023 | 00:50:44


Safely Test Your Applications And Analytics With Production Quality Data Using Tonic AI - E359

Summary The most interesting and challenging bugs always happen in production, but recreating them is a constant challenge due to differences in the data that you are working with. Building your own scripts to replicate data from production is time consuming and error-prone. Tonic is a platform designed to solve the problem of having reliable,…

Summary The most interesting and challenging bugs always happen in production, but recreating them…

22 January 2023 | 00:45:40


Building Applications With Data As Code On The DataOS - E358

Summary The modern data stack has made it more economical to use enterprise grade technologies to power analytics at organizations of every scale. Unfortunately it has also introduced new overhead to manage the full experience as a single workflow. At the Modern Data Company they created the DataOS platform as a means of driving your full analytics…

Summary The modern data stack has made it more economical to use enterprise grade technologies to…

16 January 2023 | 00:48:37


Automate Your Pipeline Creation For Streaming Data Transformations With SQLake - E357

Summary Managing end-to-end data flows becomes complex and unwieldy as the scale of data and its variety of applications in an organization grows. Part of this complexity is due to the transformation and orchestration of data living in disparate systems. The team at Upsolver is taking aim at this problem with the latest iteration of their platform…

Summary Managing end-to-end data flows becomes complex and unwieldy as the scale of data and its…

08 January 2023 | 00:44:06


Increase Your Odds Of Success For Analytics And AI Through More Effective Knowledge Management With AlignAI - E356

Summary Making effective use of data requires proper context around the information that is being used. As the size and complexity of your organization increases the difficulty of ensuring that everyone has the necessary knowledge about how to get their work done scales exponentially. Wikis and intranets are a common way to attempt to solve this…

Summary Making effective use of data requires proper context around the information that is being…

29 December 2022 | 00:59:21


Using Product Driven Development To Improve The Productivity And Effectiveness Of Your Data Teams - E355

Summary With all of the messaging about treating data as a product it is becoming difficult to know what that even means. Vishal Singh is the head of products at Starburst which means that he has to spend all of his time thinking and talking about the details of product thinking and its application to data. In this episode he shares his thoughts on…

Summary With all of the messaging about treating data as a product it is becoming difficult to know…

29 December 2022 | 00:58:46


An Exploration Of Tobias' Experience In Building A Data Lakehouse From Scratch - E354

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. In order to condense that acquired knowledge into a format that is useful to everyone Scott Hirleman turns the tables in this episode and…

Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth…

26 December 2022 | 01:12:00