This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.
Support the show!Listen in your favorite app:
Fountain TrueFans Podverse Podcast Guru Apple Podcasts Spotify Pick your app with Episodes.fmHere are shows you might like
Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data applications, but they are still inherently limiting. Because of their complete ownership of your data they constrain the possibilities of what data you can store and how it can be used. Projects like Apache Iceberg provide a viable alternative in the…
Summary Cloud data warehouses have unlocked a massive amount of innovation and investment in data…
19 February 2023 | 00:55:07
Summary Data is a team sport, but it's often difficult for everyone on the team to participate. For a long time the mantra of data tools has been "by developers, for developers", which automatically excludes a large portion of the business members who play a crucial role in the success of any data project. Quilt Data was created as an answer to…
Summary Data is a team sport, but it's often difficult for everyone on the team to participate. For…
11 February 2023 | 00:52:02
Summary This podcast started almost exactly six years ago, and the technology landscape was much different than it is now. In that time there have been a number of generational shifts in how data engineering is done. In this episode I reflect on some of the major themes and take a brief look forward at some of the upcoming…
Summary This podcast started almost exactly six years ago, and the technology landscape was much…
06 February 2023 | 00:32:21
Summary Business intelligence has gone through many generational shifts, but each generation has largely maintained the same workflow. Data analysts create reports that are used by the business to understand and direct the business, but the process is very labor and time intensive. The team at Omni have taken a new approach by automatically…
Summary Business intelligence has gone through many generational shifts, but each generation has…
30 January 2023 | 00:50:44
Summary The most interesting and challenging bugs always happen in production, but recreating them is a constant challenge due to differences in the data that you are working with. Building your own scripts to replicate data from production is time consuming and error-prone. Tonic is a platform designed to solve the problem of having reliable,…
Summary The most interesting and challenging bugs always happen in production, but recreating them…
22 January 2023 | 00:45:40
Summary The modern data stack has made it more economical to use enterprise grade technologies to power analytics at organizations of every scale. Unfortunately it has also introduced new overhead to manage the full experience as a single workflow. At the Modern Data Company they created the DataOS platform as a means of driving your full analytics…
Summary The modern data stack has made it more economical to use enterprise grade technologies to…
16 January 2023 | 00:48:37
Summary Managing end-to-end data flows becomes complex and unwieldy as the scale of data and its variety of applications in an organization grows. Part of this complexity is due to the transformation and orchestration of data living in disparate systems. The team at Upsolver is taking aim at this problem with the latest iteration of their platform…
Summary Managing end-to-end data flows becomes complex and unwieldy as the scale of data and its…
08 January 2023 | 00:44:06
Summary Making effective use of data requires proper context around the information that is being used. As the size and complexity of your organization increases the difficulty of ensuring that everyone has the necessary knowledge about how to get their work done scales exponentially. Wikis and intranets are a common way to attempt to solve this…
Summary Making effective use of data requires proper context around the information that is being…
29 December 2022 | 00:59:21
Summary With all of the messaging about treating data as a product it is becoming difficult to know what that even means. Vishal Singh is the head of products at Starburst which means that he has to spend all of his time thinking and talking about the details of product thinking and its application to data. In this episode he shares his thoughts on…
Summary With all of the messaging about treating data as a product it is becoming difficult to know…
29 December 2022 | 00:58:46
Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth of insight into the work of building and operating data systems at a variety of scales and for myriad purposes. In order to condense that acquired knowledge into a format that is useful to everyone Scott Hirleman turns the tables in this episode and…
Summary Five years of hosting the Data Engineering Podcast has provided Tobias Macey with a wealth…
26 December 2022 | 01:12:00