1X
0:00/0:00
Data Engineering Podcast
This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.
Support the show!Listen in your favorite app:
Fountain
TrueFans
Podverse
Podcast Guru
Apple Podcasts
Spotify
Pick your app with Episodes.fm
Here are shows you might like
See show recommendations
Tobias Macey
Tobias Macey
Creating A Unified Experience For The Modern Data Stack At Mozart Data - E242
Summary
The modern data stack has been gaining a lot of attention recently with a rapidly growing set of managed services for different stages of the data lifecycle. With all of the available options it is possible to run a scalable, production grade data platform with a small team, but there are…
Summary
The modern data stack has been gaining a…
27 November 2021 | 00:58:31
Doing DataOps For External Data Sources As A Service at Demyst - E241
Summary
The data that you have access to affects the questions that you can answer. By using external data sources you can drastically increase the range of analysis that is available to your organization. The challenge comes in all of the operational aspects of finding, accessing, organizing, and…
Summary
The data that you have access to affects…
27 November 2021 | 00:59:17
Exploring Processing Patterns For Streaming Data Integration In Your Data Lake - E240
Summary
One of the perennial challenges posed by data lakes is how to keep them up to date as new data is collected. With the improvements in streaming engines it is now possible to perform all of your data integration in near real time, but it can be challenging to understand the proper processing…
Summary
One of the perennial challenges posed by…
20 November 2021 | 00:52:53
Laying The Foundation Of Your Data Platform For The Era Of Big Complexity With Dagster - E239
Summary
The technology for scaling storage and processing of data has gone through massive evolution over the past decade, leaving us with the ability to work with massive datasets at the cost of massive complexity. Nick Schrock created the Dagster framework to help tame that complexity and scale…
Summary
The technology for scaling storage and…
20 November 2021 | 01:05:25
Data Quality Starts At The Source - E238
Summary
The most important gauge of success for a data platform is the level of trust in the accuracy of the information that it provides. In order to build and maintain that trust it is necessary to invest in defining, monitoring, and enforcing data quality metrics. In this episode Michael Harper…
Summary
The most important gauge of success for a…
14 November 2021 | 00:58:55
Eliminate Friction In Your Data Platform Through Unified Metadata Using OpenMetadata - E237
Summary
A significant source of friction and wasted effort in building and integrating data management systems is the fragmentation of metadata across various tools. After experiencing the impacts of fragmented metadata and previous attempts at building a solution Suresh Srinivas and Sriharsha…
Summary
A significant source of friction and…
10 November 2021 | 01:06:55
Business Intelligence Beyond The Dashboard With ClicData - E236
Summary
Business intelligence is often equated with a collection of dashboards that show various charts and graphs representing data for an organization. What is overlooked in that characterization is the level of complexity and effort that are required to collect and present that information, and…
Summary
Business intelligence is often equated…
06 November 2021 | 01:02:00
Exploring The Evolution And Adoption of Customer Data Platforms and Reverse ETL - E235
Summary
The precursor to widespread adoption of cloud data warehouses was the creation of customer data platforms. Acting as a centralized repository of information about how your customers interact with your organization they drove a wave of analytics about how to improve products based on actual…
Summary
The precursor to widespread adoption of…
05 November 2021 | 01:02:07
Removing The Barrier To Exploratory Analytics with Activity Schema and Narrator - E234
Summary
The perennial question of data warehousing is how to model the information that you are storing. This has given rise to methods as varied as star and snowflake schemas, data vault modeling, and wide tables. The challenge with many of those approaches is that they are optimized for answering…
Summary
The perennial question of data…
29 October 2021 | 01:08:49
Streaming Data Pipelines Made SQL With Decodable - E233
Summary
Streaming data systems have been growing more capable and flexible over the past few years. Despite this, it is still challenging to build reliable pipelines for stream processing. In this episode Eric Sammer discusses the shortcomings of the current set of streaming engines and how they…
Summary
Streaming data systems have been growing…
29 October 2021 | 01:09:32