1X
0:00/0:00
Data Engineering Podcast
This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.
Support the show!Listen in your favorite app:
Fountain
TrueFans
Podverse
Podcast Guru
Apple Podcasts
Spotify
Pick your app with Episodes.fm
Here are shows you might like
See show recommendations
Tobias Macey
Tobias Macey
Scale Your Analytics On The Clickhouse Data Warehouse - E88
Summary
The market for data warehouse platforms is large and varied, with options for every use case. ClickHouse is an open source, column-oriented database engine built for interactive analytics with linear scalability. In this episode Robert Hodges and Alexander Zaitsev explain how it is…
Summary
The market for data warehouse platforms…
08 July 2019 | 01:11:19
Stress Testing Kafka And Cassandra For Real-Time Anomaly Detection - E87
Summary
Anomaly detection is a capability that is useful in a variety of problem domains, including finance, internet of things, and systems monitoring. Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he…
Summary
Anomaly detection is a capability that is…
02 July 2019 | 00:38:03
The Workflow Engine For Data Engineers And Data Scientists - E86
Summary
Building a data platform that works equally well for data engineering and data science is a task that requires familiarity with the needs of both roles. Data engineering platforms have a strong focus on stateful execution and tasks that are strictly ordered based on dependency graphs. Data…
Summary
Building a data platform that works…
25 June 2019 | 01:08:26
Maintaining Your Data Lake At Scale With Spark - E85
Summary
Building and maintaining a data lake is a choose your own adventure of tools, services, and evolving best practices. The flexibility and freedom that data lakes provide allows for generating significant value, but it can also lead to anti-patterns and inconsistent quality in your analytics.…
Summary
Building and maintaining a data lake is a…
17 June 2019 | 00:50:50
Managing The Machine Learning Lifecycle - E84
Summary
Building a machine learning model can be difficult, but that is only half of the battle. Having a perfect model is only useful if you are able to get it into production. In this episode Stepan Pushkarev, founder of Hydrosphere, explains why deploying and maintaining machine learning…
Summary
Building a machine learning model can be…
10 June 2019 | 01:02:40
Evolving An ETL Pipeline For Better Productivity - E83
Summary
Building an ETL pipeline can be a significant undertaking, and sometimes it needs to be rebuilt when a better option becomes available. In this episode Aaron Gibralter, director of engineering at Greenhouse, joins Raghu Murthy, founder and CEO of DataCoral, to discuss the journey that he…
Summary
Building an ETL pipeline can be a…
04 June 2019 | 01:02:22
Data Lineage For Your Pipelines - E82
Summary
Some problems in data are well defined and benefit from a ready-made set of tools. For everything else, there’s Pachyderm, the platform for data science that is built to scale. In this episode Joe Doliner, CEO and co-founder, explains how Pachyderm started as an attempt to make data…
Summary
Some problems in data are well defined…
27 May 2019 | 00:49:01
Build Your Data Analytics Like An Engineer With DBT - E81
Summary
In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. As a result, the tooling for those transformations needs to be reimagined. The data build tool (dbt) is designed to bring battle tested…
Summary
In recent years the traditional approach…
20 May 2019 | 00:56:46
Using FoundationDB As The Bedrock For Your Distributed Systems - E80
Summary
The database market continues to expand, offering systems that are suited to virtually every use case. But what happens if you need something customized to your application? FoundationDB is a distributed key-value store that provides the primitives that you need to build a custom database…
Summary
The database market continues to expand,…
07 May 2019 | 01:06:02
Running Your Database On Kubernetes With KubeDB - E79
Summary
Kubernetes is a driving force in the renaissance around deploying and running applications. However, managing the database layer is still a separate concern. The KubeDB project was created as a way of providing a simple mechanism for running your storage system in the same platform as your…
Summary
Kubernetes is a driving force in the…
29 April 2019 | 00:50:55