Latest Episodes

Making Spark Cloud Native At Data Mechanics - Episode 184

May 7, 2021 tmacey Comments Off on Making Spark Cloud Native At Data Mechanics - Episode 184

Spark is one of the most well-known frameworks for data processing, whether for batch or streaming, ETL or ML, and at any scale. Because of its popularity it has been deployed on every kind of platform you can think of. In this episode Jean-Yves Stephan shares the work that he is doing at Data Mechanics to make it sing on Kubernetes. He explains how operating in a cloud-native context simplifies some aspects of running the system while complicating others, how it simplifies the development and experimentation cycle, and how...

Play Episode

The Grand Vision And Present Reality of DataOps - Episode 183

May 4, 2021 tmacey Comments Off on The Grand Vision And Present Reality of DataOps - Episode 183

The Data industry is changing rapidly, and one of the most active areas of growth is automation of data workflows. Taking cues from the DevOps movement of the past decade data professionals are orienting around the concept of DataOps. More than just a collection of tools, there are a number of organizational and conceptual changes that a proper DataOps approach depends on. In this episode Kevin Stumpf, CTO of Tecton, Maxime Beauchemin, CEO of Preset, and Lior Gavish, CTO of Monte Carlo, discuss the grand vision and present realities...

Play Episode

Self Service Data Exploration And Dashboarding With Superset - Episode 182

April 27, 2021 tmacey Comments Off on Self Service Data Exploration And Dashboarding With Superset - Episode 182

The reason for collecting, cleaning, and organizing data is to make it usable by the organization. One of the most common and widely used methods of access is through a business intelligence dashboard. Superset is an open source option that has been gaining popularity due to its flexibility and extensible feature set. In this episode Maxime Beauchemin discusses how data engineers can use Superset to provide self service access to data and deliver analytics. He digs into how it integrates with your data stack, how you can extend it...

Play Episode

Moving Machine Learning Into The Data Pipeline at Cherre - Episode 181

April 20, 2021 tmacey Comments Off on Moving Machine Learning Into The Data Pipeline at Cherre - Episode 181

Most of the time when you think about a data pipeline or ETL job what comes to mind is a purely mechanistic progression of functions that move data from point A to point B. Sometimes, however, one of those transformations is actually a full-fledged machine learning project in its own right. In this episode Tal Galfsky explains how he and the team at Cherre tackled the problem of messy data for Addresses by building a natural language processing and entity resolution system that is served as an API to...

Play Episode

Exploring The Expanding Landscape Of Data Professions with Josh Benamram of Databand - Episode 180

April 13, 2021 tmacey Comments Off on Exploring The Expanding Landscape Of Data Professions with Josh Benamram of Databand - Episode 180

"Business as usual" is changing, with more companies investing in data as a first class concern. As a result, the data team is growing and introducing more specialized roles. In this episode Josh Benamram, CEO and co-founder of Databand, describes the motivations for these emerging roles, how these positions affect the team dynamics, and the types of visibility that they need into the data platform to do their jobs effectively. He also talks about how his experience working with these teams informs his work at Databand. If you are...

Play Episode

Put Your Whole Data Team On The Same Page With Atlan - Episode 179

April 6, 2021 tmacey Comments Off on Put Your Whole Data Team On The Same Page With Atlan - Episode 179

One of the biggest obstacles to success in delivering data products is cross-team collaboration. Part of the problem is the difference in the information that each role requires to do their job and where they expect to find it. This introduces a barrier to communication that is difficult to overcome, particularly in teams that have not reached a significant level of maturity in their data journey. In this episode Prukalpa Sankar shares her experiences across multiple attempts at building a system that brings everyone onto the same page, ultimately...

Play Episode

Data Quality Management For The Whole Team With Soda Data - Episode 178

March 30, 2021 tmacey Comments Off on Data Quality Management For The Whole Team With Soda Data - Episode 178

Data quality is on the top of everyone's mind recently, but getting it right is as challenging as ever. One of the contributing factors is the number of people who are involved in the process and the potential impact on the business if something goes wrong. In this episode Maarten Masschelein and Tom Baeyens share the work they are doing at Soda to bring everyone on board to make your data clean and reliable. They explain how they started down the path of building a solution for managing data...

Play Episode

Real World Change Data Capture At Datacoral - Episode 177

March 23, 2021 tmacey Comments Off on Real World Change Data Capture At Datacoral - Episode 177

The world of business is becoming increasingly dependent on information that is accurate up to the minute. For analytical systems, the only way to provide this reliably is by implementing change data capture (CDC). Unfortunately, this is a non-trivial undertaking, particularly for teams that don't have extensive experience working with streaming data and complex distributed systems. In this episode Raghu Murthy, founder and CEO of Datacoral, does a deep dive on how he and his team manage change data capture pipelines in production.

Play Episode

Managing The DoorDash Data Platform - Episode 176

March 16, 2021 tmacey Comments Off on Managing The DoorDash Data Platform - Episode 176

The team at DoorDash has a complex set of optimization challenges to deal with using data that they collect from a multi-sided marketplace. In order to handle the volume and variety of information that they use to run and improve the business the data team has to build a platform that analysts and data scientists can use in a self-service manner. In this episode the head of data platform for DoorDash, Sudhir Tonse, discusses the technologies that they are using, the approach that they take to adding new systems,...

Play Episode

Leave Your Data Where It Is And Automate Feature Extraction With Molecula - Episode 175

March 9, 2021 tmacey Comments Off on Leave Your Data Where It Is And Automate Feature Extraction With Molecula - Episode 175

A majority of the time spent in data engineering is copying data between systems to make the information available for different purposes. This introduces challenges such as keeping information synchronized, managing schema evolution, building transformations to match the expectations of the destination systems. H.O. Maycotte was faced with these same challenges but at a massive scale, leading him to question if there is a better way. After tasking some of his top engineers to consider the problem in a new light they created the Pilosa engine. In this episode...

Play Episode

Subscribe Wherever You Listen To Podcasts

Apple PodcastsApple PodcastsOvercastOvercastGoogle PodcastsGoogle PodcastsGoogle PlayGoogle PlaySpotifySpotifyPocketCastsPocketCastsRadio PublicRadio PublicCastBoxCastBoxStitcherStitcherTuneInTuneInRSSRSSiHeartRadioiHeartRadioBreakerBreaker

Join The Mailing List