Digging Into Data Replication At Fivetran - Episode 93

August 12, 2019 tmacey

The extract and load pattern of data replication is the most commonly needed process in data engineering workflows. Because of the myriad sources and destinations that are available, it is also among the most difficult tasks that we encounter. Fivetran is a platform that does the hard work for you and replicates information from your source systems into whichever data warehouse you use. In this episode CEO and co-founder George Fraser explains how it is built, how it got started, and the challenges that creep in at the edges...

Play Episode

Solving Data Discovery At Lyft - Episode 92

August 5, 2019 tmacey

Data is only valuable if you use it for something, and the first step is knowing that it is available. As organizations grow and data sources proliferate it becomes difficult to keep track of everything, particularly for analysts and data scientists who are not involved with the collection and management of that information. Lyft has build the Amundsen platform to address the problem of data discovery and in this episode Tao Feng and Mark Grover explain how it works, why they built it, and how it has impacted the...

Play Episode

Simplifying Data Integration Through Eventual Connectivity - Episode 91

July 29, 2019 tmacey

The ETL pattern that has become commonplace for integrating data from multiple sources has proven useful, but complex to maintain. For a small number of sources it is a tractable problem, but as the overall complexity of the data ecosystem continues to expand it may be time to identify new ways to tame the deluge of information. In this episode Tim Ward, CEO of CluedIn, explains the idea of eventual connectivity as a new paradigm for data integration. Rather than manually defining all of the mappings ahead of time,...

Play Episode

Straining Your Data Lake Through A Data Mesh - Episode 90

July 22, 2019 tmacey

The current trend in data management is to centralize the responsibilities of storing and curating the organization's information to a data engineering team. This organizational pattern is reinforced by the architectural pattern of data lakes as a solution for managing storage and access. In this episode Zhamak Dehghani shares an alternative approach in the form of a data mesh. Rather than connecting all of your data flows to one destination, empower your individual business units to create data products that can be consumed by other teams. This was an...

Play Episode

Data Labeling That You Can Feel Good About - Episode 89

July 15, 2019 tmacey

Successful machine learning and artificial intelligence projects require large volumes of data that is properly labelled. The challenge is that most data is not clean and well annotated, requiring a scalable data labeling process. Ideally this process can be done using the tools and systems that already power your analytics, rather than sending data into a black box. In this episode Mark Sears, CEO of CloudFactory, explains how he and his team built a platform that provides valuable service to businesses and meaningful work to developing nations. He shares...

Play Episode

Subscribe Wherever You Listen To Podcasts

Home 1Apple PodcastsHome 2OvercastHome 3Google PodcastsHome 4Google PlayHome 5SpotifyHome 6PocketCastsHome 7Radio PublicHome 8CastBoxHome 9StitcherHome 10TuneInHome 11RSSHome 12iHeartRadioHome 13Breaker

Join The Mailing List