Escaping Analysis Paralysis For Your Data Platform With Data Virtualization - Episode 107

November 18, 2019 tmacey

With the constant evolution of technology for data management it can seem impossible to make an informed decision about whether to build a data warehouse, or a data lake, or just leave your data wherever it currently rests. What's worse is that any time you have to migrate to a new architecture, all of your analytical code has to change too. Thankfully it's possible to add an abstraction layer to eliminate the churn in your client code, allowing you to evolve your data platform without disrupting your downstream data...

Play Episode

Designing For Data Protection - Episode 106

November 11, 2019 tmacey

The practice of data management is one that requires technical acumen, but there are also many policy and regulatory issues that inform and influence the design of our systems. With the introduction of legal frameworks such as the EU GDPR and California's CCPA it is necessary to consider how to implement data protectino and data privacy principles in the technical and policy controls that govern our data platforms. In this episode Karen Heaton and Mark Sherwood-Edwards share their experience and expertise in helping organizations achieve compliance. Even if you...

Play Episode

Automating Your Production Dataflows On Spark - Episode 105

November 4, 2019 tmacey

As data engineers the health of our pipelines is our highest priority. Unfortunately, there are countless ways that our dataflows can break or degrade that have nothing to do with the business logic or data transformations that we write and maintain. Sean Knapp founded Ascend to address the operational challenges of running a production grade and scalable Spark infrastructure, allowing data engineers to focus on the problems that power their business. In this episode he explains the technical implementation of the Ascend platform, the challenges that he has faced...

Play Episode

Build Maintainable And Testable Data Applications With Dagster - Episode 104

October 28, 2019 tmacey

Despite the fact that businesses have relied on useful and accurate data to succeed for decades now, the state of the art for obtaining and maintaining that information still leaves much to be desired. In an effort to create a better abstraction for building data applications Nick Schrock created Dagster. In this episode he explains his motivation for creating a product for data management, how the programming model simplifies the work of building testable and maintainable pipelines, and his vision for the future of data programming. If you are...

Play Episode

Data Orchestration For Hybrid Cloud Analytics - Episode 103

October 22, 2019 tmacey

The scale and complexity of the systems that we build to satisfy business requirements is increasing as the available tools become more sophisticated. In order to bridge the gap between legacy infrastructure and evolving use cases it is necessary to create a unifying set of components. In this episode Dipti Borkar explains how the emerging category of data orchestration tools fills this need, some of the existing projects that fit in this space, and some of the ways that they can work together to simplify projects such as cloud...

Play Episode

Subscribe Wherever You Listen To Podcasts

Home 1Apple PodcastsHome 2OvercastHome 3Google PodcastsHome 4Google PlayHome 5SpotifyHome 6PocketCastsHome 7Radio PublicHome 8CastBoxHome 9StitcherHome 10TuneInHome 11RSSHome 12iHeartRadioHome 13Breaker

Join The Mailing List