Data Labeling That You Can Feel Good About - Episode 89

July 15, 2019 tmacey

Successful machine learning and artificial intelligence projects require large volumes of data that is properly labelled. The challenge is that most data is not clean and well annotated, requiring a scalable data labeling process. Ideally this process can be done using the tools and systems that already power your analytics, rather than sending data into a black box. In this episode Mark Sears, CEO of CloudFactory, explains how he and his team built a platform that provides valuable service to businesses and meaningful work to developing nations. He shares...

Play Episode

Scale Your Analytics On The Clickhouse Data Warehouse - Episode 88

July 8, 2019 tmacey

The market for data warehouse platforms is large and varied, with options for every use case. ClickHouse is an open source, column-oriented database engine built for interactive analytics with linear scalability. In this episode Robert Hodges and Alexander Zaitsev explain how it is architected to provide these features, the various unique capabilities that it provides, and how to run it in production. It was interesting to learn about some of the custom data types and performance optimizations that are included.

Play Episode

Stress Testing Kafka And Cassandra For Real-Time Anomaly Detection - Episode 87

July 2, 2019 tmacey

Anomaly detection is a capability that is useful in a variety of problem domains, including finance, internet of things, and systems monitoring. Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he could push Kafka and Cassandra for this use case. In this interview he explains the system design that he tested, his findings for how these tools were able to work together, and how they behaved at different orders of scale. It...

Play Episode

The Workflow Engine For Data Engineers And Data Scientists - Episode 86

June 25, 2019 tmacey

Building a data platform that works equally well for data engineering and data science is a task that requires familiarity with the needs of both roles. Data engineering platforms have a strong focus on stateful execution and tasks that are strictly ordered based on dependency graphs. Data science platforms provide an environment that is conducive to rapid experimentation and iteration, with data flowing directly between stages. Jeremiah Lowin has gained experience in both styles of working, leading him to be frustrated with all of the available tools. In this...

Play Episode

Maintaining Your Data Lake At Scale With Spark - Episode 85

June 17, 2019 tmacey

Building and maintaining a data lake is a choose your own adventure of tools, services, and evolving best practices. The flexibility and freedom that data lakes provide allows for generating significant value, but it can also lead to anti-patterns and inconsistent quality in your analytics. Delta Lake is an open source, opinionated framework built on top of Spark for interacting with and maintaining data lake platforms that incorporates the lessons learned at DataBricks from countless customer use cases. In this episode Michael Armbrust, the lead architect of Delta Lake,...

Play Episode

Subscribe Wherever You Listen To Podcasts

Home 1Apple PodcastsHome 2OvercastHome 3Google PodcastsHome 4Google PlayHome 5SpotifyHome 6PocketCastsHome 7Radio PublicHome 8CastBoxHome 9StitcherHome 10TuneInHome 11RSSHome 12iHeartRadioHome 13Breaker

Join The Mailing List