Summary
In this episode, I spoke with Keerti Kulkarni about the data and AI systems behind physical logistics automation at Amazon. She explained how her team supports dock-door operations across more than 100 North American facilities by turning telemetry from physical workflows into actionable operational intelligence. We discussed the end-to-end data path from CloudWatch, Firehose, and S3 into Redshift and AI-powered analysis, along with the realities of handling petabyte-scale event streams, late and duplicate data, reconciliation, cost controls, and the need to treat physical telemetry as a first-class signal rather than noisy exhaust. Kirti also shared how these systems are designed to protect worker privacy while still surfacing the right information for safety and operational decisions.
We also dug into the role of LLMs and agentic workflows in helping operations teams diagnose failures quickly through conversational interfaces. Kirti described how AI-generated summaries help identify root causes such as authorization problems, hardware issues, or connectivity failures, while human feedback loops, structured outputs, and strict vocabulary controls help manage hallucination risk. Beyond troubleshooting, we explored how the platform is evolving toward proactive maintenance, training insights, per-site cost attribution, and broader modular agent architectures that could be shared across logistics teams. It’s a detailed look at what it takes to combine data engineering, real-time operations, and AI in a large-scale physical environment.
Announcements
Interview
Contact Info
Parting Question
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
In this episode, I spoke with Keerti Kulkarni about the data and AI systems behind physical logistics automation at Amazon. She explained how her team supports dock-door operations across more than 100 North American facilities by turning telemetry from physical workflows into actionable operational intelligence. We discussed the end-to-end data path from CloudWatch, Firehose, and S3 into Redshift and AI-powered analysis, along with the realities of handling petabyte-scale event streams, late and duplicate data, reconciliation, cost controls, and the need to treat physical telemetry as a first-class signal rather than noisy exhaust. Kirti also shared how these systems are designed to protect worker privacy while still surfacing the right information for safety and operational decisions.
We also dug into the role of LLMs and agentic workflows in helping operations teams diagnose failures quickly through conversational interfaces. Kirti described how AI-generated summaries help identify root causes such as authorization problems, hardware issues, or connectivity failures, while human feedback loops, structured outputs, and strict vocabulary controls help manage hallucination risk. Beyond troubleshooting, we explored how the platform is evolving toward proactive maintenance, training insights, per-site cost attribution, and broader modular agent architectures that could be shared across logistics teams. It’s a detailed look at what it takes to combine data engineering, real-time operations, and AI in a large-scale physical environment.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- Today’s episode is sponsored by Parallel - where agents find answers. Most engineers today closely follow new model releases, but don’t pay attention to their agent’s most important tool: web search. Parallel develops enterprise-grade infrastructure for agents to retrieve high-quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto-optimal quality, cost, and speed. Whether you work on voice agents that need 200 millisecond latency, chat bots that balance speed, depth, and quality, or long-horizon agents to do thorough, overnight research for you, Parallel is a single platform for all your agentic research. Get started for free at dataengineeringpodcast.com/parallel
- Your host is Tobias Macey and today I'm interviewing Keerti Kulkarni about scaling insights for physical logistics automation with AI-enabled intelligence
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you start by unpacking what "physical logistics automation is"?
- What are the technical and organizational systems involved in the production and consumption of that data?
- What are some of the ways that inaccuracies or increased latency manifest in real-world impact?
- You recently worked on a substantial system for managing this data. Can you describe the challenges and limitations of the system that you started from?
- What was your process for identifying the weak points of the system and how to evolve it?
- What is the design of the system that you have running today?
- What are some of the design choices that had the biggest impact on the final result?
- What are the most complicated aspects of managing cost attribution to the different physical locations, data flows, and model usage?
- What is the role of AI in providing insights to the organizational users?
- How do you measure and manage accuracy of the models as the system evolves?
- What are the most interesting, innovative, or unexpected ways that you have seen your system used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on physical logistics automation?
- What do you have planned for the future of your platform?
Contact Info
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
- Amazon Key
- AWS Cloudwatch Analytics
- AWS Firehose
- AWS Kinesis
- AWS Quick
- OSHA == Occupational Safety and Health Administration
- OmniGraph
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
