Data Engineering Podcast

Share on social media:

Support Data Engineering Podcast

[00:00:14] Tobias Macey:

Hello, and welcome to the Data Engineering Podcast, the show about modern data infrastructure.   You can go to dataengineeringpodcast.com   to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.   To help other people find the show, you can leave a review on Itunes or Google Play Music and share it on social media and tell your friends and coworkers.   I'm your host, Tobias Macy, and I'd like to introduce a new podcast that I'm starting called the Data Engineering Podcast.   A little bit of background about myself. I started off as a systems administrator   and moved up through to being a full time software engineer for a while.  

Now I'm working across those boundaries as a DevOps engineer,   and I have a strong focus on automation and all things. I've also done a fair bit of data engineering myself.   I also currently host another podcast called podcast.net,   which is about the Python programming language and its community.   So I first got involved in data management through my work as a systems administrator of just managing databases   and data pipelines   through various different systems.   And in my next role, I did a fair bit of data analysis   and data management for a company that   received a lot of data from a distributed sensor network. And so debugging the issues with those sensors and making sure that there is adequate reporting data available to   be able to   report on the accuracy of those sensors, etcetera.  

I've had a interest in the data management and data engineering space for a while as well as data science.   And I've noticed that there are a lot of different podcasts available for data science as a discipline, but there aren't any that are dedicated to data engineering and data infrastructure,   which is a similar situation that I found myself in with Python where I was a big fan of the Python programming language and its community, but there weren't any podcasts that focused on that particular market.   So   in this case, I decided to start a new podcast about data engineering.  

And   it's an interesting area that's got a lot of activity, and there's a lot been a lot of growth   in the idea of data engineering and data infrastructure.   And   so as far as the kind of   subject area that I wanna cover with this podcast, I'm interested in covering things like databases   and data storage,   data pipelines and management, some of the different tooling that is useful and important for the discipline of data engineering.   I also like to focus on automation both in terms of data automation as far as collecting and aggregating it and processing it, but also automation of the underlying infrastructure,   monitoring of data pipelines so that you can be sure that the proper   processes are taking place   as well as testing of those data pipelines so that you can be sure   as you roll out changes that you're not gonna end up with bad data at the end of it. I also wanna cover some of the different best practices that are evolving in the space,   some of the common challenges that are faced by practitioners,   and also   working on trying to help define what it actually means to be a data engineer   and some of the challenges associated with finding jobs in that particular space because it is such an evolving area of technology.  

Another subject that I think would be interesting to cover is some of the relationship with   data engineers and data scientists and   what that working relationship looks like.   So   if this is something that you're interested in, then I suggest you subscribe to the feed because there are gonna be a number of interesting episodes. I've already been speaking with a few different people, including some of the folks from Packarderm,   Maxime Beauchmann from Airbnb.   I've got an episode about the   Dask   distributed   environment   coming up.  

So   go to the site, subscribe to the show,   sign up for the newsletter,   follow me on Twitter,   or send me an email. All those links are gonna be on the site. So thanks for tuning in, and I look forward to having you back   soon.