Building A Community For Data Professionals at Data Council - Episode 96

Summary

Data professionals are working in a domain that is rapidly evolving. In order to stay current we need access to deeply technical presentations that aren’t burdened by extraneous marketing. To fulfill that need Pete Soderling and his team have been running the Data Council series of conferences and meetups around the world. In this episode Pete discusses his motivation for starting these events, how they serve to bring the data community together, and the observations that he has made about the direction that we are moving. He also shares his experiences as an investor in developer oriented startups and his views on the importance of empowering engineers to launch their own companies.

linode-banner-sponsor-largeDo you want to try out some of the tools and applications that you heard about on the Data Engineering Podcast? Do you have some ETL jobs that need somewhere to run? Check out Linode at linode.com/dataengineeringpodcast or use the code dataengineering2019 and get a $20 credit (that’s 4 months free!) to try out their fast and reliable Linux virtual servers. They’ve got lightning fast networking and SSD servers with plenty of power and storage to run whatever you want to experiment on.


Listen, I’m sure you work for a ‘data driven’ company – who doesn’t these days?? Does your company use Amazon Redshift? Have you ever groaned over slow queries or are just afraid that Amazon Redshift is gonna fall over at some point??

Well, you GOTTA talk to the folks over at intermix.io. They have built the “missing” Amazon Redshift console – it’s an amazing analytics product for data engineers to find and re-write slow queries and gives actionable recommendations to optimize data pipelines. WeWork, Postmates, and Medium are just a few of their customers.

DEP listeners get a $50 discount! Just go to dataengineeringpodcast.com/intermix and use promo code DEP at sign up.


Announcements

  • Hello and welcome to the Data Engineering Podcast, the show about modern data management
  • When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • Listen, I’m sure you work for a ‘data driven’ company – who doesn’t these days? Does your company use Amazon Redshift? Have you ever groaned over slow queries or are just afraid that Amazon Redshift is gonna fall over at some point? Well, you’ve got to talk to the folks over at intermix.io. They have built the “missing” Amazon Redshift console – it’s an amazing analytics product for data engineers to find and re-write slow queries and gives actionable recommendations to optimize data pipelines. WeWork, Postmates, and Medium are just a few of their customers. Go to dataengineeringpodcast.com/intermix today and use promo code DEP at sign up to get a $50 discount!
  • You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council. Upcoming events include the O’Reilly AI conference, the Strata Data conference, the combined events of the Data Architecture Summit and Graphorum, and Data Council in Barcelona. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
  • Your host is Tobias Macey and today I’m interviewing Pete Soderling about his work to build and grow a community for data professionals with the Data Council conferences and meetups, as well as his experiences as an investor in data oriented companies

Interview

  • Introduction
  • How did you get involved in the area of data management?
  • What was your original reason for focusing your efforts on fostering a community of data engineers?
    • What was the state of recognition in the industry for that role at the time that you began your efforts?
  • The current manifestation of your community efforts is in the form of the Data Council conferences and meetups. Previously they were known as Data Eng Conf and before that was Hakka Labs. Can you discuss the evolution of your efforts to grow this community?
    • How has the community itself changed and grown over the past few years?
  • Communities form around a huge variety of focal points. What are some of the complexities or challenges in building one based on something as nebulous as data?
  • Where do you draw inspiration and direction for how to manage such a large and distributed community?
    • What are some of the most interesting/challenging/unexpected aspects of community management that you have encountered?
  • What are some ways that you have been surprised or delighted in your interactions with the data community?
  • How do you approach sustainability of the Data Council community and the organization itself?
  • The tagline that you have focused on for Data Council events is that they are no fluff, juxtaposing them against larger business oriented events. What are your guidelines for fulfilling that promise and why do you think that is an important distinction?
  • In addition to your community building you are also an investor. How did you get involved in that side of your business and how does it fit into your overall mission?
  • You also have a stated mission to help engineers build their own companies. In your opinion, how does an engineer led business differ from one that may be founded or run by a business oriented individual and why do you think that we need more of them?
    • What are the ways that you typically work to empower engineering founders or encourage them to create their own businesses?
  • What are some of the challenges that engineering founders face and what are some common difficulties or misunderstandings related to business?
    • What are your opinions on venture-backed vs. "lifestyle" or bootstrapped businesses?
  • What are the characteristics of a data business that you look at when evaluating a potential investment?
  • What are some of the current industry trends that you are most excited by?
    • What are some that you find concerning?
  • What are your goals and plans for the future of Data Council?

Contact Info

Parting Question

  • From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat

Links

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Click here to read the raw transcript...
Tobias Macey
0:00:12
Hello, and welcome to the data engineering podcast the show about modern data management. When you're ready to build your next pipeline, I want to test out the project to hear about on the show, you'll need some more to deploy it. So check out our friends over at lead node. With 200 gigabit private networking, scalable shared block storage and the 40 gigabit public network. You've got everything you need to run a fast, reliable and bulletproof data platform. If you need global distribution they've got that coverage to with worldwide data centers, including new ones in Toronto and Mumbai. And for your machine learning workloads. They just announced dedicated CPU instances. Go to data engineering podcast.com slash the node that's LI and OD today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show. And listen, I'm sure you work for a data driven company. Who doesn't these days? Does your company use Amazon redshift? Have you ever grown over slow queries or just afraid that Amazon redshift is going to fall over at some point? Well, you've got to talk to the folks [email protected] they have built the missing Amazon redshift console. It's an amazing analytics product for data engineers to find and rewrite slow queries. And it gives actionable recommendations to optimize data pipelines. We work Postmates and medium or just a few of their customers. Go to data engineering podcast.com slash intermix today and use promo code DEP at sign up to get a $50 discount. And you listen to this show to learn and stay up to date with what's happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. For even more opportunities to meet listen and learn from your peers you don't want to miss out on this year's conference season. We have partnered with organizations such as O'Reilly Media Day diversity Caribbean global intelligence data Council. Upcoming events include the O'Reilly AI conference, the strata data conference, the combined events of the data architecture summit in graph forum, and data Council in Barcelona. Go to data engineering podcast.com slash conferences to learn more about these and other events and to take advantage of our partner discounts to save money when you register today. Your host is Tobias Macey, and today I'm interviewing Pete Soderling about his work to build and grow a community for data professionals with the data council conferences and meetups as well as his experiences as an investor in data oriented companies. And full disclosure that data Council and clubhouse are both previous sponsors of the podcast and clubhouse is one of Pete's companies that he's invested in. So Pete, can you just start by introducing yourself?
Pete Soderling
0:02:44
Yeah, thanks. Thanks for the opportunity to be here, Tobias. I'm Pete Soderling, as you mentioned, and I'm a serial entrepreneur. I'm also a software engineer from the first internet bubble. And I'm super passionate about making the world a better place for other developers. And you remember it
Tobias Macey
0:02:59
you first get involved in the area of data management?
Pete Soderling
0:03:02
Yeah, I think, funnily enough, the thing that jumps out at me is how excited I was when I was a early developer, very young in my career, I discovered this book database designed for Mere Mortals. And I think I read it over my my holiday vacation one year, and I was sort of amazed at myself at how interesting such a potentially dry topic could be. So that was a, an early indicator. I think fast forward. My first company, actually my second company in 2009, that I started was a company called strata security. And originally, we were building what we thought was a API firewall for a web based API's. But it quickly morphed into a platform that's secured and offered premium data from providers like Bloomberg or Garmin, or companies that had lots of interesting proprietary data sets. And our vision was to become essentially like the electricity in between that data provider and their API. And the consumers were consuming that data to the API so that we could offer basically metered billing based on how much data was consumed. So I guess that was my first significant interest as an entrepreneur in the data space back about 10 years or so ago.
Tobias Macey
0:04:20
And now you have become an investor in data oriented companies, you've also started the series of conferences that were previously known as the data edge confident have been rebranded as data Council. And that all started with your work and founding haka Labs is a community space for people working in the data engineering area. And I'm curious what your original reason was for focusing your efforts in that direction and focusing particularly on data engineers?
Pete Soderling
0:04:51
Yeah, I guess it's, it's gets to the core a bit of who I am. And as I've looked back over my shoulder, as both an engineer and a father, I guess what I've realized, which actually, to some extent, surprised me is that all of the companies I've started, have all had software engineers, as end users, or customers. And I discovered that I really am significantly passionate about getting developers together, helping them share knowledge, and helping them with better tooling, and essentially just making the world awesome for them. And it's become a core mission of everything I do. And I think it basically is infused in all these different opportunities that I'm pursuing. Now. For instance, one of my goals is to help 1000 engineers start companies, but not it gets to some of the startup stuff, which is essentially a side project that we can, we can talk about later. But specifically, as it relates to hacker labs, hacker labs was originally started in 2010, to become a community for software engineers. And originally, we thought that we were solving the engineer recruiting problem. So we had various ideas and products that we tested, rounding, introducing engineers to companies in a trusted way. And originally, that was largely for all software engineers everywhere. And our plan was to turn it into a digital platform. So it was going to have social network dynamics where we were connecting engineers together. And those engineers would help each other find the best jobs. So that was had, you know, very much was sort of in the social networking world. But one of our marketing ideas was we wanted to build a series of events surrounding the digital platform, so that we could essentially lead users from the the community in our events, and introduce them to the digital platform, which was the main goal. And one of the niche areas that we wanted to focus on was data, because data science was becoming a hot thing on data engineering was even more nascent. But I sort of saw the writing on the wall and saw that data engineering was going to be required to support all the interesting data science goodness that was being talked about. And really, I was of interest to business. And so you know, pure the data meetups that we started, were essentially a marketing channel for this other startup that I was working on. And ultimately, that startup didn't work. And the product didn't succeed, which is often the case with network based products, especially in the digital world. But I realized that we had built this brand, surrounding awesome content for software engineers, data engineers through our meetups that we had started. And we fell back on that community, and figured out that there must be another way to monetize and to keep the business going. Because I was so passionate about working with engineers and continuing to build this this community that we had seated. And engineers love the brand. They love the events that that we're running, they loved our commitment to deeply technical content. And so one thing led to another and ultimately, those meetups grew into what data Council is today.
Tobias Macey
0:07:48
And you mentioned that when you first began on this journey, it was in the 2010 timeframe. And as you referred to data engineering as a discipline was not very widely be recognized. And I'm wondering how you have seen the overall evolution of that role and responsibility? And what the state of the industry and what the types of recognition and responsibilities were for data engineers at that time.
Pete Soderling
0:08:16
Yeah, you know, data engineering was just not really a thing at the time. And only the largest tech company is Google and Facebook even had the notion of sort of the data engineering concept. And but I guess, you know, what I've learned from, from watching engineering at those companies is that, because of their scale, they discover things more quickly and more often, or earlier than, than other folks tend to. And so I think that was just interesting, you know, leading indicator and so I, I felt like it was going to get bigger and bigger. But yeah, there was no, I don't even know if if Google necessarily had a data engineering job title at that time. So you know, that was just very early in the in the space. And I think we've seen it develop a lot since since then. And it's not just in the title. But I think, you know, we saw early on that data science was a thing and was going to be a bigger thing. Data engineering was required to to have the data scientists and the quants to do awesome stuff in the first place. And then there's also the analysts who are trying to consume the data sets, oftentimes in slightly different ways, and the data science scientists, so I think early on, we saw that these three types of roles were super fundamental and foundational to building the future of data infrastructure and business insights and data driven products. And so even though we started off running the data engineering meetup, which I think we're still known for, we pretty quickly through the conference embraced these other two disciplines as well, because I think the interplay of how these types of people work together inside organizations, is where the really interesting stuff is. And so you, you know, as these these job descriptions, the themes in these job descriptions and sort of how they unite, and how they work together on projects is fascinating. And so, through data Council, our goal has been to further the industry by getting these folks in the same room around content that they all care about. And sometimes it's teaching a data engineer a little bit more about data science, because that's what makes them stronger and better able to operate on a on a multifunctional team. And sometimes it's the data scientists getting a little bit better at some of the engineering disciplines and understanding more what the data engineers have to go through. So I think that looking at this world in a in a cohesive way, especially across these three roles, has really benefited us and made the community event very unique and strong. And now I should say that, I think the next phase of that in terms of team organization, and especially in terms of our vision with data Council is we're now embracing product managers into that group as well. I think that, you know, there's the stack, we sort of see this stuff, lack of data being data infrastructure on the bottom, then data engineering and pipelines, then data science and algorithms, then data analytics on top of that. And finally, there's the the AI features and the people that are weaponized this entire stock into AI products and data driven features. And I think the final icing on the cake, if you will, is creating space for data oriented product managers, because, you know, it used to be that maybe you think of a data Product Manager is like working for Oracle or being in charge of shipping a database. But that's, you know, that's sort of a bit older school at this point. And there's all kinds of other interesting applications of data infrastructure and data technologies that are not directly in the database world, where real world product managers in the modern era, I'm sort of need to understand how to interact with this stack, and then how to build data tooling, whether its internal tooling for developers, or customer consumer facing, beat. So I think embracing the product manager at the top of this dock has been super helpful for our community as well.
Tobias Macey
0:12:07
And I find it interesting that you are bringing in the product manager, because there has long been a division in particularly with technical disciplines where you have historically the software engineer who is at odds with the systems administrator, and then recently, the data scientist or data analyst who is at odds with the data engineer. But there has been an increasing alignment across the business case, and less so in terms of the technical divisions. And I'm curious what your view is in terms of how the product manager fits in that overall shift in perspective, and what their responsibility is within an organizational structure to help with the overall alignment in terms of requirements and vision and product goals between those different technical disciplines?
Pete Soderling
0:12:59
Yeah, well, hey, I think I think this is just a super This is a My question is a microcosm of what's really happening in the engineering world, because I think software engineers at the core at the central location are actually eating disciplines and roles that used to be sort of beneath them and above them. So again, I'm sort of sticky thinking in terms of this vertical stock. But, you know, most modern tech companies don't have DBS. Because the software engineers now the DPA, and many companies don't have designated infrastructure teams, because a software engineer is responsible for their own infrastructure. And some of that is because of cloud or the dynamics, but sort of what's happening is, you know, at its core, the engineer is eating the world. And it's bigger than just software in the world engineers in the world. And so I think the the absorption of some of these older roles into what's now considered core software engineering has happened below, and I think it's happening above. So I think, some product management is collapsing into the world of the software engineer, or engineers are sort of lathering up into product management. And I think part of that is the nature of these deeply technical products that we're building. So I think many engineers make awesome product managers. I mean, perhaps they have to step away and you know, be willing not to write as much code anymore. But because they have an intrinsic understanding of the way these systems are built. I think engineers are just sort of like reaching and, and absorbing a lot of these other roles. And so some of the best product managers that, you know, I think we've seen have been x software engineers. So I just think that there's a real emerging, this is just a larger perception that I have of the world, into the software engineering related disciplines. And I think it's actually not a far leap, to sort of see how, you know, an engineer, a product manager, who's informed with an engineering discipline is super effective in that role. So I just think this is a broader story that we're seeing overall, if that makes sense.
Tobias Macey
0:15:02
Yeah, I definitely agree that there has been a much broader definition of what the responsibilities are for any given engineer, because of the fact that there are a lot more abstractions for us to build off of, and so it empower his engineers to be able to actually have a much greater impact with a similar amount of actual effort, because of the fact that they don't have to worry about everything from the silicon up to the presentation layer, because there are so many different useful pre built capabilities that they can take advantage of and think more in terms of the systems rather than the individual bits and bytes. Yeah, exactly. And in terms of your overall efforts for community building, and community management, there are a number of different sort of focal points for communities to grow around that happen because of different shared interests or shared history. So there are programming language communities, their communities focused on disciplines, such as, you know, front end development, or product management or business. And I'm wondering what your experience has been in terms of how to orient a community focus along the axis of data, given that it can in some ways be fairly nebulous as to what are the common principles in data because there's so many different ways to come at it, and so many different formats that data can take?
Pete Soderling
0:16:31
Yeah, I think the core, you know, one of the core values for us, and I don't know if this is necessarily specific to date or not, but it's just openness. And I think especially, you know, we're, we see ourselves as much, much more than just a conference series, and we use the word community, in our team and at our events, and just to describe what we're doing dozens and dozens of times a week. And so, yeah, I think the community bond and the mentality for our brand is super high. I think that, you know, there's a, there's also an open source sort of commitment. And I think that's a mentality, I think that's a that's a coding, discipline, and style. And I think that, you know, sharing knowledge is just super important in any of these technical fields. And so, engineers are super thirsty for knowledge. And we see ourselves as being a connecting point where engineers and data scientists can come and share knowledge with each other. I think especially maybe that's a little bit accelerated, in the case of data science or AI research, because these things are changing so fast. And so there is a little bit of an accelerator in terms of the way that we're able to see our community grow and the interest in this space, because so much of this technical stuff is changing quickly. And, you know, engineers need a trusted source to come to where they can find and get surface the best, most interesting research and most interesting open source tools. So we've capitalized on that, and, and we try and be an one on one, and we're sort of a media company. You know, on the other hand, we're an events business. On the other hand, we're a community. But we're sort of putting all these two things together in a way that we think benefits careers for engineers, and it enables them to level up in their careers and make them smarter and get better jobs and meet awesome people. So really, all in all, you know, we see ourselves as building this building this awesome talent community, around data and AI, worldwide. And we think we're in a super unique position to do that and
Tobias Macey
0:18:33
succeed at it. community building can be fairly challenging because of the fact that you have so many different disparate experiences and opinions coming together. And sometimes it can work out great. Sometimes you can have issues that come up just due to natural tensions between people interacting in a similar space. And I'm wondering, what you have been drawing on, for instance, and reference for how to foster and grow a healthy community, and any interesting or challenging or unexpected aspects of that overall experience of community management that you've encountered in the process?
Pete Soderling
0:19:13
Yeah, I think it's an awesome question. Because any company that embraces community, to some degree embraces perhaps somewhat of a wild wild west. And I think some companies and brands manage that very heavily top down, and they want to, and they have the resources to, and they're able to some others, I think, let the community sprawl. And, you know, in our particular case, because we're a tiny startup, I used to say that we're three people into PayPal accounts, I'm running events all over the world, you know, even though we're just a touch bigger than that now, not much. But we have 18 meetups all over the world and forming conferences from San Francisco to Singapore. So I think the only way that we've been able to do that, and just to be clear, like we're up for profit business, but I think that's one of the other things that makes us super unique is that, yes, we're for profit. But at the same time, we're embracing a highly principled notion of community. And we use lots and lots of volunteers in order to help you know further that message worldwide, because we can't afford to hire community managers in every single city that we want to operate in. So so that that's one thing. And I guess, for us, we've just had to embrace kind of the the wild nature of what it means to scale a community worldwide and deal with the ups and downs and challenges that come with that. And, of course, there's some brand risk. And there's, you know, other sorts of frustrations, sometimes working with volunteers, but I guess my inspiration, you know, specifically in this was really through through 500 startups, and I went on geeks on a plane back in 2012, I believe. And when I saw the way that 500 startups, which is a startup accelerator in San Francisco, was building community, all around the world, basically one plane at a time. And I saw how kind of wild and crazy that community was, I sort of learned, like the opportunity and the challenge of building community that way. And I think the main thing, you know, if you can embrace the chaos, and if your business situation forces you to embrace the chaos order to scale, I think the main way that you keep that in line is you just have a few really big core values that you talk about, and you emphasize a lot, because basically, the community has to sort of manage itself against those values. And you know, this, this isn't like a detailed, like, heavy takedown process, because you just can't in that scenario. So I think the most important thing is that the community understands the ethos of what you stand for. And that's why with data Council, you know, there's a couple things I already mentioned open source, that's super important to us. And we're always looking for new ways to lift up open source, and to encourage engineers to submit their open source projects for us to promote them. we prioritize open source talks at our conference. You know, that's just one one thing. I think the other thing for data Council is that we've committed to not be an over sponsored brand. This can make it hard economically for us to be able to grow and, and build the to hire the team that we want to sometimes, but we're very careful about the way we integrate partners and sponsors into our events. And we don't want to be, you know, what we see as some of our competitors being sort of over saturated and swarming with salespeople. So there's a few like, Hi thing, I guess the other thing that that's super important for us is we're just deeply, deeply committed to deeply technical content. We screen all of our talks, and we're uncompromising in the in the speakers that we put on stage. And I think all of these things resonate with engineers like I'm, I'm an engineer. And so I know engineers think and I think these three things have differentiated us from a lot of the other conferences and, and events out there, we realized that there was space for this differentiation. And I think all these things resonate with engineers. And now it makes engineers and data scientists around the world want to raise their hands and help us launch meetups, we were in Singapore. Last month, we launched our first data data council conference there, which was amazing. And the whole community came between Australia and India and the whole region, Southeast Asia, they were there. And we left with three or four new meetups in different cities, because people came to the conference saw what we stood for, saw, they were sitting next to awesome people and awesome engineers. And it wasn't just a bunch of suits at a data conference. And they wanted to go home and take a slice of data console back to their city. And so we support them in creating meetups, and we connect them to our community, and we help them find speakers. And it's just been amazing to really see that thrive. And I think like I said, the main the main thing is just knowing the the core ethos of what you stand for. And even in the crazy times, just being consistent about the way you can you communicate that to the community, letting the community run with it and see what happens. And sometimes it's it's messy, and sometimes it's awesome. And but you know, it's a it's an awesome experiment. And I just think it's incredible that a small company like us can have global reach. And it's only because of the awesome volunteers, community organizers, meetup organizers, track host for our conference that we've been able to suck into this into this orbit. And we just want to make the world a better place for them. And they've been super, super helpful and, and kind and supporting us, and we couldn't have done it without them. So it's been an awesome experiment. And, you know, we're continuing to push forward with that model.
Tobias Macey
0:24:33
With so many different participants and the global nature of the community that you're engaging with, there's a lot of opportunity for different surprises to happen. And I'm wondering what have been some of the most interesting or exciting to paraphrase Bob Ross happy accidents that you have encountered in your overall engagement with the community? Hmm,
Pete Soderling
0:24:57
I guess, this wasn't totally unsurprising. But I just love to sort of surround myself with with geeks, you know, geeks have always been my people. And even when I stopped writing code actively, I just gravitated towards software engineers, and obviously, which is why I'm sort of, you know, I do what I do. And it's what makes me tick. I guess one of the interesting thing through running a conference like this is, you get to meet such awesome startups. And there's so many incredible startups being started outside of the valley. You know, I lived in New York City for many years, and I lived in in San Francisco for many years. And now I spend most of my time in Wyoming. So I'm relatively off the map. And one way of thinking but in the other way, you know, as the center of this conference, we just meet so many awesome engineers and startups all over the globe. And I'm really happy to see such awesome ideas start to spring up from other startup ecosystem. So, you know, I don't believe that all the engineering talent should be focused in Silicon Valley, even though it's easy to go there, learn a ton really benefit from that better from the community benefit from the the big companies with scale. But ultimately, I think, you know, not everyone is going to live in the Bay Area, I hope they don't, because it's already getting a little messy there. But I just want to see, you know, all of these things sort of democratize and distributed, both in terms of software engineering, and then the engineers that start these awesome startups. And so, you know, the the ease with which I'm able to meet and see new startups around the globe, to the data, the data council community, I think it's been a real bright spot in that. And I don't know if it was necessarily a surprise, but maybe it's been a surprise to me at how quickly it's happened.
Tobias Macey
0:26:34
So one of the other components to managing communities is the idea of sustainability and the overall approach to governance and management. And I'm wondering both for the larger community aspect, as well as for the conferences and meetup events, how you approach sustainability to ensure that there is some longevity and continuity in the overall growth and interaction with the community?
Pete Soderling
0:27:02
Yeah, I think I think the main thing, you know, this gets back to another core tenet of sort of the psychographic of a software engineer, software engineers need to know how things work. And that's sort of the core of our mentality in building things. We want to know how things work, if we didn't build it ourselves. We prefer to like, rip off the covers and understand how it works. And you know, to be honest, part of the way that for instance, we select talks at our conference, you know, I think this applies to and we're learning to get better about. I mean, I think as a as a value, we believe in openness and transparency. In our company, I think externally facing, we're getting better about how we actually enable that with the community. But for instance, for our next data council conference that's coming up in New York, and in November, we've published all of our track summaries on GitHub, and we've opened that up to the community where they can contribute ideas, questions, maybe even speakers, theme sub themes, etc. And I think just the nature that, you know, we have the culture to start to plan, our events like this in the open, I think, brings a lot more transparency. And then I guess the other thing about a community that's just sort of inherent, I think, in a well run community, is the amount of diversity you get. And obviously, you know, we're all aware of that, that software engineering as a discipline, is just suffering from a shortage of diversity in certain areas. And I think as we commit to that, locally, regionally, globally, there's so many types of different diversity that we get through the event. So I think both of these things are, you know, are super meaningful in like keeping the momentum of that community moving forward, because we want to continue to grow. And we want to continue to grow by welcoming folks that maybe necessarily didn't necessarily previously identify with the data engineering space, you know, into the community so that they can see what it's like and evaluative if they want to take a run that in their career. So I think all these things, transparency, openness, diversity, these are all Hallmark hallmarks of a great community and, and these are the engines that keep the community going and moving forward. Sometimes in spite of the resources or the lack of resources, you know, that a company like data council itself, can muster at any one time.
Tobias Macey
0:29:22
In terms of the conference itself, the tagline that you focused on, and we've talked a little bit about this already, is that they are no fluff, paraphrasing your specific wording, and as a way of juxtaposing them against some of the larger events that are a little bit more business oriented, not calling out any specific names. And I'm wondering what your guidelines are for fulfilling that promise, and why you think it is an important distinction? And conversely, what some of the benefits are for those larger organizations, and how the two scales of event play off each other to help foster the overall community?
Pete Soderling
0:30:02
Yeah, well, one, one thing here is, I think, comes to the mentality of the engineer. And then the other side of it is the mentality of the sponsor and the partner. And, you know, hey, I think engineers are just Noble. And like I said, engineers want transparency, they want to know how things work. They don't want to be oversold to, you know, they want to try product for the self. There's just all of these sort of things baked into the engineering mindset. And first and foremost, we want to be known as the conference in the community that respects that, like, that's the main thing, because engineers like without engineers, and our community, loving and getting to know each other, we're not careful about the opportunities in the context that we create for them, they're just going to run in the other direction. And so like, first and foremost, like those are the hallmarks of of what we're building from the engineering side. Then on the partnership side, I think companies are not great at understanding how engineers think recruiters are not great at talking to engineers, marketers are not great at talking to engineers. Yes, engineers need jobs. And yes, engineers need new products and tools, but to find companies that actually know how to respect the mental hurdles that engineers have to get through, in order to like get interested in your product or get interested in your job. You know, that's a super significant thing. And through my years of working in the space, I've done a lot of coaching and consulting with CTOs, specifically surrounding these two things, recruiting and, and marketing to engineers. And I think that awesome companies who respect the the central place that engineers have, and will continue to have in the innovation economy that's coming, realize that they have to change their tune in the way they approach these engineers. So I, you know, our conference platform is a mechanism that we can use to gently sort of steer and even teach some partners how to interact with engineers in a way that doesn't scare them away. And so just broadly speaking, like I mentioned, we're just super careful about how we integrate partners with our event. And we're always as a team trying to come up with, with better ways to message this and, and better ways to educate and, and sort of welcome sponsors, you know, into the special, the special network that we've built, but it's challenging, you know, like not not all marketers think alike. And not all marketers know how to talk to engineers, but we're committed to creating a new opportunity for them to engage with awesome data scientists and data engineers in a way that's valuable for both of them. And that's a really fun, big challenge. And, you know, we're not as worried about how much as it scales right now, as much as we were the quality, enhancing the quality of those interactions. And so that's what we're committed to as a brand. And, you know, it's not always easy. But we've we learned a lot, and we have a lot to learn. And we always sort of touch touch base with the community after the events and sort of asked the community what they thought and how they interact with the partners, then did they find a new job? And how did that happen? And so we're always trying to pour gasoline on what works, not respecting continue to innovate and move forward in that way,
Tobias Macey
0:33:03
in addition to your work on building these events, and growing the meetups, and overall network opportunities for people in the engineering space, you have also become an involved as an investor. And you've mentioned that you focus on data oriented businesses. And I'm curious how you ended up getting involved in that overall endeavor, and how that fits into your work with data Council and what your overall personal mission is. Oh, yeah.
Pete Soderling
0:33:30
Well, that's, that's definitely one of my side projects. As I mentioned, I want to help 1000 engineers start companies, and, you know, this is just part of what makes me tick, just helping software engineers through the conference, through advising their startups, you know, through investing through helping them figure out go to market, I guess a lot of this, this energy for me came, you know, from having started for companies, myself, and as an engineer who didn't go to school, but instead opted to start, you know, my first company in New York City in 2003. You know, there weren't a lot of people that had started three companies in New York by the time, the early, you know, sort of, or the layoffs came around. So yeah, I guess I've learned a lot of things the hard way. And I think a lot of engineers are kind of self taught. And they also learn things, they tend to learn things the hard way. So I guess, a lot of my passion there is again, sort of meeting engineers, where they're at how they learn. And you know, to them, like, I'm kind of a business guy. Now, I have experience with starting companies, building products, fundraising, marketing, building sales teams, and you know, most of those things are not necessarily been Top of Mind, for many software engineers that want to start a company, they have a great idea. They're totally capable of engineering it and building a product, but they need help, and all the other, you know, software, businesses, LZ stuff, as well as fundraising. So I guess I just figured, I've discovered the sort of special place I have in the world where I'm able to help them coach them through a lot of those businesses is I could never build the infrastructure that they're building or figure out the the algorithms or the models that they're building, but I can help them figure out how to pitch it to investors, how to pitch it to customers, how to go to market, how to hire a team that scales. And so I discovered that I just had, you know, through my ups and downs, as an entrepreneur, I've developed a large set of early stage hustle, experience, and I'm just super hot, happy to pass that on to other engineers who are also passionate about starting companies. So that's just something I find myself doing anyway, you know, as a mentor for 500 startups or as a mentor for other companies. And one thing led to another and soon I started to do angel investing. And now I have an Angeles syndicate, which is actually quite interesting, because it's backed by a bunch of CTOs and heads of data science and engineers from our community, who all co invest with me. And as I'm able to help companies bring their products to market startups come to market, oftentimes will be an investment opportunity there. And so I'll be another network of technical people who add value to that company even more. So I'm just sort of the, you know, a connector in this community. And the community is doing all kinds of awesome stuff, you know, inside and even sometimes outside of data console, which is just a testament to the power of community overall. And, you know, I just happened to be, I'm super grateful that I'm along for the ride. And I got to I got engineers who come to me and trust me for help. And I'm able to connect these dots and and help them succeed as well,
Tobias Macey
0:36:30
in terms of the ways that businesses are structured. I'm wondering what it is about companies that are founded by engineers and led by engineers that makes them stand out, and why you think that it's important that engineers start their own companies, and how that compares to businesses that are founded and run by people who are coming more from the business side. And just your overall experience in terms of the types of focus and levels of success that two different sort of founding patterns end up playing out?
Pete Soderling
0:37:03
Yeah, well, yeah. I mean, you can tell based on what I've been saying that I'm just super bullish on the software engineer. And, you know, does that mean that the software engineer as a persona or a mentality or a skill set, you know, is inherently awesome and has no weaknesses? And no problems? Like hell? No, of course not. And I think the some of the challenges of being a software engineer and how your mentality fits into the rest of the business are well documented. So I think all of us as engineers need to grow and diversify and increase the breadth of our skills. And so that has to happen. But on the other hand, if we believe that innovation is going to continue to sweep the world, and highly technical solutions, perhaps to sometimes non technical problems, perhaps sometimes the technical problems are going to continue to emerge. I feel like people who have the understanding of the the technical implications and the realities and the architectures and the science of those solutions, just have an interesting edge. So I think there's a lot of hope in teaching software engineers how to be great CEOs. And I think that's, that's increasingly happening. I mean, look at Jeff Lawson from Twilio. Or the guys from stripe, even Uber was started by an engineer, right? There was the the the quiet engineer at goober at Uber, Garrett, sort of quiet in terms of Travis, you know, who, who was a co founder of that company. So I think we're seeing engineers, not just start the most amazing companies that are that are changing the world, but they're increasingly in positions of becoming CEOs. And those companies, you know, I guess you might even take that one step further. And I'm kind of trying to be an engineer who's also been an operator and a founder. But now I'm, I'm stepping up to becoming a VC and, and being an investor. So I think there's the engineer VC, which is really interesting, as well. But I think that's a slightly different conversation. But but suffice it to say that engineers are bringing a valid mentality into all of these disciplines. And yes, of course, an engineer has to be taught to think like a CEO, and has to learn how to be strategic and has to learn sales and marketing skills. But I think it's just an awesome, awesome challenge to be able to layer those skills on top of engineering discipline that they already have. And I'm not saying this is the only way to start a company or that business people, you know, can't find awesome engineers to partner with them. I mean, honestly, I think an engineer often needs a business co founder, to help get things going. But I I'm coming at it from the engineering side, and then figuring out like, all the other support that the engineer needs to make a successful company, and that's just because I've chosen that particular way, but other people will be coming at it from the business side, and, and I'm sure that will be fine for them as well,
Tobias Macey
0:39:47
in terms of the challenges that are often faced for an engineering founder in growing a business and making it viable. What are some of the common points of conflict or misunderstandings or challenges that they encounter? And what are some of the ways that you typically work to help them in establishing the business and setting out on a path to success? Well, I
Pete Soderling
0:40:10
think the the biggest thing, you know, that I see is, many engineers are just afraid to sell. And unfortunately, you know, you can't have a business if you can't have some sales. And so somehow, engineers have to get over that hurdle. And that can be a long process. It's been a long process for me. And I still undersell what we do at data council to be honest, in some ways, and I have people around me to help me do that. And we want to do that, again, in a way that's in line with the community. But I'm constantly trying to figure out how to be essentially a better salesperson. But for me, that means that still retaining sticking to the core of my engineering values, which is honesty, transparency, enthusiasm, you know, value and really understanding how to articulate the value of what you're bringing in a way that's, that's unabashedly honest and transparent. So I think that's a, that's a really big thing for a pure engineer founder is, it can be difficult to go out there and figure out how to get your first customers, you know, how to start to sell yourself personally. And then the next step is how do you build a sales culture and a process and a team that's in line with your values as a company, and that scares, you know, that scares some engineer, because it's just terrifying to think about building a sales org, when you can barely accept that your product needs to be sold yourself. But I think that's just a you know, that's sort of ground zero for starting a company. And so, you know, I try and be as gentle as possible and, and sort of guiding engineers through that process. But I guess that's the one of the core hiccups that I think engineers have to figure out how to get over by bringing in other people they trust or getting advice, or, you know, you have to approach it sometimes with kid gloves, but teaching engineers how to sell in a way that's true to their values. I think it's just a really big, big elephant in this room that, you know, I constantly run into and, and try to help engineer founders with
Tobias Macey
0:42:08
in terms of the businesses that you evaluate for investing in what is your strategy for finding the opportunities? And what are the characteristics of a business that you look for when deciding whether or not to invest? And then as a corollary to all of that? Why is it that you're focusing primarily on businesses that are focused mainly in the data space, and the types of services that you're looking for, that you think will be successful and useful?
Pete Soderling
0:42:37
Yeah. Well, I guess last question. First, I think, you know, it's important to have focus as an investor, and not everybody can do everything awesome. I think it's also a strategy to building a new brand, and a niche fund in the face of the sequoias and the corners of the world. I think we might have like we might be past the day is when a new fund can easily build a brand. That's, that's that expansive. So I think this is just kind of, you know, typical marketing strategy. I think if you start to focus on a niche and do one niche really well, I think that produces network effects, smaller network effects inside that niche that then grow and grow and grow and develop. So I mean, I've chosen to focus in my professional life, on data, data, data science, data engineering, data analytics, that's data console. Partly that's because I just believe in the upside of that market. So I think that's just a natural analogue to a lot of my investing activity, because I'm knowledgeable about the space because I have a huge network in the space, because I'm looking at interested in talking to these companies all the time. Um, it's just a natural fit. That's not to say that I don't I mean, I'm also passionate about broader developer tools. And as you mentioned earlier, I'm an investor in clubhouse. I'm interested in security companies. So I think, you know, for me, there are some analogues to just like, deeply technical companies, you know, look by technical founders, that that also fit my thesis. But, you know, still it's a fairly niche, narrow thesis, like most of the stuff I do. On the investing side is b2b, I meet the companies through my network and, and through data Council, I think they're solving you know, meaningful problems in the b2b space. And other criteria often is, they may be supported by some previous open source success, or the company might be built on some current open source project that I feel gives them an unfair advantage when it comes to distribution, and getting a community excited about the product. So these are a few of the things that I look for, in terms of the investing thesis
Tobias Macey
0:44:39
in terms of the overall industry and some of the trends that you're observing and your interaction with engineers and with vetting the different conference presentations and meetup topics, what are some of the trends that you're most excited by? And what are some of the ones that you are either concerned by or some potential issues that you see coming down the road in terms of the overall direction that we are heading as far as challenges that might be lurking around the corner?
Pete Soderling
0:45:10
Well, I think, you know, one big thing there is just data science and ethics, ai fairness, bias in AI, and in deep learning models, ethics, when it comes to, you know, targeting customers, what data you keep on people, like I think all these things are just super interesting is business issues, their policy issues or business issues. At one level, they're also technical issues. So there's technical implementation stuff that's required. But I just think raising that discussion is important. And so that's one area that we're focusing on, and data Council in the next series of events that we run later this year. And next year, is elevating that content in front of our community so that it can be a matter of discussion, because I think those are important topics are always seen as the most technical. But I think they're super important in terms of us, trying to help the community steer and figure out where the ship is going in the future. So I think that's super interesting. And then, I guess on the technical side, I think the data ops world is starting to mature. I think that there's a lot of interesting opportunities there in the same way that the DevOps revolution, you know, revolutionized the way that software was built, tested, deployed, monitored, and companies like chef and New Relic, you know, came came out in perhaps the mid 2000s. I think we're at a similar inflection point, with data Ops, if you will. And there's more repeatable pipeline process. There's back testing and an auditing capabilities that are required for data pipelines that aren't always there. There's monitoring infrastructure that's being built, and some some cool companies that I've seen recently. So I think data Ops, and basically just elevating data engineering to the level that software engineering has been out for a while. It's definitely something that seems to be catching fire. And we also, you know, try and support to the conference as well.
Tobias Macey
0:47:06
Looking forward, what are some of the goals and plans that you have for the future of the data council business and the overall community and events?
Pete Soderling
0:47:17
Well, I think, as I mentioned, our biggest goal is to build the data and AI talent community worldwide. And I think that there's, we're building a network business. So I guess it kind of takes me back to when I started, hacker labs, which was the digital social network, and I thought we were building a digital product. And as I already mentioned, one thing led to another and now we have data console instead. Well, data console is, you know, butts and seats at events and community. And it's IRL, engineers getting together. But it's still a network. It's not a super fast digital, formal network. But it's a network. And it's a super meaningful network. So it's kind of interesting that after all, the ups and downs, like I still see ourselves as in the network building business. And I think the cool thing about building a network is once you build a network, there's lots of different value that you can add on it. So I think in terms of there's, there's really big ideas here, there's formalizing education and education offerings, there's consulting services that can be built on top of this network, where companies can help each other, or recruiting sort of fits in the same vein, I think there's there's other things as well, there's a fund, which I have mentioned is a side project that I have to help engineers in this community start companies. So I think there's, there's all kinds of interesting things that you can build on top of a network, once you've gotten there. And for now, you know, our events are essentially a breakeven business that just gives us an excuse, and the ability to grow this network, all around the world globally. But I think, you know, there's a much bigger, like phase two, or phase three of this company where we can build really awesome stuff based in this engineering network, and network and a brand that engineers trust that we've laid down in the in the early part of the building phase. So I'm really excited to see that and, and develop that strategy and mission going forward.
Tobias Macey
0:49:14
Are there any other aspects of your work on data council or your investment, or your overall efforts in the data space that we didn't discuss yet that you'd like to cover before we close out the show?
Pete Soderling
0:49:26
No, I think I think we covered a lot of stuff. I hope it was interesting for you and your audience. And I encourage folks to reach out to me and, and get in touch. If there's engineers out there that want to start companies. If there's engineers that want to participate in our community worldwide, we're always looking for awesome people to help us find screen talks. We're interested in awesome speakers as well. I'm always interested in talking to deep learning and AI researchers who are out there who might have ideas that they want to bring it to market. But yeah, you can reach me at Pete data council dot A. And I'm happy to plug you into our community. And yeah, if I can be helpful to anyone out there, I would just really encourage them to reach out.
Tobias Macey
0:50:09
And for anyone who does want to follow up with you, or keep in touch or follow along with the work that you're doing. I'll have your contact information in the show notes. And as a final question, I just like to get your perspective on what you see as being the biggest gap and the tooling or technology that's available for data management today.
Pete Soderling
0:50:26
Yeah, I think, as I mentioned, I think maybe it comes down to this, this data ops thing, right? There's there's really interesting open source projects coming out, like Great Expectations is one interesting companies coming out like elemental, which is built around Dexter, which is an open source project. So I think that this is a really interesting niche sort of tooling area, specifically in the data engineering world that I think we should all be watching. And then I guess the other sort of category of tooling, I'm seeing it sort of related. It's also in the monitoring space, it's watching the data in your data warehouse, to see if there's anomalies that sort of pop up, because we're all we're pulling together data from so many different hundreds of sources now that I think it's a little bit tricky to watch for data quality, and integrity. And so I think there's a new suite of tools that are popping up in that data monitoring, um, space, which are very interesting. So those are a couple of areas that I'm interested that I'm interested in and looking at, especially when it comes to data engineering applications.
Tobias Macey
0:51:26
Well, thank you very much for taking the time today to share your experiences and building and growing these events series and contributing to the overall data community as well as your efforts on the investment and business side. So definitely an area that I find valuable and I've been keeping an eye on your conferences. There's been a lot of great talks that come out of it. So I appreciate all of your work on that front, and I hope you enjoy the rest of your day.
Pete Soderling
0:51:50
Yeah, thanks, Tobias. We'll see you at the data council conference sometime soon.
Tobias Macey
0:51:59
Listening, don't forget to check out our other show podcast.in it at python podcast.com. To learn about the Python language its community in the innovative ways it is being used. visit the site at data engineering podcasts. com to subscribe to the show, sign up for the mailing list and read the show notes. If you've learned something or tried other projects from the show, then tell us about it. Email hosts FD to engineering podcast.com with your story, and to help other people find the show. Please leave a review on iTunes and tell your friends and co workers