<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pete Soderling on Medium]]></title>
        <description><![CDATA[Stories by Pete Soderling on Medium]]></description>
        <link>https://medium.com/@petesoder?source=rss-e038f19de4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*E4FYPEUJJLthjoK7GP-c3g.jpeg</url>
            <title>Stories by Pete Soderling on Medium</title>
            <link>https://medium.com/@petesoder?source=rss-e038f19de4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:00:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@petesoder/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hiring: Head of Platform & Community]]></title>
            <link>https://petesoder.medium.com/hiring-head-of-platform-community-368b35179aa0?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/368b35179aa0</guid>
            <category><![CDATA[venture-capital]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Thu, 15 Apr 2021 14:46:04 GMT</pubDate>
            <atom:updated>2021-11-17T20:48:55.747Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*27xrVTUFocGoTq8zwei4Ow.jpeg" /></figure><p>Data Community Fund is hiring a <strong>Head of Platform &amp; Community</strong> to be responsible for all network building efforts across our fast growing firm. We’re a bold new venture firm backed by top funds such as <strong>Sequoia, Bain Capital Ventures &amp; AngelList</strong> and we’re completely rethinking venture from the “outside in” to focus on building a strong community of technical professionals, mentors, founders, and go-to-market experts as our core differentiator.</p><p>Data technologies are rapidly changing the world, and our community is at the epicenter of the hottest technical advancements and research in machine learning, computer vision, artificial intelligence, neural networks and deep learning. These technologies are literally transforming industries while simultaneously creating new opportunities and defining our future.</p><p>Money is cheap, experience isn’t. This is why we built our community first — to give technical founders launching B2B data-oriented startups an unfair advantage to succeed. Based on the immensely popular <a href="http://datacouncil.ai/">Data Council</a> conference series, we provide the natural platform for technical founders to find early design partners &amp; customers, hire technical talent, and network with other founders who have been through similar early stage challenges.</p><p>This network is our not-so-secret weapon — so far we’ve quietly backed more than 40 top B2B data &amp; infrastructure startups such as <strong>Superconductive (Great Expectations), EraDB, Hex, Hightouch, Acryl Data (Datahub), Wasmer, Truera, Privacy Dynamics, Bit.io, Watchful</strong> and many others.</p><p>We’re on a mission to build a breakout venture firm, and are building out our platform team to achieve our vision.</p><p><strong>The Role</strong></p><p>The Head of Platform &amp; Community is responsible to design and scale our platform strategy. This includes events &amp; conferences, content creation, designing portfolio support functions and building internal &amp; external tooling. This is a generalist role, and if you’re a previous startup founder or early employee who’s been through the fire, baked a brand, built community, connected people, handled business development, tamed scale and the multitude of other similar responsibilities that come from working at startups, we’d love to talk to you.</p><p>We believe that our core differentiation lies in how we build our platform and the community around it. Where some VCs start with a fund and try to “bolt on” a community, we started with community first (no, really) and are building out our fund from there. This gives us a first-principles approach to building a new kind of venture fund, and you’ll experience it directly when designing and building the platform features that are direct manifestations of our core values.</p><p><strong>About You</strong></p><ul><li>Have 5+ years experience in a related role (community, marketing, events, ops, etc.)</li><li>Passionate about data technologies, machine learning, AI and the positive impact it can have on our world</li><li>Have previous experience running events at a meaningful scale, managing operations for a team or startup, or have stories of similarly rigorous execution performance</li><li>Are excited to help build a sourcing strategy of the best AI/data startups at their earliest stages</li><li>Have previous experience building communities, especially in technical fields like open-source software (OSS) or related areas like developer marketing &amp; evangelism</li><li>Possess experience with high-quality content creation (newsletters, podcasts, talks, online events, etc.)</li><li>Understand how challenging startups can be — your personal experience of the pain gives you empathy and drives you to build network-based solutions to help other founders</li></ul><p><strong>Our Team &amp; Approach</strong></p><p>Our team is fully distributed and embraces the remote work lifestyle. You’ll have the freedom to work from your desired location while staying connected with the team and portfolio companies via Data Council, our portfolio events series, and by building out our community in the U.S. and Europe.</p><p>If you’re excited to join an inspired team that’s rethinking the venture firm of the future to better serve deeply technical founders, please reach out to me via a shared connection.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=368b35179aa0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are the most popular OSS data projects of 2021?]]></title>
            <link>https://petesoder.medium.com/what-are-the-most-popular-oss-data-projects-of-2021-84ef021bb5a2?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/84ef021bb5a2</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Sun, 04 Apr 2021 18:10:05 GMT</pubDate>
            <atom:updated>2021-04-04T18:10:05.395Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kewz0xLvwocuc8INzphzmA.jpeg" /></figure><p>We successfully ran our Data Council 2021 OSS Data Tools Community Survey in February, and were thrilled with the response from the community. We received more than 500 individual responses with 1,133 tool entries from more than a dozen countries.</p><p>By our count, there were 214 different tools mentioned. For me, this was one of the largest confirmations that we’re still in a time of massive innovation at all layers of the data stack. (Might be time for another one of <a href="https://towardsdatascience.com/25-hot-new-data-tools-and-what-they-dont-do-31bf23bd8e56">these</a> <a href="https://towardsdatascience.com/20-more-hot-data-tools-and-what-they-dont-do-46bc365bea74">posts</a>!)</p><p>A couple notes on how we cleaned &amp; interpret the data:</p><ol><li>We attempted to normalize the project names for accuracy wherever possible. However, in limited cases respondents mentioned a commercial company name vs. a specific OSS project. If there is only one known OSS project associated with that company, we normalized that vote into a vote for the OSS project (i.e. ‘Coiled’ became ‘Dask’). However, if there were multiple OSS projects related to the company (i.e. a vote for “Databricks”, which is ambiguous as it could refer to Spark or Delta Lake, for example) we left the data as-is since these cases were limited.</li><li>If you look through the summarized results (or the <a href="https://gist.github.com/petesoder/625f4e17a80ba6f9ce1c74c1bec5844a">raw data here</a>), you’ll note that in the long tail of projects that received a single vote there are some that aren’t necessarily pure data projects (hello Java &amp; Github!). To us, it didn’t make sense to prune these out, as they didn’t skew the main results in any way. We kept them in to reflect the organic output of the community.</li><li>Regardless of point #2, the majority of long tail votes were for projects that are specifically focused on working with data — and some are even large and entrenched projects (like Neo4j, Redis &amp; Keras), though they didn’t receive significant attention in the survey from our community.</li></ol><p>By order of popularity, the <strong>Top 20 OSS data tools</strong> are listed here:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0a0d2cdd85618dddba50c27ab09031e1/href">https://medium.com/media/0a0d2cdd85618dddba50c27ab09031e1/href</a></iframe><p>Thank you to all who participated, shared, tweeted — and to those OSS authors and community leaders who cajoled your networks to do the same. If you’re interested in the full data you can check out the <a href="https://gist.github.com/petesoder/fb39536a8c17e0f319a3eca2c4866c92">summary data</a> or the <a href="https://gist.github.com/petesoder/625f4e17a80ba6f9ce1c74c1bec5844a">raw data</a>.</p><p>Footnote: while our survey focused purely on open-source data tools, our friends at Kaggle also produce an awesome State of Data Science &amp; ML survey each year that covers a broader range of topics &amp; themes. (If you’re curious, we suggest you check out their <a href="https://www.kaggle.com/kaggle-survey-2020">2020 edition</a>.)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84ef021bb5a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Investing in the Next 1,000 Engineer-Founders]]></title>
            <link>https://medium.com/swlh/investing-in-the-next-1-000-engineer-founders-69fb74193b36?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/69fb74193b36</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[investing]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Tue, 01 Dec 2020 17:31:34 GMT</pubDate>
            <atom:updated>2021-01-04T20:22:31.514Z</atom:updated>
            <content:encoded><![CDATA[<h4>Startups / Venture Capital</h4><h4>My journey into founding, supporting and funding engineer-led startups</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*EegUNqF_i82xnE49_42zCA.gif" /><figcaption>Sometimes starting a company as a technical person feels like the worst kind of trial and error</figcaption></figure><p>It was late 2010 and I had jumped on a flight from NYC to SF. Though I had made the trip many times before, this time my ticket was one-way.</p><p>In the air, I remembered I didn’t have a place to stay that night, so I opened my laptop mid-flight and jumped on Airbnb to see what types of listings were available in SF. Earlier that day I had turned over the keys for my NYC apartment to a nice couple from Germany — my first experience as an Airbnb host. “Why not complete the loop the same day and become a guest, too,” I thought. In ten minutes I had found and successfully booked a small place in a central neighborhood of SF that I could afford using the revenue from my rented NYC place. Founder arbitrage FTW!</p><p>As any startup founder will recognize, I was in full hustle mode. This one-way trip to SF was to take a final swing at saving my second startup, an early cloud-based API management &amp; security product that seemed to be ahead of its time. As a security-minded engineer who had conceived of the idea, I was convinced that security was a crucial, yet oft-overlooked, aspect of modern application and API development. This, I reasoned, coupled with the birth of the cloud, was creating an entirely new set of vectors for attackers.</p><p>But so far, my baby company had struggled. While we had successfully built our product, we had been having a hard time finding early customers and investors in NYC. I hoped that by spending several months in SF and pounding the pavement in person, I might be able to find more innovative partners ready to try our solution and become customers, or network with a more savvy group of tech-minded investors to extend our runway.</p><h3>Helping Engineers Thrive</h3><p>My experience as an engineer-founder struggling to bring a product to market (and even moving to the Bay Area to do it) might be familiar to other engineers who have found themselves in a similar situation. In the years of starting my own companies, as well as coaching hundreds of other engineer-founders, I’ve discovered that technical people like myself, while having strong engineering skills and product development sensibilities, sometimes agonize when it comes to other aspects of getting their startup off the ground — namely with sales, marketing and pitching (customers OR investors), and sometimes with softer skills like communication or team building &amp; leadership.</p><p>I don’t want to let that happen, because I strongly believe that the keys to the most significant innovations in our future are held by deeply technical founders, especially those who are harnessing and manipulating data in new and significant ways — including software engineers, data scientists and AI researchers. But perhaps unsurprisingly, these are also the same types of professionals who seem unlikely to apply to business school or risk stepping away from the fast-developing engineering &amp; AI world to get an MBA lest their technical skills become outdated and eclipsed by someone else. So what is the solution to this “geeks vs. suits” conundrum?</p><p>My own path to helping engineer-founders successfully start companies started shortly after my fateful flight to SF in 2010. Even though that particular startup didn’t succeed, I came out of the fire with a whole new set of experiences; I had first-hand lessons about mistakes to avoid, as well as a clearer head around what had worked. Because you can’t learn them in a book, it turns out those lessons in the startup world are actually like gems.</p><p>What happened next was remarkable — based on my insights I quickly discovered ample opportunity to share key advice with technical friends regarding their startups. And I was thrilled to see how they benefited from it when they raised their first round of funding, made early product strategy decisions, or shared the story of their startup to successfully close new hires.</p><p>Soon, the opportunity to work with more founders grew, and in addition to offering mentorship to dozens of founding teams, I found myself occasionally writing small checks into the earliest stages of their startups. I had become an advisor and angel investor. And that was just the beginning.</p><p><strong>Over the past several years, in support of other engineer-founders, I’ve quietly made 50+ investments in deeply technical startups founded by amazing engineers.</strong> I’ve been fortunate enough to support them both with time and with capital, and I’ve discovered my calling for the second half of my career.</p><h3>Passion + Experience = Mission</h3><p>Why take things to the next level now? There are 3 reasons:</p><ol><li>After more than 20 years of being both an engineer &amp; repeat founder, I fully accepted my passion for innovation via early stage, engineering-led startups and clearly established one of my life goals: to help 1,000 engineers start companies.</li><li>I also realized that as a founder who had figured out many things about startups the hard way, I had a lot of helpful advice to share with other engineer-founders about starting &amp; building their companies.</li><li>Finally, <a href="http://datacouncil.ai/">Data Council</a>, the event &amp; community for data engineers, scientists and AI researchers that I founded in 2015, was producing thousands of connections with top technical people who were rapidly launching data-oriented startups thanks to their unique expertise. I had, almost unwittingly, created a community to discover the best data startups at scale <em>even before they started</em>.</li></ol><p>So, in the spirit of “pouring gasoline on what’s working” I decided to honor what was already working and moved forward to crystalize the advantages I had discovered in both finding and helping the earliest-stage engineering-led data-oriented startups.</p><h3>New Vehicle, Same Values</h3><p><strong>As such, I’m excited to announce the launch of my first venture fund, the</strong> <a href="http://dcf1.vc/"><strong>Data Community Fund</strong></a>. Backed by a “who’s who” list of data executives, technical founders and enterprise VCs, DCF is a micro-fund dedicated to helping the earliest stage (pre-product, pre-revenue) engineering-led B2B startups that have significant differentiation via applications of innovative data technology. This includes all manner of data infrastructure, tooling for data science/machine learning/deep learning/data analytics, as well as applications of AI to specific business verticals (healthcare, manufacturing, insurance, finance, etc.).</p><p>Via our deeply-technical community, we are focused not just on offering advice and capital, but also true support and company-building help to engineers launching companies at the earliest stages of their startup journey. Specifically, we help with finding a co-founder, hiring, connections to technical experts, finding &amp; speaking with early customers, go-to-market strategy, pitching VCs/fundraising. And with our deep experience in the data ecosystem we have a unique perspective on promoting and commercializing open-source projects.</p><p>We rank high on the “helpfulness” side of the “Check Size to Helpfulness” ratio — meaning that because we’re founders and operators first we <em>always</em> have our founders’ backs, and offer them extraordinary support during the earliest stages of their company launch and building.</p><h3></h3><p>The CS:H ratio is a metric to watch as more operators start their own funds1. Access to highly relevant network2. Trusted relationships to close candidates3. Operating experience aligned to current co. needsThe best early investors operate as an extension of your lean team https://t.co/WNEUFGqxR7</p><p>The Data Community Fund invests in 15–20 companies/year across North America and the E.U., sourced from the network of deals we discover thanks to our community, co-investor relationships and referrals from industry &amp; personal networks.</p><h3>Gearing up for 2021</h3><p>I personally couldn’t be more excited to share a positive update with the community near the end of 2020. Though it’s been a challenging year for us all globally, inspiration and innovation will always find ways to break through.</p><p>And for all the engineers reading this who are thinking about starting companies, I urge you to<a href="https://twitter.com/petesoder"> tell me about your vision</a> and to tap into the immense network of support available to you. Gone are the days where engineers need to feel like they’re fighting alone to bring the earliest versions of their data-oriented innovations to life.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69fb74193b36" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/investing-in-the-next-1-000-engineer-founders-69fb74193b36">Investing in the Next 1,000 Engineer-Founders</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[21 MORE Hot Data Tools and What They Don’t Do]]></title>
            <link>https://medium.com/data-science/20-more-hot-data-tools-and-what-they-dont-do-46bc365bea74?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/46bc365bea74</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Tue, 25 Aug 2020 02:01:29 GMT</pubDate>
            <atom:updated>2020-09-11T15:53:18.411Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c2dddVdSUGYXLl5apyqtwA.jpeg" /><figcaption>Photo via Unsplash</figcaption></figure><p>In the past few months, the data ecosystem has continued to burgeon as some parts of the stack consolidate and as new challenges arise. Our first attempt to help stakeholders navigate this ecosystem highlighted <a href="https://towardsdatascience.com/25-hot-new-data-tools-and-what-they-dont-do-31bf23bd8e56?source=friends_link&amp;sk=26a52764a670df4a87914d5e8d88bc59">25 Hot New Data Tools and What They DON’T Do</a> — clarifying specific problems the featured companies and projects did and did NOT solve.</p><p>This effort was positively received by the data science, engineering and analytics communities, and spurred more engagement than we originally anticipated. Further, we were flattered to see the original post motivate other thought-provoking pieces such as <a href="https://medium.com/work-bench/20-hot-new-data-tools-and-their-early-go-to-market-strategies-47e93a554e0b">20 Hot New Data Tools and their Early Go-to-Market Strategies</a>.</p><h3>Taking it Further</h3><p>Regardless, we quickly recognized our original post did not go far enough as we received dozens of emails, Twitter messages and Slack DMs about other solutions that were not covered. We had shed light on a small corner of the expanding universe of data tools and platforms, yet there was an opportunity to cover even more.</p><p>Although we cannot chronicle every additional data tool in just one follow-up post, here we continue our efforts to cultivate this ecosystem by highlighting a few more. The creators of these tools are not only occupying meaningful parts of the ever-evolving modern data stack, they graciously responded to our requests to help us understand where they fit in.</p><p>They sound-off here in their own words.</p><h3>More Tools and Responses</h3><ol><li><a href="http://www.shipyardapp.com/">Shipyard</a>: Shipyard is a workflow orchestration platform that helps teams quickly launch, monitor, and share data solutions without worrying about infrastructure management. It lets users create reusable blueprints, share data seamlessly between jobs, and run code without any proprietary setup, all while scaling resources dynamically. Shipyard is NOT a no-code tool and does not support data versioning or data visualization.</li><li><a href="http://count.co/">Count</a>: Count is a data notebook that replaces dashboards for reporting and self-service, and supports data transformation. Count is uniquely good at team collaboration, enabling technical and non-technical users to work within the same notebook. Count is NOT a data <em>science</em> notebook.</li><li><a href="https://en.castordoc.com/">Castor</a>: Castor is uniquely good at organizing information about data to support data discovery, GDPR compliance, and knowledge management. Through a plug-and-play solution, Castor builds a comprehensive and actionable map of all data assets. Castor is NOT a data visualization or BI tool.</li><li><a href="http://www.getcensus.com/">Census</a>: Census is uniquely good at syncing data models from a warehouse to business tools like Salesforce. It complements existing warehouses, data loaders &amp; transform tools to enable data teams to drive business operations. It is NOT a no-code tool nor does it automagically model your data; it relies on analysts writing models in SQL.</li><li><a href="http://iterative.ly/">Iteratively</a>: Iteratively is a schema registry that helps teams collaborate to define, instrument, and validate their analytics. With Iteratively, you can ship high-quality analytics faster and prevent common data quality &amp; privacy issues that undermine trust. Iteratively is NOT a BI tool, data pipeline, or transformation tool.</li><li><a href="https://streamsql.io/">StreamSQL</a>: StreamSQL handles deploying, versioning, and sharing model features. Using your definitions, it generates features for both serving and training. Its registry facilitates re-using features across teams and models. Stream does NOT model management and is completely agnostic to what you do with the features once you get them.</li><li><a href="http://www.xplenty.com/">Xplenty</a>: Xplenty is a cloud-based ETL solution providing simple visualized data pipelines for automated data flows across a wide range of sources and destinations. It is uniquely good at ingesting large volumes of data, performing code-free data transformations, and scheduling workflows. Xplenty does NOT do event streaming.</li><li><a href="http://www.vectice.com/">Vectice</a>: Vectice is uniquely good at tracking, documenting, organizing all AI assets (e.g datasets, features, models, experiments, dashboards, notebooks) and the underlying domain knowledge to successfully manage and scale the enterprise AI initiatives. Vectice does NOT provide any runtime or computational environment.</li><li><a href="http://snowplowanalytics.com/">Snowplow Analytics</a>: Snowplow is a streaming behavioral data engine that is uniquely good at generating event data from dedicated web/mobile/server SDKs, enhancing that data and delivering it to your data warehouse. Snowplow is NOT a data integration (ELT) tool, nor a general streaming framework, nor a BI tool.</li><li><a href="http://www.datafold.com/">Datafold</a>: Datafold is uniquely good at comparing datasets in a SQL data warehouse or across data warehouses. It enables running “git diff” on a table of any size. Datafold is NOT a database itself (it works on top of existing infrastructure) and it does NOT work with files.</li><li><a href="https://www.splitgraph.com/docs/getting-started/introduction">Splitgraph</a>: Splitgraph is a tool for building, extending, versioning, and sharing SQL databases that is uniquely good at enhancing existing tools. Splitgraph also features a data catalogue including 40K open datasets that can be queried (and joined) with any SQL client. Splitgraph is NOT a database.</li><li><a href="http://www.datacoral.com/">Datacoral</a>: Datacoral is uniquely good at automatically generating data ingestion and transformation pipelines from SQL-based declarative specifications, and automatically capturing and displaying schema level lineage. Datacoral plays nice with data ingestion tools like Segment, and workflow management tools like Airflow. Datacoral is NOT a data warehouse or a query engine.</li><li><a href="https://arrow.apache.org/">Apache Arrow</a>: Apache Arrow is uniquely good as a language-independent standard for fast in-memory analytical processing and efficient interprocess transport (with minimal overhead) of large tabular datasets. While intended as a computational foundation for data frame projects, it is NOT a replacement for end-user facing tools like pandas.</li><li><a href="http://datasaur.ai/">Datasaur</a>: Datasaur is built to support NLP labeling via ML-assisted suggestions. It supports workforce management, maintains data privacy, and can be integrated via API to any ML workflow. Datasaur does NOT handle bounding boxes for image/video labeling.</li><li><a href="http://datakin.com/">Datakin</a>: Datakin is a DataOps solution that helps guarantee that data pipelines run without disruption and resulting data can be trusted. It does so by automatically discovering data lineage and providing tools to quickly identify and resolve issues. Datakin is NOT a data catalog nor does it replace any existing data infrastructure components (workflow orchestration, data processing, …).</li><li><a href="http://www.aperturedata.io/">ApertureData</a>: ApertureData is a database for visual data like images, videos, feature vectors, and associated metadata like annotations. It natively supports complex searching and preprocessing operations over media objects, and integrates with cloud-based storage and ML frameworks like PyTorch/Tensorflow.. ApertureData does NOT extract metadata or features from images/videos.</li><li><a href="https://www.orchest.io/">Orchest</a>: Orchest is uniquely good at assisting data scientists in interactively building data science pipelines by providing a visual pipeline editing environment in the browser. Pipeline steps are containerized notebooks or scripts. Orchest does NOT replace Jupyter notebooks, provide a no-code tool, or bring its own computational infrastructure.</li><li><a href="https://gazette.readthedocs.io/en/latest/">Gazette</a>: Gazette is an open source streaming platform that breaks down the divide between batch and real-time data, enabling users to build real-time applications with exactly-once semantics. It offers real-time message streams, which are natively and durably stored as regular files in cloud storage. Gazette is NOT an ETL tool or an analytics platform.</li><li><a href="https://coiled.io/">Coiled</a>: Coiled excels at scaling data science and machine learning workflows in native Python using Dask, which is familiar, widely adopted, and gives great feedback. Coiled is an opinionated way of bursting to clusters and the cloud while staying in the PyData ecosystem. Coiled/Dask is NOT a database or Kubernetes replacement.</li><li><a href="http://upsolver.com/">Upsolver</a>: Upsolver is a cloud-native solution for integrating structured and unstructured data on cloud storage. It utilizes a visual, SQL interface for quick and easy data transformation. Upsolver is NOT a Platform as a Service solution that requires developers to write additional code and learn low-level concepts to process data.</li><li><a href="https://www.soda.io/">Soda</a>: Soda is uniquely good at creating and monitoring data observability metrics. Its open-source metrics collection agent provides data and analytics engineers with a simple, extensible framework to create observability across their data lake and warehouse. Soda is NOT a data orchestration, -scheduling or -modeling framework.</li></ol><p>As authors (<a href="https://twitter.com/sarahcat21">Sarah</a>, <a href="https://twitter.com/AbeGong">Abe</a> &amp; <a href="https://twitter.com/petesoder">Pete</a>) we’re collectively brainstorming about how we can extend this effort and create an ever-growing list that helps practitioners find and adopt the right tools, founders align with the best partners, and investors map companies to their investment theses. We look forward to hearing your thoughts on the best medium to continue this exploration with the support of the community.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46bc365bea74" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/20-more-hot-data-tools-and-what-they-dont-do-46bc365bea74">21 MORE Hot Data Tools and What They Don’t Do</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[25 Hot New Data Tools and What They DON’T Do]]></title>
            <link>https://medium.com/data-science/25-hot-new-data-tools-and-what-they-dont-do-31bf23bd8e56?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/31bf23bd8e56</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Thu, 14 May 2020 17:51:52 GMT</pubDate>
            <atom:updated>2020-09-22T14:57:28.876Z</atom:updated>
            <content:encoded><![CDATA[<h4>“Wait, do tool X and tool Y work together? I thought they were competitive.”</h4><p>There are dozens of new tools in the fast-growing data ecosystem today. Together, they are reshaping data work in exciting, productive and often surprising ways. The seeds of the data landscape for the next decade have been planted, and they’re growing wildly.</p><p>Turns out, cultivating a new ecosystem is messy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w97KaUhqy8hZng5D" /><figcaption>Photo via Unsplash</figcaption></figure><h3><strong>Confusion</strong></h3><p>One symptom of messiness is that many of these tools are perceived as competitive, even when they’re not.</p><p>The perception of false competition is not surprising, since:</p><ul><li>There’s a lot of overlapping functionality among tools. Partly this is because tools and products need to “stub out” basic functionality — sometimes in areas that aren’t their primary focus;</li><li>Growth trajectories and category boundaries are still uncertain;</li><li>No single tool is yet ubiquitous, even within a given niche;</li><li>Entrepreneurs are incentivized to tell a “Big Story” both when pitching VCs and selling their product.</li></ul><p>So the confusion is natural.</p><p>It’s also bad. Bad for tool builders trying to focus, bad for investors trying to assess markets, and especially bad for data scientists and engineers trying to build productive data stacks utilizing the best new tooling options.</p><h3><strong>Time for Clarity</strong></h3><p>This blog post is a collaboration between <a href="https://twitter.com/sarahcat21">Sarah Catanzaro</a> (partner, Amplify Partners), <a href="https://twitter.com/AbeGong">Abe Gong</a> (co-founder, Superconductive/Great Expectations), and <a href="https://twitter.com/petesoder">myself</a> (founder, Data Council &amp; Data Community Fund) to start to clear the confusion and tangle. It grew out of conversations in the startup community among people who were tired of “Wait, aren’t you competitors?”-style questions.</p><p>We’ve encountered these questions enough to recognize the pattern. We’ve seen the unfortunate drag that they impose on adoption and collaboration. Now it’s time to do something about it.</p><h3><strong>What does your tool NOT do?</strong></h3><p>To start to attack this problem, we reached out to dozens of entrepreneurs and open source maintainers in the data ecosystem, and asked two questions:</p><p>1. What is your tool uniquely good at?</p><p>2. What does your tool NOT do?</p><p>We limited our search to open-source projects and pre-Series B companies. Responses were kept short and sweet, and edited for clarity.</p><p>All of the participating tools are listed below, listed in the order that they responded.</p><p>Confusion in the ecosystem won’t clear up overnight, but our hope is that this post is a good start.</p><h3><strong>Tools and Answers</strong></h3><ol><li><a href="https://greatexpectations.io/?utm_source=data-council&amp;utm_medium=blog&amp;utm_campaign=hot-tools-052020">Great Expectations</a>/Superconductive: Great Expectations is uniquely good at testing data systems and creating documentation from those tests. Great Expectations also does data profiling. Great Expectations is highly pluggable and extensible and is entirely open source. It is NOT a pipeline execution framework or a data versioning tool.</li><li><a href="https://databand.ai/">Databand</a>: Databand is a DataOps solution that’s uniquely good at monitoring production pipelines and detecting issues on a workflow code, data, or system level, and helping engineers do root cause analysis of the problem. Databand is NOT a point solution for pipeline orchestration, data quality testing, or data versioning.</li><li><a href="https://www.liquidata.co/">Dolt</a>/Liquidata: Dolt is a SQL database with Git versioning. You can commit to, diff, clone, pull, branch, and merge a SQL database just like you would in Git. Dolt is a uniquely good format for sharing data. Dolt is NOT designed for a specific data use case like feature storage or data transformation. Dolt is a general purpose database that may be applied in those use cases.</li><li><a href="https://bayes.com/">Bayes</a>: Bayes is a visual, exploratory data analysis tool. It guides you through recommended visualizations, and enables easy, interpretable insight sharing with interactive narrative-based reports. Bayes is NOT a business intelligence dashboard, nor is it a code-based notebook for programmers.</li><li><a href="https://hex.site/">Hex</a>: Hex is a computational notebook platform that is uniquely good at sharing. Users can connect to data, develop their analyses, and then easily build a fully-interactive, beautiful app that anyone in their organization can use. Hex is NOT a ML engineering platform or a charting tool.</li><li><a href="https://sisu.ai/product/">Sisu Data</a>: Sisu is a proactive analytics platform uniquely suited to rapidly exploring complex enterprise data and helping analysts explain why key business metrics are changing. Sisu can test hundreds of millions of hypotheses in seconds and guide users to the highest impact drivers of change. Sisu is NOT a predictive or model-building tool, nor a descriptive dashboard.</li><li><a href="https://www.ascend.io/">Ascend</a>: Ascend is uniquely good at building, running, and optimizing cloud-only data pipelines with significantly less code. Ascend links data to the code that produces it, enabling declarative data pipelines with automated maintenance, data profiling, lineage tracking, cost optimizing, and easy integration to databases, warehouses, notebooks, and BI tools. Ascend is NOT a general-purpose Spark solution, but rather, the data engineering platform on top.</li><li><a href="https://dataform.co/">Dataform</a>: Dataform is uniquely good at helping you manage hundreds of datasets within your data warehouse. Dataform helps teams turn raw data into a suite of well-defined, tested, and documented datasets for analytics. Dataform is NOT an extraction tool.</li><li><a href="https://datakitchen.io/">DataKitchen</a>: DataKitchen is a DataOps process tool uniquely good at bringing continuous deployment, observability, testing, and environment management to the complex world of data science and analytics. DataKitchen automates analytics creation/deployment workflows, pipeline execution, code/data testing, and pipeline monitoring/maintenance. DataKitchen is NOT a data science, ETL or dedicated orchestration tool.</li><li><a href="https://www.snorkel.org/">Snorkel</a>: Snorkel is a platform for programmatically building training datasets. In Snorkel, rather than hand-labeling data, users write labeling functions which Snorkel combines using theoretically-grounded modeling techniques. Snorkel is NOT about unsupervised learning; it is a human-in-the-loop platform focused on radically accelerating how users inject their domain knowledge into ML models.</li><li>Transform (stealth): Transform is building a metrics repository which ensures that businesses can capture metric definitions in a standardized, well-formatted, and organized way to streamline analysis and enable decision-making with confidence and speed. Transform is NOT a data pipelining framework or a business intelligence tool.</li><li><a href="http://materialize.io/">Materialize</a>: Materialize is uniquely good at executing and then maintaining PostgreSQL queries (including joins) on top of streaming data, keeping those queries up to date with millisecond latencies under high throughput. Materialize is NOT a time-series database or another streaming microservice platform.</li><li><a href="https://github.com/linkedin/datahub">Data Hub</a>/LinkedIn: DataHub is a search and discovery application powered by metadata, and built to boost AI and data science productivity. It features a unique stream-first distributed metadata architecture that has made it successful at LinkedIn’s people and big data scale. It’s NOT a data integration or processing tool, or an orchestrator for running data quality checks.</li><li><a href="https://www.prefect.io/">Prefect</a>: Prefect is a workflow orchestration tool that allows you to define flows of tasks using a pure Python API, and deploy them easily using modern, scalable infrastructure. Prefect gives you the semantics you need to make robust pipelines, such as retries, logging, caching, state transition callbacks, failure notifications, and more, without getting in the way of your code. Prefect is NOT a no-code tool or infrastructure provider.</li><li><a href="https://github.com/mara">Mara</a>/Project A: Mara is uniquely good at composing SQL, Bash &amp; Python scripts into pipelines. Run pipelines from command line or through web UI. Local execution, no queues, no workers, no magic. Mara is NOT a scheduling, data movement, or dependency detection tool.</li><li><a href="http://getdbt.com/">dbt</a>/Fishtown Analytics: dbt excels at creating, maintaining, and documenting DAGs of SQL-based business logic in data warehouses. dbt is NOT a general-purpose job orchestrator.</li><li><a href="https://www.watchful.io/">Watchful</a>: Watchful is uniquely good at creating high-quality, probabilistically-labeled training data quickly and at scale without an army of human labelers. Watchful lets you build, test, and prototype models fast through feedback. It is NOT a managed labeling service or an analytics tool.</li><li><a href="https://preset.io/">Preset</a>: Preset is a SaaS-based data exploration and visualization platform by the makers of Apache Superset. Preset is about visualization and data consumption, it is NOT a computation or orchestration platform.</li><li><a href="https://kedro.readthedocs.io/en/stable/index.html">Kedro</a>: Kedro shines at software engineering best practices for data and ML pipelines. Kedro enables a seamless transition from experimentation to production using a reproducible analytics workflow, I/O abstraction, and pipeline modeling. Kedro is NOT a workflow orchestrator or experiment tracking framework.</li><li><a href="https://torodata.io/?gclid=EAIaIQobChMI94qd9buv6QIVwxd9Ch2hVAqxEAAYASAAEgKHofD_BwE">Toro Data</a>: Toro is uniquely good at helping teams deploy monitoring on their data, suggesting what to monitor for and making it easy to do without writing and deploying code. Toro does NOT clean or transform data, or natively control the pipelines/workflows.</li><li><a href="https://tecton.ai/">Tecton</a>: Tecton is great at curating and serving features. Tecton is NOT a data processing engine (e.g. Spark) or a model management tool. Instead, it leverages existing data processing engines to process raw batch/streaming/real-time data, turn it into features, and deploy the features for training and serving.</li><li><a href="https://dagster.readthedocs.io/en/0.7.12/">Dagster</a>/Elementl: Dagster is a data orchestrator that’s uniquely good at structuring data applications for local development, testing, deployment, and ops. Dagster pipeline components are authored in any language or framework and combine to form a unified data application through common metadata and tools. Dagster is NOT a processing engine, or a data warehouse/object store.</li><li><a href="http://getselectstar.com/">Select Star</a>: Select Star is a data catalog and management tool that solves data discovery problems. It’s uniquely good at helping you understand your data — what data you have, where it lives, how it’s structured, and how it’s being used. Select Star does NOT provide a SQL client or ETL processing.</li><li><a href="http://www.montecarlodata.com/">Monte Carlo</a>: Monte Carlo is a data reliability platform that includes data monitoring, troubleshooting, and incident management. Monte Carlo is NOT a testing framework, pipeline, or versioning tool.</li><li><a href="https://flyte.org/">Flyte</a>/Lyft: Flyte is uniquely good at iteratively developing scalable, container-native and repeatable pipelines that connect disparate distributed systems while putting data flow front and center. Flyte is not a machine learning platform but can be a core component of one.</li></ol><h3>Starting a Dialogue</h3><p>Our desire in compiling this list is two-fold. On one hand, we want to give credit to these amazing tools, founders and OSS leaders for pushing the evolution of our data tooling ecosystem forward. On the other hand, we wish to start a helpful dialogue in the community surrounding a clear understanding of the intentional limitations of these tools. No one can do everything better than everyone; even innovators!</p><p>We hope that this article has shed some helpful light on the wild garden that is our modern data-tooling ecosystem. May it continue to thrive as we cultivate it with intention.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31bf23bd8e56" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/25-hot-new-data-tools-and-what-they-dont-do-31bf23bd8e56">25 Hot New Data Tools and What They DON’T Do</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why We Became Data Council (1/14/19)]]></title>
            <link>https://petesoder.medium.com/why-we-became-data-council-1-14-19-b7c29c49c823?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7c29c49c823</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Wed, 16 Jan 2019 18:05:31 GMT</pubDate>
            <atom:updated>2019-01-28T21:34:12.723Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Why We Became Data Council (1/14/19)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lv6l23zZwoK6lj-LAe_xcQ.jpeg" /></figure><p>I’m excited to announce that our community &amp; event series for deeply technical data professionals around the globe is no longer called DataEngConf; we are now <a href="http://datacouncil.ai/"><strong>Data Council!</strong></a></p><p>Since our origin as a data engineering conference (in 2015 when a ‘data engineer’ was rarely a specific role on most tech teams), we quickly branched out to also embrace data scientists. We knew it was important to recognize the technical and cultural interplay between both data engineers and scientists as they worked to support each other on nascent data teams.</p><p>But for the past year, our expanded aim has been to embrace all technical topics surrounding data — infrastructure, engineering, science, analytics, machine learning, deep learning, AI product development, the list goes on. Our new name reflects both the breadth of the high quality technical data-oriented content we’ll continue to curate and distribute in our events, as well as our focus on building the top community of data influencers. We knew from the beginning that we might not be the biggest data conference series, but we are certainly working hard to be the best one.</p><p>As a founder, I believe the identification of a startup’s vision and values are core not only to its brand, but also its name. In our specific case, we’re on our fourth year of running technical data engineering and science conferences, and as the newly minted Data Council, I’m excited that our team is ready to go out and tell our story at scale to the world.</p><p><strong>So, what’s the big idea?</strong></p><p>From my earliest email newsletter experiments dating back to our origins in 2012 the goal of the company has always been about building a global network of technical professionals. Regardless of how many years have passed, experiments run, funding raised, MVPs launched and team members changed, the goal is still the same, though many details of our approach have been refined based on our learnings.</p><p>To put it simply, our goal is to create <strong>human connections amongst technical professionals</strong>.</p><p>Deeply technical professionals (software engineers, data scientists, CTOs, etc.) are lifelong learners, and they acquire information voraciously in any way possible. It’s easy to understand how access to digital content has been game changing when it comes to coding education and continuing skill development. But there is another key way knowledge is shared that might seem both obvious yet under-appreciated at the same time: receiving information orally and in-person from trusted colleagues, mentors and friends. Is it any wonder that the Bay Area seems to have the deepest concentration of experienced software engineers in the U.S., and arguably, the world? The university engineering programs, continual tech events, face to face meetings and a plethora of work experiences are the flywheel of technical knowledge sharing opportunities. And all these mechanisms transfer knowledge in primarily the same way — via human, IRL, connections.</p><p>It’s obvious that engineers have long resisted typical online ‘social networks’ instead preferring to vet the fidelity of the sources of their information in their own heuristic ways. For example, there is nothing a typical software engineer hates more than self-promotion, and it’s hard to wade into any online social network without running headlong into a staggering amount of navel-gazing, at least by others, if not sucked into it oneself.</p><p>But we’re currently undergoing a major technological shift that’s creating an asymmetry of technical knowledge — and it’s most apparent between the <a href="https://www.amazon.com/AI-Superpowers-China-Silicon-Valley/dp/132854639X/ref=asc_df_132854639X/?tag=hyprod-20&amp;linkCode=df0&amp;hvadid=312643571004&amp;hvpos=1o1&amp;hvnetw=g&amp;hvrand=2121565303107625834&amp;hvpone=&amp;hvptwo=&amp;hvqmt=&amp;hvdev=c&amp;hvdvcmdl=&amp;hvlocint=&amp;hvlocphy=9029381&amp;hvtargid=pla-524214934470&amp;psc=1">U.S. and China</a>. Interest in Machine Intelligence (some call it Artificial Intelligence) is sweeping the globe, and the very rapid advancement in machine and deep learning research is creating an unevenness in the distribution of technical knowledge in AI tools and techniques. All AI advancements are driven by mathematics, algorithm development and a deep understanding of data-oriented technologies. However, due to the rapid pace of new developments in the knowledge, innovation and algorithms driving machine intelligence, the potential for meaningful information gaps in this field between those not connected to each other is real.</p><p>The desire of technical professionals to gain more knowledge and skills in data technologies related to the advancement of machine intelligence has created a new opportunity to build a fast-growing network of human connections across the globe. <strong>Data Council’s vision is to capitalize on this moment, and to connect all data-oriented technical professionals in the world together in one community.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ikg8elvPOVap-LE68ZvpEQ.jpeg" /></figure><p><strong>How do we achieve it?</strong></p><p>Our go-to-market is, in a way, reverse from some recent social networks, and it’s based on our proprietary experience in creating physical spaces for engineers in the form of high quality events that foster the special trust they demand for relationship building and knowledge sharing:</p><p>1st: <strong>Live events</strong> (high quality conferences &amp; meetups build person-to-person and brand trust) -&gt;</p><p>2nd: <strong>Knowledge sharing</strong> amongst community (via digital content from our community influencers that’s differentiated by technical depth) -&gt;</p><p>3rd: <strong>Intentional matching </strong>across members (“social networking” features designed to connect members in order to solve problems faster, share increased knowledge, find the best jobs more quickly, etc.)</p><p>The output of this sequence produces the world’s first social network specifically designed for technical data-oriented professionals.</p><p><strong>And why the name “Data Council?”</strong></p><p>Here are some more specific reasons we feel “Data Council” connects well with our vision and values:</p><p><em>1.) A Community of Influencers</em></p><p>The current, and future technical leaders of AI are found in our community. We like the way the word ‘Council’ infers high-quality discourse amongst a smallish committee (in our case, a high-quality <em>community</em>) of thoughtful individuals.</p><p>Above all, we see ourselves as being in the business of building community. As opposed to most other data/AI conferences that exist, we organically go deeper through our monthly meetups in 10 cities (and growing!) across the globe. We pride ourselves in our commitment to this grassroots community building by giving our volunteer meetup organizers the support and help they need to build successful outposts of Data Council in their cities. These efforts help build meaningful community fabric year round by making connections between local data practitioners, teams and companies each month.</p><p>The output of these groups shows the true power of our movement, er, model. At our meetups we see engineers educated, tools discovered, open-source contributors matched and team members hired regularly.</p><p>While we currently monetize via our largest annual events (currently three main conferences each year in SF, NYC and Barcelona), we see ourselves as having a much bigger vision than a traditional events business due to our focus on building a grassroots global community.</p><p><em>2.) Open-source Knowledge Sharing</em></p><p>We firmly believe in open-source: code, content and mentality. We feature top open source contributors and tools at our events. Afterwards we publish all event content for the broader data community, for free.</p><p>Many top open-source contributors and leaders participate regularly in our community (Apache Hadoop, Spark, Arrow, Kafka, Flink and others). If you want to hear about the vision and roadmap of these crucial data projects, tune into our <a href="https://www.youtube.com/DataCouncil">YouTube channel</a> for the most up-to-date info on open-source data tools and platforms. And attend our events to meet and converse with the committers themselves!</p><p><em>3.) Identifying the Best Innovations</em></p><p>Our community of experts is a natural filter for discovering the best AI and data innovations and products. As an engineer/founder myself, I have a passion for supporting other technical founders, and the most innovative AI startups will always find a home and support in our community.</p><p>We routinely have data-oriented startups and new open-source projects ‘launch’ in conjunction with our events, and discovering and featuring the top innovative companies of the future is a priority for us. (Here are some <a href="https://www.datacouncil.ai/startups">awesome startups</a> that presented last year at just one of our conferences.)</p><p><em>4.) Careful Partner Integration</em></p><p>Finally, we believe it’s our unique mission to foster and encourage connection across all layers of industry and academia. Our community features AI researchers from universities such as MIT, Stanford, Columbia and Carnegie-Mellon, mixed in with industry leaders from Google, Facebook, Lyft, Netflix and Spotify. Some partners wish to share knowledge, some wish to recruit new team members, some wish to obtain feedback on their early products and some wish to find new customers.</p><p>We know how sensitive technical professionals are to certain methods of sales, marketing and recruiting, so we carefully screen and integrate partners into our community in meaningful ways without the over-sponsored glitz and glitter of sales-and-marketing-driven data conferences. (Yuck!)</p><p>We consider it a privilege to build this community, and we take care in how we connect the dots amongst various community members as we focus on maximizing value for all parties in each interaction we create.</p><p><strong>Let’s Go!</strong></p><p>We’re thrilled and honored to be building a brand and community that technical professionals respect. And after three years of successfully building a deeply technical data community across the U.S. and Europe, we’re extremely excited to expand Data Council to our next region of Asia where we plan to launch a Singapore conference in 2019!</p><p>Imagine what the world will look like when all the deeply technical data professionals and AI researchers are connected together in one community; advancements spread more quickly, challenging problems become easy, AI ethics are debated openly, startups are incubated, amazing technical content is created and new team members are matched.</p><p>This is our vision of the future, and the vision we’re building towards at Data Council!</p><p>I’d like to sincerely thank all of our community members and supporters for their help in getting us to this inflection point. We could not have done it without our amazing community of speakers, volunteers and community organizers.</p><p>And if you personally would like to come and work with our awesome remote-only team to spread AI knowledge across the globe, we’re currently looking for a <a href="https://hakka-labs.breezy.hr/p/a817ecd8888f01-head-of-content-community?state=published">Head of Content and Community</a> to join us!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7c29c49c823" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 Key Ways a VC Thinks About a Data-Oriented Startup]]></title>
            <link>https://petesoder.medium.com/3-key-ways-a-vc-thinks-about-a-data-oriented-startup-4ebc67c2783?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ebc67c2783</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[venture-capital]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Tue, 01 Nov 2016 19:47:04 GMT</pubDate>
            <atom:updated>2016-11-01T20:25:49.832Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OvVXYD76lsoRRYHR7k3Zmw.jpeg" /><figcaption>Evan Nisselson, Partner LDV Capital (photo ©Ron Haviv)</figcaption></figure><p>How does a venture capitalist (aka VC) think about data-oriented startups? I recently interviewed Evan Nisselson, General Partner of LDV Capital, to find out.</p><p>Evan has an interesting investment focus on what he calls ‘visual technology businesses.’ It partially stems from his early experiences with digital photography in as far back as the early-mid 90s, when he was one of the earliest digital photo editors on the internet for @Home Network.</p><p>Evan’s focus on visual technology is a strategic bias based on the explosion we’re presently seeing in visual images driven by internet connected devices (mobile, IoT, wearables, <a href="https://canary.is/">home security cameras</a>, etc.). This, coupled with the advancements in machine learning and neural networking approaches to image recognition, signal determination and building intelligence from visual assets, is leading to advancements that comprise much of the innovations at growing startups today: VR, self-driving cars, drone piloting and Snapchat’s specs.</p><p>Admittedly, I found Evan’s approach to how he considers startups fascinating, so much so, that I asked him to participate on our special VC Panel at <a href="http://dataengconf.com/">DataEngConf</a> this week in NYC. The panel is designed to offer helpful entrepreneurial information to data engineers and data scientists who might be thinking about starting their own companies, including how to attract (and retain) VC interest.</p><p>But what are the special challenges a startup faces in jumping into the visual technology fray that VCs often consider? Evan and I discussed three things:</p><h3>1. Team</h3><p>The common challenges with building a founding team should be relatively well understood by anyone presently involved in startups, in my opinion. Having the right balance of technical, product, business and distribution focus in a startup is that amazing chemistry experiment where the successful constellation seems to only manifest itself a small percentage of the time.</p><p>But, more specifically, how does a data-oriented business affect the earliest composition of your team? Evan says that if you’re a business-oriented founder looking for a technical complement, the danger could be to see the first data scientist you bring on board as being the ‘tech’ member of the team. In reality, many data scientists, while technical, don’t often come from a strong engineering background and are ill-equipped to build the software systems to wrangle data in the earliest stages of a company (let alone your first viable commercial product).</p><p>On the other hand, if you already have a strong technical founder/player on the team, the temptation can be for that person to want to bring in a more experienced data scientist to learn sophisticated techniques from — even when often in the earliest days, a startup doesn’t necessarily have enough ‘data to science’, from the perspective of volume. I’ve seen this in starting my own data-oriented companies as well — sometimes the most rudimentary rules that drive good-enough data intelligence can have the quickest impact in early stages when proving your product and getting traction. And often this can be done by the existing technical players on your team (ElasticSearch configuration params anyone?), without an early company falling headlong into the notion of “we must build our data science team now!”</p><h3>2. Data Scale</h3><p>How can a company build data scale — that seemingly impenetrable moat coveted by savvy founders, which has become the hallmark of the most successful and largest internet companies? Well, obviously through usage; but that only truly comes after a company already has many users. In the interim, Evan’s main point here, interestingly, wasn’t as much of a technical one as a personality one — founders need to be able to foresee their own obstacles to acquiring data and be creative in figuring out ways to overcome that barrier. Personally, I’ve seen this occur through data partnerships, scraping public datasets, and digitization of previously non-digital records.</p><p>For instance, a data-oriented company in NYC that I’ve worked with was highly successful in building data partnerships early on. They were successfully in this approach because they had sophisticated technology built to ingest, clean, and even add value (new data) on top of the partner’s data — some of which they would return to the partner as part of the deal.</p><h3>3. Data Uniqueness</h3><p>Finally, this brings us to data uniqueness — the other form of data augmentation (both in scale and uniqueness) — that comes in an accretive way via any additional data your users add on top of whatever data you started with. If you give your users meaningful ways of interacting with your platform, they can add tremendous value. This is now obvious to us in the data-exhaust-laden days of social networking platforms, but think about how this might affect B2B-based products as well. Interestingly enough, many current data-oriented businesses are in the B2B, not the B2C space. Apparently, investors think one to three social networks of Facebook-scale or Snapchat potential are enough and are now moving away from ‘social’ investments.</p><p>For a B2B use-case, medical imaging comes to mind as an example of both points. It’s easy to see how data partnerships are required to access your x-ray images across different medical facilities, hospitals, doctors and even insurance providers. At the same time, any of those providers could add value by tagging or annotating those images depending on their role in the overall process of saving you in an emergency, keeping you healthy or determining insurance benefits.</p><p>While these are the most common challenges that VCs take into consideration when looking at data-oriented startups, there are many more. To learn other key ways that VCs think about a data-oriented startup, check out the VC panel at DataEngConf.</p><p><a href="http://dataengconf.com/">Get more information on DataEngConf here.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ebc67c2783" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Secret to Throwing a Tech Conference that Reflects How Diverse We Really Are]]></title>
            <link>https://petesoder.medium.com/the-secret-to-throwing-a-tech-conference-that-reflects-how-diverse-we-really-are-2015f1acb789?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/2015f1acb789</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[diversity]]></category>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Thu, 17 Mar 2016 17:49:15 GMT</pubDate>
            <atom:updated>2016-03-19T01:40:47.440Z</atom:updated>
            <content:encoded><![CDATA[<h4>Diversity is what you make of it</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ELPsj7KAH53IQ3zczLF5Kw.jpeg" /></figure><p>Even though it was early in the early planning stages, I was already getting excited about the event. We had a cohesive theme “Bridging the gap between data engineering and data science,” and there seemed to be significant interest from multiple tech companies in San Francisco — the sort of place where massive amounts of collected data make this the Mecca of engineers and data scientists. We were even ahead of the curve in triangulating event space. Things were coming together nicely!</p><p>Then someone asked me what my perspective was on issues surrounding <strong><em>diversity</em></strong> at tech conferences like the one I was planning.</p><p>I responded that I felt that diversity was important, but internally I listened to my own response with trepidation. “Who am I to champion diversity?” and “Am I even qualified to have an opinion on the subject?” After all, I’m a white male programmer geek. Not someone folks typically look to these days for inspiration on diversity in the tech world in the US.</p><h4>But my actions shifted after that brief moment of realization when I discovered I held an internal belief that the event I was creating needed to feel as inclusive as possible to everyone involved. This would apply to speakers, sponsors, attendees, staff, everyone.</h4><p>And if that sense of <strong>inclusion</strong> was the goal, then we could, and even should, broaden the scope of outreach to attempt to attract a more diverse set of participants that I would have casually thought would be the target market of participants.</p><p>But before I had the time to think of <strong>how</strong> we would do this — it happened. Emails from students who wanted to attend. Veterans. Folks who couldn’t afford tickets looking for help with scholarships.</p><p>Next came conversations with sponsors who expressed interest in supporting other underrepresented groups in attending. After that, a natural broadening of my initial thoughts on what the ideal profile of a speaker might be.</p><p>I began explaining to folks who wanted to help us find speakers that we were looking specifically for more female speakers — not because I was adhering to some mathematical conference formula that equals ‘diversity’ — but to try to demonstrate the attitude of inclusion I had discovered inside, and to do something to <strong>work</strong> to expand my field of vision and to attempt to find qualified speakers that didn’t tend to have profiles like me, or look like me at all.</p><p><strong>And this is when I became truly surprised — I found them.</strong> Note that I’m not talking about the supply side problem here. We all know that we have a lot of work to do to bring more women and other underrepresented groups into STEM education and careers, and also to create a welcoming, safe environment for them to practice in. But that’s where my conference planning experience plays a part — by focusing on an attitude of inclusion, we were able to attract a far more diverse set of speakers than I would have expected to participate. They were there — all I needed to do was to look for them.</p><p>Fast-forward to this point in the story, I’m realizing how proud I am of the event our team has designed. We have an amazing set of sponsors that are in support of diversity in data science, like <a href="http://www.galvanize.com/">Galvanize</a>. The constellation of attendees is amazing, and our <a href="http://www.dataengconf.com/schedule">list of speakers might be one of the more diverse I’ve ever seen in a software engineering conference</a>. We even have a speaker from Airbnb talking about how they <a href="http://techcrunch.com/2016/02/18/how-airbnbs-data-science-team-doubled-the-ratio-of-female-employees-last-year/">used data science to help address their own diversity issues</a> in hiring!</p><p>It just goes to show how powerful simple ideas can be. A little bit of the right attitude and a small amount of focus can drastically change an expected result. And in our case, it didn’t even take a lot of work. Just sensitivity to the issue, and a focus on an internal attitude that brings us all together. On inclusion.</p><p>(And if you’re interested in attending, too, we hope <a href="http://www.dataengconf.com/about">you’ll join us</a>.)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2015f1acb789" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Was inspired to see Net Neutrality win big last week! Reminds me that we all play a part in…]]></title>
            <link>https://petesoder.medium.com/was-inspired-to-see-net-neutrality-win-big-last-week-reminds-me-that-we-all-play-a-part-in-b3658efd049d?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3658efd049d</guid>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Mon, 02 Mar 2015 23:59:57 GMT</pubDate>
            <atom:updated>2015-03-02T23:59:57.821Z</atom:updated>
            <content:encoded><![CDATA[<p>Was inspired to see Net Neutrality win big last week! Reminds me that we all play a part in ‘activism,’ and thought back to the success of the Stop SOPA movement that Aaron Schwartz lent a hand to.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3658efd049d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[6 Tips to Jumpstart your Team’s Engineering Blog in 2015]]></title>
            <link>https://petesoder.medium.com/6-tips-to-jumpstart-your-teams-engineering-blog-in-2015-44099c9f22c0?source=rss-e038f19de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/44099c9f22c0</guid>
            <dc:creator><![CDATA[Pete Soderling]]></dc:creator>
            <pubDate>Thu, 08 Jan 2015 19:17:22 GMT</pubDate>
            <atom:updated>2015-12-04T16:43:37.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*m04XrCYlTZW7cBMJgnK2AA.jpeg" /></figure><p>One thing that’s jumped out at me in my early 2015 tech/startup reading is how everyone seems to be motivated to write more, blog more, express themselves more. As a founder of a startup and an engineering manager I’ve learned first-hand the power of content marketing, and for the past few years I’ve been helping CTOs use this and other tactics to build their unique engineering brands, which, in return, attracts better talent to their teams.</p><p>Most startup CTOs now realize that publishing a team engineering blog is table stakes. It builds their team brand, is helpful for recruiting, propels the careers of their individual engineers, and shows the team’s attitude of <a href="https://hbr.org/2014/10/the-benefits-of-giving-away-what-your-company-knows/">generosity</a>. But at the same time, many blogs languish or die a slow, miserable death with a <em>handful</em> of posts and only a few <em>dozen</em> hits (ugh!).</p><p>The reasons? It’s <em>hard</em> to inspire your engineers to write. It’s <em>hard</em> to figure out a marketing/distribution strategy (besides the flash in the pan you might occasionally get as a trending story on HN). And it’s <em>hard</em> to build an internal culture of writing and communicating. After all, engineers hate writing documentation, right? Isn’t an engineering blog the equivalent of writing public documentation?</p><p><strong>But don’t despair! I assure you there’s hope!</strong></p><p><strong>(To get instructor-led help for your team, sign up for our Writing Training for Engineers course: </strong><a href="https://www.hakkalabs.co/training"><strong>https://www.hakkalabs.co/training</strong></a><strong>)</strong></p><p>Having struggled, and learned, through this process myself I’ve put together a few ideas to help you jumpstart (or resuscitate!) your engineering blog in 2015.</p><p><strong>1.) Hone in on your hardest challenge</strong></p><p>Figure out what your main technology or implementation challenge is and learn to talk about it. Throughout their careers, the best engineers will ‘swim upstream’ looking to join teams that are full of devs smarter than themselves.</p><p>You don’t need to talk about everything your team is doing — just start with the most interesting thing.</p><p>Writing about one deeply interesting challenge will create a halo effect for the rest of your org. Observers often assume that if one team–or even one person–is working on a hard challenge that the other members of the engineering org must also be smart and working on interesting stuff, too.</p><p><strong>2.) Don’t write tutorials!</strong></p><p>In my experience, most companies specifically struggle with recruiting senior engineers to their teams. If you want to attract senior candidates my advice is to <em>not</em> write step-by-step guides, tutorials, or how-to implementation articles that laboriously explain how you got ‘up and running’ using OS package du jour.</p><p>While this type of article might be a stepping stone for less-experienced devs to contribute to your blog, we’ve found that in reading, most senior devs tend to look for something different.</p><p>Senior engineers want to see how you <em>thought through the process</em> of solving a challenge. Try this simple outline:</p><p>a.) Explain how you discovered you had a problem in the first place</p><p>b.) Discuss alternatives you tried that failed. (<strong>Don’t </strong>skip this step. Most devs don’t want to see the fairyland solution you found, they want to see <em>how </em>you found it, <em>what else </em>you considered, and <em>why </em>you chose to abandon those other options.)</p><p>c.) Reveal your ultimate solution (Based on your analysis of other options above, you should already have teed yourself up to explain clearly why the solution you chose was superior in your situation in light of the competing alternatives.)</p><p><strong>3.) Learn to use the narrative</strong></p><p>The simple outline above also introduces another powerful concept in the best engineering blog writing I’ve seen — using a narrative structure to better engage your audience.</p><p>Too many engineering blog posts are like cereal (dry, yet nourishing) and aren’t engaging enough to hook the reader through to the end. If you tell a story — i.e. when was the exact moment you realized you had a problem, what the dire consequences were that you faced, how the business was interrupted, what were the dead ends you hit, etc., you’ll set yourself up to connect, as a human, with human readers.</p><p>Even though we sometimes write like it, the truth is that engineers really aren’t machines. And posts that show both a lot of hardcore technical thought AND engage the audience via narrative are simply more fun to finish reading.</p><p>If you want to see an example of points 2 &amp; 3 in action (structure and narrative), check out this post from a senior engineer @ Spotify: <a href="https://www.hakkalabs.co/articles/torching-reducers-taught-us-essential-lesson">https://www.hakkalabs.co/articles/torching-reducers-taught-us-essential-lesson</a></p><p><strong>4.) Side projects are ok</strong></p><p>Sometimes engineers don’t feel especially passionate about what they’re working on directly at the office. Perhaps they’re refactoring older code or building something that doesn’t really feel like it’s blogworthy. Let’s be honest, not every engineering project is earth-shatteringly awesome.</p><p>If this is the case for your team, why not encourage your devs to blog about a side project? The most passionate engineers are often working on interesting things on the side, including all manner of open-source projects. If your engineers are having a challenging time writing about work projects why not, instead, turn them loose to talk about something that excites them?</p><p>Similar to the halo effect mentioned above, having smart people on your team talk about <em>something</em> is better than nothing since it will positively impact the external perception of your team. It also shows that you’re truly interested in helping your engineers succeed in their careers and that you view helping them build their brands as being just as important as building your company’s engineering brand. (And think about it — having individual developers with strong brands on your team is massively helpful to your company’s engineering brand, too.)</p><p><strong>5.) Motivate your team</strong></p><p>You probably already know it’s not going to be cake, but if keeping an engineering blog up and running is strategically important, it’s time for you as an engineering manager to act like it!</p><p>In my experience, the engineering leaders that are most successful in building a writing culture do two things:</p><p>a.) reward engineers for blogging/speaking/participating in the external community</p><p>This can be tied to an individual’s performance bonus (Etsy has been known to do this, for example), or can be associated with other fun swag you give away based on each engineer publishing a blog post each quarter. (Try celebrating their writing by giving them a Kindle, or some cool reads via Audible.)</p><p>b.) make writing a priority by giving their engineers sufficient time and space to do it</p><p>When we teach our Writing Training for Engineers class, we have each student send their manager a ghost calendar invite for 1 hr/week so that the manager knows that if the engineer isn’t at their desk or responding on Slack during that time, it’s because he/she is squirreled away in some other part of the office (or home!), offline, focusing on their writing. That’s ok, and should be celebrated.</p><p>Writing takes time, focus, peace and quiet. You’ll want to bake this into the <a href="http://www.slideshare.net/chaddickerson/code-as-craft-building-a-strong-engineering-culture-at-etsy">culture</a> of your team. If you’re actively creating this kind of space for your engineers you’ll see them progress much faster than if you simply say your engineering blog is a priority without clearing the way for your engineers to be successful at writing.</p><p><strong>6.) Don’t forget the marketing!</strong></p><p>There’s nothing more frustrating than to have gone through all the work to motivate your team to create a great post and to publish it on your engineering blog and then …. no traffic. BUMMER! This is the single biggest source of fizzle I’ve seen amongst otherwise inspired teams.</p><p>The reason? Engineering blog marketing typically falls through the cracks because it’s simply too difficult to find the right people on the team to do it.</p><p>CTOs don’t typically have the time or know-how to get distribution for their posts (let alone possess any kind of marketing budget for the engineering blog), product marketing has their hands full with other things, and PR folks think that the engineering blog posts should be mixed in with the corporate blog. (For the record: bad idea.)</p><p>Here’s what I’ve learned:</p><p><strong>Someone within your tech team should own it</strong></p><p>Be it an engineering manager, an IC, a dev evangelist or even someone in technical support, just make sure they can speak fluently in whatever stack you’re using. Also, find someone who really cares about your blog and your public image. By finding someone that’s intrinsically motivated you protect against the scenario where your marketing efforts get delayed — permanently. I’ve found that engineering team marketing is the sort of high-priority, low-urgency task that tends to be the first thing to go when you have other things pressing for your time and attention. Plan for this!</p><p><strong>Goals &gt;&gt; strategies &gt;&gt; tactics, and not the other way around</strong></p><p>Often when we begin thinking about marketing, it’s easy (and fun) to do a brain dump of different things you can do to get your content out there. This is a great impulse, but can be a little premature.</p><p>If you spend your time generating and implementing tactics you’ll burn out <em>fast </em>from all the up and down and running around.</p><p>My advice:</p><ul><li>Ask yourself or your team “<em>Why are we doing this?</em>” and take the top answer or two as your <strong>Goal(s)</strong></li><li>For each goal ask “<em>What are the specific markers along the way to this goal? How will I know if I’m on the right path? And when will I be able to judge my progress?</em>” Answers to these questions define your <strong>Strategies.</strong></li><li>Then, and only then, start a brain dump of all your “<em>Oh! We could try this!</em>” ideas. These are your <strong>tactics. </strong>Ask around, read around, look around: these are everywhere.</li><li>BUT, be vigilant; if a tactic doesn’t feed a strategy of yours, dump it. You can’t do it all. You should only do the things that really matter. And what matters to another company may or may not matter to you.</li></ul><p><strong>Use the best channels</strong></p><p>a.) Start with the people you know…</p><p>Do a quick audit on your team’s promotional reach. Who has hefty follower counts? Who’s active on the interwebs? Do you have an engineering twitter handle for your team? Are you following all your engineers? Do you have a mailing list or newsletter for engineers who have come to your meetups? What about a list of candidates that you want to stay in touch with for the future?</p><p>Find out where your team is hanging out in online tech communities, and ask if they will post, tweet, or share a particular article that’s relevant. Keep in mind that relevance is key, so start by getting the lay of the land before you ask your devs to share posts willy-nilly.</p><p>b.) Post in high quality places</p><p>Got a post up? Great! Who submitted it to HN? How’s their karma? Is the author in the thread commenting and conversing?</p><p>Reddit? Which subreddits? Did the person submitting have good karma points there?</p><p>(Protip: <strong>always </strong>look for subreddits that have 10,000–14,000 members. The larger ones may look more tempting, but your content will surface longer–and hit more of the right eyeballs–in targeted ones this size.)</p><p>Other channels: Quora, StackOverflow, LinkedIn’s content network, Slack channels. I’m starting to see engineering-related posts on Medium, too.</p><p>What mailing lists are you subscribed to? Do you know the curators? If you don’t, reach out and say hi. Send them interesting things they’ll often be up for adding them to their blast.</p><p>If you want an additional channel for marketing your posts, check out our company profiles on <a href="https://www.hakkalabs.co/company-profiles">Hakka Labs</a>. Our growth team has built up a huge network of engineers that regularly visit our site to discover new posts and info on projects that top teams are working on.</p><p>Hopefully 2015 is the year that you’ll rock a strong engineering blog. And as your brand grows, so will your ability to attract awesome talent. Not to mention that investing in your engineers, and their personal growth, is a good way to keep them around a bit longer, too.</p><p>(Thanks to Kinshuk Mishra, Noel Cody, Anna Marie Clifton and Julia Neznanova for reviewing this post.)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44099c9f22c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>