<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kyle Roemer on Medium]]></title>
        <description><![CDATA[Stories by Kyle Roemer on Medium]]></description>
        <link>https://medium.com/@kyleroemer?source=rss-f3456880de04------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*DWchfCr8fCKa4Z4HbxRfaw.jpeg</url>
            <title>Stories by Kyle Roemer on Medium</title>
            <link>https://medium.com/@kyleroemer?source=rss-f3456880de04------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:49:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kyleroemer/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Charlotte — a  New Technology Hub]]></title>
            <link>https://medium.com/state-of-analytics/charlotte-a-new-technology-hub-50093c5befde?source=rss-f3456880de04------2</link>
            <guid isPermaLink="false">https://medium.com/p/50093c5befde</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[charlotte]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Kyle Roemer]]></dc:creator>
            <pubDate>Fri, 15 Nov 2019 15:37:45 GMT</pubDate>
            <atom:updated>2019-11-26T18:31:28.001Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QPmnwQLjiBjVuUR6Jz4OdA.png" /></figure><h3>Charlotte — a New Technology Hub</h3><p><em>Slalom Charlotte has now been around for 1 year — traversing WeWork and our new home at the Railyard in Southend. We are about to have our launch party and I wanted to reflect on the last 6 months in Charlotte, post the last 7 years in San Francisco. It’s been a growth-fueled ride, with all the trappings you want to see in a market you’re invested in. I wanted to capture all the amazing things that have been happening but also set the stage on what we’re building: </em><strong><em>THE most impactful consulting firm in the Charlotte region</em></strong><em>. Folks that join our Charlotte market are going to grow to heights they didn’t think possible, and work on products and solutions that transform companies and industries here.</em></p><p>I spent the last 7 years in San Francisco working with some of the most innovative companies on the planet. I’ve taken much of that learning to Charlotte, and I’m floored at the opportunity here. It’s an incredible time to be in Charlotte. It’s not only the abundance of BBQ or the myriad of breweries, but the passion and care the community has here. (There are a lot of breweries, wowza) It’s an environment that you can grow and flourish in. My family and I have now been out here for 6 months, it’s felt like a few days. I’m excited to share more on the great things happening in Charlotte and Slalom.</p><h3><strong>So, what is happening in Charlotte?</strong></h3><p>IT’S BANKING! Yes, and it’s actually more — it’s manufacturing, retail, healthcare, telecom, defense and also banking. :) I am amazed at the diversity of industry here, which allows for us to really stand by the statement of diversity of work for our consultants, designers and engineers. There is not just diversity of industry, but diversity of work — modern application design, development, machine learning, IoT, cloud native architecture and transformation across [insert favorite buzzword].</p><p>A myriad of companies have announced technology hubs and new centers in Charlotte over the last few months. Why you might ask? Talent to cross train, attractive wages, lower housing costs than major 5 cities, good weather, proximity to top university areas, mountains and beaches.</p><p>Here’s what I’ve noticed coming from one of the technology epicenters — you can do interesting, challenging, dynamic work in the field and it doesn’t solely have to be the Bay Area. I imagine that’s obvious to folks in other regions, but oftentimes the Silicon Valley influence can steer the conversation on tech work. There is no longer a requirement to be on the coast for technology adoption and high wages. The proliferation of services across the cloud vendors, availability of open source experts and education has catapulted these supposed second cities into soon-to-be-powerhouses.</p><h3><strong>Charlotte on the verge — it will be a 3 year journey</strong></h3><p>I’ve spent a lot of time with technology and business leaders in Charlotte, across most industries, and I’ve learned that these leaders are starved for talent, adoption of new technologies, practices and evolution. Many of the companies here are getting underway with cloud technologies, but what is so attractive is that many are grabbing onto AI &amp; ML use cases as a catalyst for adoption. That will result in both super interesting initial work, and some complex enterprise work to set foundational cloud architecture and services. <em>If you are a data scientist, designer or engineer, I would strongly consider the curve here.</em></p><p>As I alluded to earlier, there are numerous companies announcing technology hubs, a new presence or a relocation to Charlotte in the last 6–12 months. It’s encouraging, but also a sign of a soon to be challenging talent war. Not so dissimilar to what many large cities face today, but perhaps pointed given the nature of industry and frantic pace of new build outs. By nature of industry, I mean many of the financial and manufacturing firms have a tremendous opportunity to modernize — in service of attaining new customers, delighting those customers and creating compelling products. <em>We are and will continue to train the uber talented folks coming from these firms to grow into world class technologists and leaders.</em></p><p>An example of the tremendous growth in Charlotte is our Slalom office. I relocated mid March and we had roughly 20 employees — today we sit north of 110 employees…</p><h3><strong>A Call Out for Technologists</strong></h3><p>I have been asked a number of times from local leaders on my perception of the market, and while some of this is highlighted above, I always return with a question — who do you look up to in the technology space, who are your technology leader or leaders? I have been getting a lot of blank stares. Let’s change this, together.</p><p>Let’s attract and develop those technology leaders of tomorrow in Charlotte. If you aren’t in Charlotte yet, now is the time. You’re going to have diversity of industry, diversity of projects, diversity of thought and be a part of the rocket ship.</p><p>Similar to getting asked about the technology scene in Charlotte, I get asked quite a bit on “Why Slalom” and frankly spend a lot of timing talking with clients and recruits on “Why Slalom”. The rationale is fairly simple, we are a very different consulting firm than many — here you can advise and build, design and innovate, and be on the ground floor to transform our clients. It’s rare to be a part of a firm of our size and not be beholden to powerpoint decks. (As much I like a good set of slides.) The second and third reasons are that folks that join Slalom can have variety (you don’t get pigeon-holed), learn new technologies and patterns, and work with the some profound &amp; passionate experts. Our ability and desire to blend management consulting with technology, in a local market model with global services, is simply different.</p><p>Charlotte is a great place to live now and will continue to grow into a powerhouse. If any of this peaks your interest, I would love to chat.</p><p><em>Originally posted on LinkedIn </em><a href="https://www.linkedin.com/pulse/charlotte-new-technology-hub-kyle-roemer/"><em>here</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50093c5befde" width="1" height="1" alt=""><hr><p><a href="https://medium.com/state-of-analytics/charlotte-a-new-technology-hub-50093c5befde">Charlotte — a  New Technology Hub</a> was originally published in <a href="https://medium.com/state-of-analytics">State of Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[11 Companies to Look at to Stay Ahead in the Data & Analytics Space]]></title>
            <link>https://medium.com/state-of-analytics/11-companies-to-look-at-to-stay-ahead-in-the-data-analytics-space-b14da4a3574?source=rss-f3456880de04------2</link>
            <guid isPermaLink="false">https://medium.com/p/b14da4a3574</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kyle Roemer]]></dc:creator>
            <pubDate>Thu, 15 Feb 2018 22:19:18 GMT</pubDate>
            <atom:updated>2018-02-19T23:31:25.561Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R4glclcqPm8eLWQll7cegw.jpeg" /><figcaption>Photo from Alex Holyoake on www.unsplash.com</figcaption></figure><p>Here is my take on companies you should be keeping track of in the analytics space. My hope is to curate a list like the one below every quarter to reflect changes in the analytics marketplace and inform on technologies that are gaining momentum and market traction. It’s also a forcing function for me to be keeping tabs on a number of these companies. If you want to continue to see these lists, please add a comment!</p><p>I love to see the continued proliferation of teams building products needed out there. One growing theme, as you can imagine if you pay attention to headlines, is that many companies starting today claim AI capabilities or that they’re building the next AI for x industry that will be game changing. Some are interesting, most are not. In the list below, there areonly 2–4 companies that have machine learning and AI as a part of their product offering. Some are making model development and deployment easier (DataRobot) and some are incorporating these ML techniques into their core product (Alation, Looker, etc).</p><p>The age of “Smarter” applications (enhanced through ML, Deep Learning, etc) is upon us and you should be excited about that! That’s hype I can get behind…</p><p>Now, moving on to the list of 11 companies which I’ve categorized broadly. The 4 categories of companies cover product maturity, market traction and funding. Those categories are:</p><ol><li>The “We’ve been funded SUPER well and have tons of market traction” Companies</li><li>The “We’re starting to take off, so hurry up and follow” Companies</li><li>The “We have an interesting v0 — vX of our product, strong founders, so please take our demo” Companies</li><li>The “We are well funded but haven’t released our product yet” Companies</li></ol><p>I won’t be providing a lot of commentary on certain companies which I’ve written about in the past but <strong>if you have a question on them please leave a comment.</strong></p><h3><strong>The “We’ve been funded SUPER well and have tons of market traction” Companies</strong></h3><ol><li><a href="https://www.snowflake.net/"><strong>Snowflake</strong></a> ($263M* — Series E in January ‘18)<br>One of the more interesting data platform companies to come around in some time. They are making the modern data platform much easier, sitting on AWS. Many new data platform builds are occurring with Snowflake. Lots of funding and strong market traction.</li><li><a href="https://looker.com/"><strong>Looker</strong></a> ($81.5M — Series D in March ‘17)<br>Looker continues to make traction in the market and have received more and more funding in 2017. I really like their product, but I’m hopeful they’ll continue to invest in “ease-of-use” to start to capture more of the casual analyst and user demographic.</li><li><a href="https://databricks.com/"><strong>Databricks</strong></a> ($140M — Series D in August ‘17)<br>Databricks continues to make waves as the productized / enterprise-ified version of Apache Spark. I like that they are now supporting AWS &amp; Azure, but they need to get that 3rd cloud support of GCP. Lots of funding and strong market traction. I wonder how they will grow as cloud providers start providing the same direct services at much lower costs…</li><li><a href="https://segment.com/"><strong>Segment</strong></a><strong> </strong>($64M — Series C in July ‘17)<br>Segment bridges the gap between front end and back end systems, allowing for easy data integration across 200+ connectors. Segment also has a strong streaming capability and their new personas Beta is super, super interesting. They continue to gain market traction and have one of the better user experiences in the space.</li></ol><h3><strong>The “We’re starting to take off, so hurry up and follow” Companies</strong></h3><ol><li><a href="https://alation.com/"><strong>Alation</strong></a> ($23M — Series B in July ’17)<br><a href="https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2016-f657c0594e36">I’ve written a decent amount</a> in the past about Alation, but I continue to like the problem they’re tackling. Data discoverability within an organization to streamline governance, common metrics and data usage.</li><li><a href="https://www.datarobot.com/"><strong>DataRobot</strong></a> ($67.2M — Series C in July ’17)<br>Lowering the barrier of entry for Machine Learning and accelerating development / deployment of ML models…seems like a win, win, win, right? Indeed and they’re gaining more and more market traction. As I’ve said in other articles, it’s more important now than ever to be able to translate these models and the business impact. Nice to see some publications are <a href="https://hbr.org/2018/02/you-dont-have-to-be-a-data-scientist-to-fill-this-must-have-analytics-role">saying the same thing</a>.</li><li><a href="https://usermind.com/"><strong>Usermind</strong></a> ($23.5M — Series C in January ’18)<br>Usermind built a customer hub product. It integrates customer touch point systems IE CRM, MKT Automation, etc. Solving a top challenge for many companies especially in Retail, Ent. Software, Fin Services.</li></ol><p><strong>My opinion</strong>: having led a # of projects where we’ve built similar “hubs”, I love to see a company productizing customer data hub. For this to really be appealing, they need to solve for data quality issues in all of these source systems (or rely on upstream data quality functionality from the sources — BUT none of those sources are prioritizing this yet as features). Just integrating these sources isn’t enough, the biggest challenge companies face is not data integration but single customer records and clean data!</p><p><strong>Competitors</strong>: ActionIQ is similar with same funding from Andreessen as well. Amperity is another, similar company with $28M in their latest Series B round.</p><p>4. <a href="https://www.tamr.com/product/"><strong>Tamr</strong></a> ($41.2M — total funding, latest round undisclosed)<br>Machine learning applied to master data domains within an organization. Taking a lot of the manual analysis, connecting disparate data schemas, and identifying mastering elements out of the equation. That’s all lovely and needed, but I do want to see some additional governance capabilities to make this an even more compelling solution for enterprise data governance and management.</p><p>5.<strong> </strong><a href="https://streamsets.com/"><strong>Streamsets</strong></a> ($20M — Series B May ’17)<br>Data operations platform focus on streaming data from many sources (APIs, IoT, clickstream, etc) to intermediate sources to eventually power analytics, applications, etc. former Informatica and Cloudera engineering. Well funded by top VCs.</p><p><strong>My opinion: </strong>they are solving for a challenge in IoT, Marketing and Product areas via real-time data needs. I like they are not tied to a single cloud provider but work across all 3. It will be interesting to see if they start to expand a bit beyond streaming and challenge the traditional ETL industry as well.</p><h3><strong>The “We have an interesting v0 — vX of our product, strong founders, so please take our demo” Companies</strong></h3><p>1.<strong> </strong><a href="https://insightengines.com/"><strong>Insight Engines</strong></a><strong> </strong>($15.8M — Series A in July ‘17)<br>A product that sits on top of Splunk and allows users to ask questions of their machine data. Relies on natural language processing.</p><p><strong>My opinion</strong>: a lot of potential for this to lower the barrier of Splunk adoption in an enterprise, especially for less development focused users. However, unless bought by Splunk, I don’t love companies that are tied so directly to another tech product (unless that product is more of an ecosystem, like AWS, GCP, etc)</p><h3><strong>The “We are well funded but haven’t released our product yet” Companies</strong></h3><ol><li><a href="http://ascend.io/"><strong>Ascend.io</strong></a><strong> </strong>($15M — Series A in May ‘17)<br>Prominent VCs behind this startup. Ex founder of ooyala. Set to productize data operations and the data lifecycle. Strong founding team. Typically these are interesting products to keep an eye if they make it to GA before faltering or getting acquired.</li></ol><p><strong>My opinion</strong>: No strong opinion yet. To be updated once I see more but conceptually I’m interested.</p><p>Is there an interesting startup or high growth analytics company I missed? Would love to hear about it in the comments. I’ll be updating this list via another post in the next few months.</p><p><em>*Funding amounts for each company were sourced via </em><a href="http://www.crunchbase.com/"><em>www.crunchbase.com</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b14da4a3574" width="1" height="1" alt=""><hr><p><a href="https://medium.com/state-of-analytics/11-companies-to-look-at-to-stay-ahead-in-the-data-analytics-space-b14da4a3574">11 Companies to Look at to Stay Ahead in the Data &amp; Analytics Space</a> was originally published in <a href="https://medium.com/state-of-analytics">State of Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Themes Observed in Data & Analytics in 2017]]></title>
            <link>https://medium.com/state-of-analytics/10-themes-observed-in-data-analytics-in-2017-cddc5e74f315?source=rss-f3456880de04------2</link>
            <guid isPermaLink="false">https://medium.com/p/cddc5e74f315</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[tech]]></category>
            <dc:creator><![CDATA[Kyle Roemer]]></dc:creator>
            <pubDate>Mon, 18 Dec 2017 16:32:19 GMT</pubDate>
            <atom:updated>2017-12-18T16:43:01.350Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BXT4cZQOgxi3hyV-Di7XuQ.png" /><figcaption>Photo from Mashable article on Nvidia’s machine learning model for celebrity face generation. <a href="http://mashable.com/2017/10/30/ai-machine-uses-celebrity-photos-to-create-fake-faces/#nXT75nSPbmq0">http://mashable.com/2017/10/30/ai-machine-uses-celebrity-photos-to-create-fake-faces/#nXT75nSPbmq0</a></figcaption></figure><p>2017 has come and gone. Looking back at the year, it was filled with promising technologies and a fevering hype pitch regarding AI and Machine Learning. We saw the big 3 cloud providers (AWS, GCP, Azure) release major steps forward in Data Science and Machine Learning tooling (more on that below). We continue to see a talent shortage in this space and it’s only getting worse as Data &amp; Analytics becomes an integral fiber of an organization (but universities are catching up!).</p><p>When I look back at my <a href="https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2017-4edce4481e34">original predictions for 2017</a>, there were some (IE GCP playing catch up, Data Quality &amp; Governance, Experimentation) which have rung true but others that have not (Combining Clickstream + Rev / Sales Data). While those predictions are interesting, there is much more happening in the space…</p><p>Now, let’s move on to those broader themes from 2017 that will carry into 2018:</p><blockquote><strong>1. Data Science without Data Engineering is a Problem</strong></blockquote><p>Does this sound familiar in your organization <em>Data scientists build their models and then hand off to a separate engineering group for deployment and productionalization?</em></p><p>We see this EVERYWHERE. This model of development can be successful but oftentimes is not given the potential challenges around model changes over time, interpretation required from engineering, collaboration between data scientist and engineer, non scale considerations in the model from data scientist, etc.</p><p>Encourage your data scientists to better understand data engineering and deployments of their models → <strong>up level your data scientists into data savants</strong>.</p><p>The good news is that cloud providers are making this easier than ever with services (like <a href="https://cloud.google.com/ml-engine/">Google Cloud ML Engine</a>, <a href="https://aws.amazon.com/blogs/aws/sagemaker/">AWS SageMaker</a> &amp; <a href="https://azure.microsoft.com/en-us/services/machine-learning-services/">Azure ML</a>) but it doesn’t remove the need for data scientists to both understand how deployments work and performance / scale considerations in their model and approach. These services strip out a lot of the “work” but it is yet to be determined to what degree and how difficult it will be to take more complex modeling and scoring scenarios to these services.</p><blockquote>2. “With great power comes great responsibility” — stats education paramount in the age of drag &amp; drop machine learning</blockquote><p>Tools like Sagemaker, Google Cloud ML and Azure ML are bringing Machine Learning (ML) to the masses, but now more than ever foundational knowledge of how to interpret models, assess models is more important than ever.</p><p>We’ve seen this trend before in other areas of data, like Data Visualization. Tools like Tableau and Qlik made it very easy to drag and drop visualizations and analysis. We then saw a proliferation of bad data viz, requiring more education and training on principles.</p><p>This will become just as important with these type of ML services as interpreting the results of a model and assessing models requires education in stats and math, of course. Formal, university studies aren’t necessarily required as there are some strong on-demand courses out there to learn. These short-term courses however only go so far in educating and preparing a data scientist for the actual work itself. My fear or reservation here is that training becomes more watered down given the accessibility these services are making the field.</p><p>The punchline if your data scientist or analysts are using these services, please question the approach, validation and how they’re interpreting the results of their model.</p><blockquote><strong>3. The Pace of Innovation at AWS, GCP and Azure is Staggering</strong></blockquote><p>2017 was the year where major competition and service parity took shape across AWS, GCP and Azure. Each of these cloud providers sent their respective haymakers at one another with each of their conferences and multiple product announcements throughout the year. This competition is fantastic for everyone and in a world where enterprises are moving to hybrid / multi cloud ecosystems, it’s reassuring to know you’re in good hands with each provider.</p><p>If you’re eager to learn more on some of the things released this year, check out this article by my colleague <a href="https://medium.com/u/8da2222ce15b">Andrew Tubley</a> on <a href="https://medium.com/state-of-analytics/5-things-google-cloud-platform-didnt-say-loud-enough-at-next-82a4134eb288">GCP</a> here or from my other colleague <a href="https://medium.com/u/fa292d1c0e78">Jeremy Gilmore</a> on <a href="https://medium.com/state-of-analytics/evolving-landscape-of-analytics-on-aws-63386b6dfacc">AWS</a>.</p><p>This brings me an interesting question given all this innovations from the cloud providers: how do tools solving a single step in the process compete and innovate to stay relevant? Looking at tools like Tableau, Alteryx, Datarobot, [insert all other competitors doing data viz, data eng, or data science], how do they augment their offerings to become more “platform-like”?</p><blockquote><strong>4. If you Lead a Data or BI or Analytics Group, Here’s a Friendly Warning…</strong></blockquote><p>You need to ensure you’ve created a data ecosystem that is cloud-first and provides all the leading tools to your data engineers, data scientists and analysts.<strong> </strong>Otherwise, <strong>you will be out of work in 2018.</strong></p><p>I’ve started to see this in multiple places this year, even at “Tech” companies, where IT leaders are clinging to legacy technologies and not meeting the pace needed from their stakeholders / users. Look, I get it, it’s what you know and you have a whole large team on and offshore who built a platform over a decade in [Insert legacy DB &amp; Tool]. It’s time to move on…</p><p>If you are promising your users that you’ll be enabling a new ecosystem, but the POCs are taking months, you have another problem to deal with. If you don’t more rapidly enable these services, then you’ll find yourself in a situation where those stakeholders / users are “going rogue” and deploying their own. I don’t blame them, it’s quite easy to get something up and running with AWS, GCP or Azure.</p><p>If your team lacks the skills with these new technologies, get them trained up and quickly! (It’s critical as you can’t take a group well versed in only Oracle and Informatica and expect them to understand the paradigm changes with BigQuery and Airflow, as an example.) Otherwise, reach out and I’m happy having a chat on how you can move more quickly.</p><blockquote><strong>5. Clean, Governed Data is Still the Biggest Challenge for Companies</strong></blockquote><p>I’ve wrote about this a few times, but I would love to see source / business applications get better and smarter at ensuring clean data is created, updated and removed in their applications. Could you imagine if Salesforce was smarter (out of the box / cloud) at ensuring data entered (New accts, opptys, etc) was correct?</p><p>This will allow companies to be less reliant on downstream tools for that cleanup. While these tools for cleanup will be needed in some form, cleaner upstream data should mean cleaner downstream.</p><p>Now, making source business applications smarter to ensure data is clean is just one step. Data needs to be governed in those systems and across business processes that touch that data. It’s not easy for many reasons, but a big reason for that is that we are undisciplined and don’t make it easy for business folks to be disciplined. So, as you start to better govern and put controls in place for clean data, remember that everyone will lack some degree of discipline so make it easier.</p><blockquote><strong>6. Pendulum Swings between Code-first Data Engineering and GUI based Data Engineering Tools</strong></blockquote><p>We see these pendulum swings frequently across database and analytics tools, but the current trend is that developer and engineer friendly are in. This is primarily due both to these tools working nicely with cloud provider environments and the flexibility in integration patterns. One of the most common combinations we see at Tech companies is using the combination of Python + SQL with <a href="http://airbnb.io/projects/airflow/">Airflow</a> as the orchestrator.</p><p>This approach is not for everyone and not every company, but as data engineering and data science become more symbiotic, it’s important that the tooling is aligned to what they like to use. Data scientists increasingly are using Python so having the data integration patterns leveraging Python is a useful approach. (See also theme #1)</p><p>There are some good GUI first data engineering and integration tools out there (See Matillion, SSIS, Informatica, etc.), but if you’re building a new platform that should align ultimately to factors of speed to creation, cost, what your team makeup is now and in the future, source connectivity and flexibility to change patterns.</p><blockquote><strong>7. The “best practices” of DevOps/SRE is Now the New Norm in Data</strong></blockquote><p>Everyone let out a collective YAY! I remember that the concepts of DevOps in data and analytics were quite abstract and challenging a few years ago, but that has changed dramatically.</p><p>Everything has to be versionable, trackable, reproducible, testable, and deployed seconds after leaving a developers fingers. This is the new norm and the ecosystems with the cloud providers make it easier than ever, and in containers no less!</p><p>This is important for a few important reasons:</p><ol><li>Reduction of manual deployment and code out-of-sync errors</li><li>Speed to deploy data transformation and data model changes reduce drastically. Long gone are the manual, week long deployment activites.</li><li>Automated unit and data quality tests greatly remove the onus on developers and QA functions for spot checking. Their testing can now be more focus on value added data validity tests.</li></ol><blockquote><strong>8. The Advent of Chief Analytics Officer now owning these data functions (vision, strategy) with IT as Support</strong></blockquote><p>Chief Data Officers came in vogue a number of years ago, primarily in the Financial and Healthcare industries given regulation and compliance. I won’t exhaust how well this is working, but an interesting theme of late if having a Chief Analytics Officer. These folks have less of a compliance and governance focus, and more of a using data-as-a-strategic-asset focus.</p><p>An important aspect of this role is having both an internal and external purview. They should be seen as champions internally of data, breaking down walls between Product, IT and Biz Ops groups but also being the external evangelist for your organization’s data.</p><p>We’ll see how many organizations start to adopt this role in 2018, stay tuned…</p><blockquote><strong>9. Data Sharing (and soon Data Exchanges) via New Tooling is Finally Becoming Mature Enough to Use</strong></blockquote><p>Snowflake, BigQuery, and Athena/Presto are finally making sharing data simple and easy, hooray! The developers and product folks for these technologies have smartly thought about dataset sharing as a feature needed to be adopted across data-centric orgs. Rather than having to rely on additional tools for this sharing, you can enable this directly in the databases themselves. (A tech enabler / unlocker has been splitting compute &amp; storage for databases. This is firmly established for these database technologies now.)</p><p>Governance and security are as important as ever, but when haven’t they been? (See theme #5 above)</p><p>This trend will also take shape in external data exchange markets. While the enabling technologies could look and feel different, this ability for companies to share and exchange data is starting to bubble up in interest. Imagine an ecosystem where companies who share customers (or industry data) could openly buy and sell relevant data (with appropriate agreement from those customers). This is both exciting and a bit scary at the same time given the criticality of security in ecosystems and exchanges like this.</p><blockquote><strong>10. AI/ML is Still Untapped, see Recent News about Generating Celebrity Faces as “unique” Application of Emerging Tech</strong></blockquote><p>Sigh…well, we’re still in the early days of wider adoption of AI &amp; ML. 2018 is going to see more and more applications of these emerging technologies. Will we reach an even higher hype pitch in 2018 for AI/ML? We will see, but there will certainly be more business applications of these technologies, many of which will never reach the public eye. We’ll see this, along with the pop culture applications, but I’m hopeful some of the applications help the greater good. For every “Celebrity Face Swapper”, it would be great to see an AI application in an area of public need.</p><p>I’ve believed this for years now, but today more than ever the field of Data &amp; Analytics is one of the most promising and fulfilling areas to be working in Technology.</p><p>I hope you enjoyed 2017 and where our industry is heading. Please add your thoughts to the comments section as I’m eager to learn what you observed this year.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cddc5e74f315" width="1" height="1" alt=""><hr><p><a href="https://medium.com/state-of-analytics/10-themes-observed-in-data-analytics-in-2017-cddc5e74f315">10 Themes Observed in Data &amp; Analytics in 2017</a> was originally published in <a href="https://medium.com/state-of-analytics">State of Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Looking forward at the Analytics Space in 2017]]></title>
            <link>https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2017-4edce4481e34?source=rss-f3456880de04------2</link>
            <guid isPermaLink="false">https://medium.com/p/4edce4481e34</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Kyle Roemer]]></dc:creator>
            <pubDate>Mon, 13 Feb 2017 19:42:12 GMT</pubDate>
            <atom:updated>2017-02-13T19:42:12.601Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TLjKo-wT2JEVBB12yRc1Aw.jpeg" /></figure><p>We’re 1.5 months into 2017 and I think I’m ready to predict what’s to come in the analytics space. I did a similar (albeit earlier in the year) exercise for 2016, which can be found <a href="https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2016-f657c0594e36#.4rn17sxc7">here</a>. I think many of these themes are still true, but they perhaps have a longer trajectory than a statement of change in 2016.</p><p>Now, what is going to take place this year in Analytics? I think there are 5 continuing themes for the year and I’ll do my best to steer clear of the over played themes like AI, MACHINE LEARNING, and IOT. (Disclaimer: I think these are all interesting, disruptive and happening in all sorts of places. However, the majority of those who will read this will not be enlightened by me proclaiming “AI is here to stay” or “Machine learning is taking over your coffee maker” or wait…)</p><p>Anyways, onwards to those 5 things in analytics that just might happen this year.</p><h4><strong>Someone, somewhere will make it easier to combine clickstream data and revenue / sales related data.</strong></h4><p>I’ve spent time with digital marketing groups at some large companies and I continue to see the challenge of combining clickstream and sales data. By clickstream data I mean web activity data (often provided by Google Analytics, Adobe or a myriad of other small tech companies). Combining this with customer sales related data is the challenge. [Caveat: most born of the web retailers are doing this well, because they invested in it. However, I haven’t seen many enterprise software companies do this well, yet.]</p><p>Some of the above web usage / analytics companies will tell you that they provide an easy to use, integrated view of your customer journey. FALSE. Those products likely provide detailed usage / activity on your various web properties, and perhaps even allow you to upload some sales related data. However, they are not providing a solution for navigating complex relationships between web visitors, paying customers, returning customers, etc. This is especially true when those visitors continue to use non-tracking software or flush the cache.</p><p>That’s enough backstory. I think that either one of the big players or an up and comer will make this abundantly easier. Especially for marketing groups that lack the technical abilities to create their own custom solution OR don’t have a fruitful relationship with their IT counterparts.</p><h4><strong>As many market analysts have predicted, I too think there will be more market consolidation in Data and Analytics.</strong></h4><p>I thought companies like Tableau would get purchased last year when they were at a nice discount. That may no longer be true, but I still think they are a great target for some of the legacy players or new(er) cloud companies.</p><p>I don’t think data prep companies can live a long life on their own, so they are also a prime target this year for either Single Discipline Toolsets (IE Data Viz, Data Governance) or mature ETL companies.</p><p>Lastly, I think many of the “easy to connect to your cloud data source” startups and earlier stage companies will falter over the next couple years. I talked about this in last years predictions, but believe it more fervently today. Unless they UP their value proposition, they won’t last.</p><h4><strong>Google plays catch up to AWS and Microsoft Azure in the enterprise cloud race, but they will catch up soon(ish).</strong></h4><p>A wise individual recently told me that Google’s foray into the cloud platform space has been akin to Microsoft 90’s strategy re: browsers. I thought this was really interesting and seems to align.</p><p>They have been a bit slower on the uptake here, compared to their large Seattle competitors. However, their innovations with Google Cloud Platform (GCP) and the underlying data technologies and libraries is exciting. Now, the Google ecosystem is still maturing. If you’re a company looking to leverage GCP for your app or data platforms, just know that you may be one of the earlier adopters. (IE not a ton of documentation, not a ton of similar companies that have tried with success or failures, and not as much talent in the marketplace.)</p><p>Lastly, who doesn’t like a 3 team race? It’s better for everyone really. I’m not rooting here for anyone in particular, but I do like the idea that we’ll see more frequent innovations, lower costs and options to deliver the next [insert buzzword] tech.</p><h4><strong>The quality of data and how it’s governed [managed, integrated, analyzed and reported on] is still a massive challenge for just about everyone but tools will <em>hopefully</em> make it easier!</strong></h4><p>This is the un-sexy part of data and analytics and it continues to be a massive challenge for everyone. I could explain why that is, but I think you know why.</p><p>So Kyle, how do we solve it? Not so fast. I’m not here to solve your problems (I have a day job for that) but I’m here to predict, perhaps, what may happen this year that could help. I wrote last year about companies like <a href="http://www.alation.com/">Alation</a>, which are tackling this problem head on with their easy to use tool. I still like what they are doing.</p><p>I think a larger theme here is that this will become more integrated into your data tools. The more mature data companies (think Oracle, SAP or IBM) all have created tools that help with this. However, they are mostly separate tools. Who wants that? The visibility and transparency for developers, analysts and end users needs to be integrated into their creation or consumption flow.</p><p>You might still be asking, WHY HASN’T ANYONE PRIORITIZED THIS? Here’s a fun activity for you, go ask your new data scientists if they want to document their data preparation process or clean up data from your Salesforce platform. Once you’ve done that, go ask your Director of Marketing and your Director of Sales Ops their definitions for MRR (Monthly Recurring Revenue). Report back in the comments on what you learn.</p><h4><strong>Experimentation will increasingly become adopted with proliferation of hypothesis driven analytics projects.</strong></h4><p>Companies like Google have integrated this philosophy into many aspects of their business, however it is not as common at many other organizations. With the influx of data scientists trying to use advanced statistical techniques to glean insights for their businesses, there is a need and a surge of experimentation approaches.</p><p>Generally, this will start with a business hypothesis such as “We believe we can increase qualified leads in our healthcare segment by 25% in Q1, compared to the prior 2 years.”</p><p>From there a series of activities will take place, but the important component here is that companies are starting with a strategic statement and proving or disproving that statement, with data. Ideally, it will lead to some real-world experiments like “We are going to remove all discounts from store x for 1 month to see if our premise for discounting and sales are accurate”.</p><p>This isn’t necessarily new as you’re probably thinking, but it will become more common in the business setting this year.</p><h4><strong>No prediction on companies to be on the lookout for this year…</strong></h4><p>Unlike last year, I won’t be adding a section on analytics companies to be paying attention to this year. It’s not for lack of interesting companies in the market, but I’ve yet to be thoroughly intrigued. It’s still early in 2017 and I know there are a number of yet to be revealed startups in the space, so more to come here. I do plan on posting in the next few months on some of these new companies.</p><p>That’s it. My predictions for 2017 and I only used AI, Machine Learning and IOT in the intro and conclusion. Enjoy the next 10.5 months.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4edce4481e34" width="1" height="1" alt=""><hr><p><a href="https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2017-4edce4481e34">Looking forward at the Analytics Space in 2017</a> was originally published in <a href="https://medium.com/state-of-analytics">State of Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Looking Forward at the Analytics Space in 2016]]></title>
            <link>https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2016-f657c0594e36?source=rss-f3456880de04------2</link>
            <guid isPermaLink="false">https://medium.com/p/f657c0594e36</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kyle Roemer]]></dc:creator>
            <pubDate>Mon, 11 Jan 2016 20:03:26 GMT</pubDate>
            <atom:updated>2016-01-12T00:25:09.218Z</atom:updated>
            <content:encoded><![CDATA[<p>Yes, this is another set of predictions you probably didn’t ask for but hopefully it will shed some light in the ever evolving analytics space. I lead an analytics consulting practice for <a href="http://www.slalom.com/"><strong>Slalom</strong></a> in San Francisco and I have the privilege of talking to many large &amp; small companies. It’s my role to understand what is taking place in the analytics market but equally important is understanding where things are heading. The list below is what I expect to happen in the next year, but as most predictions go will likely be inaccurate so read at your own peril.</p><h4>Data Viz &amp; Enterprise “BI” will Quickly Follow the Web Tech &amp; Digital Evolution</h4><p>I’m using Web Tech &amp; the Digital Design Evolution as a lens for where Data Visualization and Analytics Apps are heading. Too long have analytics applications lagged behind modern design &amp; development paradigms; customers are now demanding their enterprise analytics apps work in a similar fashion as the websites &amp; apps they use in their daily lives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6uhoh8fOcUEyfoGX2mEYUA.png" /><figcaption>Web Tech Evolution as visualized by evolutionoftheweb.com — we can expect to see Data Visualization &amp; Analytics Apps to follow this evolution more quickly than prior years.</figcaption></figure><p>A lingering design principle in web based analytics solutions is designing for many user roles; this results in cumbersome user flows, and ultimately makes it challenging for users to find the most pertinent content for them. While it’s necessary to have multiple roles within an analytics application due to security implication and functionality needs, it’s more important to start simple and allow the user to discover more functionality. An example of a couple sites that do this well in my opinion are Pinterest and Tumblr. While there are social aspects to content discovery on those sites, I think analytics applications can learn from the practices they employ.</p><p>While more modern design practices are being employed at new analytics companies, many of these experiences are unfortunately similar. When you start looking at startups like <a href="http://www.datahero.com/"><strong>DataHero</strong></a> (recently acquired), <a href="http://www.chartio.com/"><strong>Chartio</strong></a>, or <a href="http://www.argo.io/"><strong>Argo</strong></a> in the Data Viz space you see don’t a ton of differentiation, which brings me to my next thought…</p><h4>New Data Viz &amp; Analytics Companies Need to and Will Differentiate Themselves more than “Can connect to your favorite cloud source”</h4><p>This was the barometer for most new data viz &amp; analytics startups from 2013–2015 (not including predictive, large data, or storage related startups): we are easy to use, can connect to that cloud source you like, and we don’t cost all that much. While there was and is a place in the market for these companies, this has been done at length now. Who is going to tackle the lack of collaboration in analytics today? <a href="http://www.clearstorydata.com/"><strong>ClearStory Data</strong></a> is trying, but do we yet understand how people want to collaborate with data? Who will make a mobile experience that people <em>want</em> to use? <a href="http://vizable.tableau.com/"><strong>Vizable</strong></a> is interesting from Tableau, but it’s just the start.</p><p>New analytics companies started this year will hopefully move beyond the “simple interface, connect to cloud sources” paradigm of the last couple years and focus on unsolved issues in the enterprise.</p><h4>Deploying Predictive Models will get Easier</h4><p>A continued problem for companies that have data science teams building advanced models is the need to operationalize / deploy / productionalize. This isn’t a new problem but the proliferation of data science in the enterprise has magnified the challenge. Although we have standards like <a href="http://dmg.org/pmml/v4-2-1/GeneralStructure.html"><strong>PMML</strong></a> and newer standards like <a href="http://dmg.org/pfa/index.html"><strong>PFA</strong></a>, they haven’t quite hit the mainstream.</p><p>The reality of the issue is that data scientists like to use different languages to build their models (R, Python, Java, etc) and oftentimes don’t have the desire or background to automate and deploy models, regardless of scale. A common paradigm I see at most places is that a data science team builds the models and then hands them off to a data engineering team to deploy &amp; automate. There are few places I’ve seen this work well as the models can be complex, require validation and something get’s lost along the way in a highly iterative, discovery based environment. The Data Mining Group explains it well <a href="http://dmg.org/pfa/docs/motivation/"><strong>here</strong></a>. I expect a company or two to solve for this, likely leveraging these standards but lowering the barrier of learning / deploying the standards. Companies like <a href="http://zementis.com/"><strong>Zementis</strong></a> are focused squarely in this area with a leaning towards large scale deployments and datasets.</p><h4>Data Viz Programming Libraries will get Easier &amp; Easier to Use</h4><p>Libraries like <a href="http://d3js.org/">D3.js</a> continue to pick up momentum amongst analytics groups and I don’t foresee this declining. Given the prominence D3 has in the market, I expect it to get easier and easier to build leveraging these libraries. I’m also expecting more abstraction to lower the bar to building custom visualizations like <a href="http://c3js.org/">C3.js</a> or the fantastic work being done on <a href="https://vega.github.io/">Vega</a> by <a href="http://homes.cs.washington.edu/~jheer/"><strong>Jeffrey Heer</strong></a>. (This type of web first based approach to data viz is the path forward.) It’s always exciting to see the continued evolution of the D3 library by <a href="https://github.com/mbostock"><strong>Mike Bostock</strong></a><strong> </strong>as well as projects like <a href="http://blockbuilder.org/"><strong>Block Builder</strong></a> by Ian Johnson.</p><h4>As Analytics get “Easier to build”, Core Analyst Skills are More Paramount</h4><p>It’s now easier than ever to integrate data sources, visually explore that data and run advanced analytical models on that data. With tools like Tableau, Qlik, Alteryx and Paxata the bar has been lowered. This has opened up the ability for folks to explore and analyze their data in ways never available to them. Finance, Marketing, HR analysts are empowered to do things only “technical” individuals could do in the past. It’s been incredible to see at clients I serve!</p><p>With this ease of use, comes the need to have foundational skills and understanding of data to avoid mistakes / wrong conclusions. The idea that you can drag a R model in Alteryx to a dataset to predict an outcome is amazing; if you don’t know how to interpret that model or understand what is taking place in that model it could lead to false conclusions. Many have written on this, but my hope with these tools is that a continued education on data &amp; statistical analysis will happen in the workplace. Those core skills will continue to be paramount in drawing data driven conclusions.</p><h4>Hadoop will continue to be deployed at a higher frequency but there will be innovation on solving computing at scale with common languages and skills</h4><p>Computing at scale on commodity hardware has worked for a # of enterprises, although the silver bullet aspects of Hadoop seen through the last few years has been lessoned which is a good thing. Given the complexity of the Hadoop ecosystem and need for learning &amp; supporting new languages, I expect some innovation on lowering that barrier. We will see more push towards common languages like SQL and standard DBA skills vs proliferation of languages like Hive, Pig, etc. The Cosmos platform from Microsoft is a step in the right direction, and I’m interested in what <a href="http://www.mapd.com/"><strong>MapD</strong></a> will bring to the table as well.</p><h4>Data Viz “Mashups” will see innovation in a world where “one tool across the enterprise” will struggle to work</h4><p>Large enterprises oftentimes want to standardize on a single reporting, data visualization or analytics tool. While there are many pros to this argument, the reality is people want to use what they want to use…that could be because one tool is simple, another more powerful or they’ve used another for many years. Given that, I expect companies to start considering Mashups where they’ll allow their users to combine data visualizations from multiple tools that are supported within the organization.</p><p>The legacy word for this was a BI Portal, but those were mostly information in nature helping departments understand how to get a license, trained, etc on the tools available to them. The idea of a Data Viz Mashup is taking that a step further and allowing users to combine visualizations from all the tools they use to curate experiences &amp; stories. Now doing this elegantly won’t be easy as every tool’s API is different and there isn’t a great solution today to bridge these APIs. Alongside the API bridging complexity, there is a an inherent tool knowledge gap and required investment in learning the ins and outs of these tools to be able to create a mashup.</p><p>All that complexity aside, I think there are numerous use cases to make this a compelling solution (imagine being able to update your Google Sheet data that feeds your Tableau visualization in real-time). Expect to see more of this in 2016 and the idea of “one tool to rule them all” to fade slowly.</p><h4>The Next Tableau will be Born or Revealed</h4><p>I’ve discussed with my colleagues at length over the last couple years on who the next <a href="http://www.tableau.com/"><strong>Tableau</strong></a> would be in the marketplace. They’ve built an incredible following, an amazing product and have grown up as a company. This has quickly led to them becoming the enterprise standard in data visualization among many major companies. (Replacing traditional players like Oracle, IBM, MicroStrategy and Microsoft.) While they are the “it” toolset to use for analysts today, this can quickly change.</p><p>In 2016 I expect Tableau’s hold on the market to continue, with hopefully an ever evolving product, but I also expect a newcomer to make a lot of noise. Who will that be and will it be in 2016? Well, I expect a company or two to make traction this year at larger enterprises. Companies like <a href="http://www.looker.com/"><strong>Looker</strong></a> have a great chance as they truly differentiate themselves (with their powerful SQL abstraction and analyst collaboration) vs being another data visualization tool. I’ve always found <a href="http://www.clearstorydata.com/"><strong>ClearStory Data’s</strong></a> product interesting as they’ve focused on a few, key areas.</p><p>My hope is that someone, somewhere starts or evolves a company that can truly compete with Tableau. They are poised to grow even more across the enterprise. Do I expect someone to innovate to the degree that Tableau did in the same data viz space this year, no, but there are huge opportunities to address the areas they and others are not today. Areas like collaboration, mobile, customization and engineering friendly tools are all areas a company can gain traction around.</p><h4>Analytics Companies you should be paying attention to in 2016</h4><p><a href="http://www.looker.com/"><strong>Looker</strong></a>: I’ve known the team for a couple years now, and their commitment to building an application for modern data companies is applaudable. Their SQL abstraction layer is powerful and they could very well be poised for making a big jump this year. They’ve focused on evolving an interface that allows data engineers &amp; analysts to collaborate and build models in a single environment. Lately, however, they are in the same conversations at customers considering Tableau, Qlik, etc.</p><p><a href="http://www.alation.com/"><strong>Alation</strong></a>: One comment I’ve heard over and over again across large Tech, Financial Services, Retail and Healthcare companies is “I don’t know what data is available to me or what reports have been built.” I’ve seen a # of IT departments aim to build solutions that expose this type of information, but oftentimes these solutions can’t be supported due to engineering cost, capacity, etc. Alation is tackling this challenging topic and it couples quite nicely as companies look to better govern their data. I expect Alation to make big moves in the market and/or more likely get acquired this year.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f657c0594e36" width="1" height="1" alt=""><hr><p><a href="https://medium.com/state-of-analytics/looking-forward-at-the-analytics-space-in-2016-f657c0594e36">Looking Forward at the Analytics Space in 2016</a> was originally published in <a href="https://medium.com/state-of-analytics">State of Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>