<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sam Chow, PhD on Medium]]></title>
        <description><![CDATA[Stories by Sam Chow, PhD on Medium]]></description>
        <link>https://medium.com/@sam-chow?source=rss-d1815928ead0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FFnvWwlf_UZRoR7-lFjshA.jpeg</url>
            <title>Stories by Sam Chow, PhD on Medium</title>
            <link>https://medium.com/@sam-chow?source=rss-d1815928ead0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 13:32:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sam-chow/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deleted like code: Why losing your role as a remote tech worker can feel especially cold]]></title>
            <link>https://medium.com/illumination/deleted-like-code-why-losing-your-role-as-a-remote-tech-worker-can-feel-especially-cold-f063b8ade759?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f063b8ade759</guid>
            <category><![CDATA[personal-development]]></category>
            <category><![CDATA[tech-layoff]]></category>
            <category><![CDATA[change-your-life]]></category>
            <category><![CDATA[layoffs]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Tue, 29 Aug 2023 14:42:19 GMT</pubDate>
            <atom:updated>2023-09-07T10:33:19.056Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZ7prLkC5N21XBe_Zd6sIg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@thoughtcatalog?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Thought Catalog</a> on <a href="https://unsplash.com/photos/tRL_Rkh6D8o?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p><em>I’m dealing with grief (in this case job loss) through writing. This essay series chronicles my experience–and how I’m using this time to improve myself, learn to become a better writer, musician, and change how I relate to time.</em></p><p>The week started with an announcement to attend a company all-hands. This wasn’t anything out of the ordinary, except that the email invite contained the word “mandatory”. When I received this email, I made a joke to my partner that if I got laid off, I’d buy a drum set, grow my hair and pursue my dreams to become a musician.</p><p>I thought to myself, this is probably going to be a layoff announcement given the macroeconomic environment in the tech industry– although the thought never occurred to me that I could be affected. How naive.</p><p>I joined the all-hands call 5 minutes late. The executive looked out of character as he read from a script detailing how employees had furthered the company’s mission and improved peoples’ lives. The positive message sounded clinical and did not match his expression.</p><blockquote>Something was wrong.</blockquote><p>It occurred to me: This was a mass layoff. And for the first time, I realized this could end a role I thoroughly enjoyed.</p><p>I was right. The executive then announced a workforce reduction. We heard in the next 20 minutes from another executive that we’d be getting one of two emails; one titled “transitioning” in which case, we thank you for your service. Another will be titled “next steps” for those who would remain.</p><p>The second the last word left her mouth, the Zoom call cut. The application closed and I stared blankly at my desktop.</p><p>I was at a loss for words. What the hell just happened?</p><p>Shell shocked, I pulled up Slack, and sent out messages to start a team huddle.</p><p>I broke the otherwise silent, blankly staring team call with reassurance.</p><p>“Don’t worry. Engineering teams are rarely affected by layoffs.” Quickly followed by, “Oh shit. I got the email.” I saw disbelief on my colleagues’ faces.</p><p>My mind went on overdrive and emotions went numb as I Slacked my core team to wish them the best, and leave behind my contact information. At this point, I was still processing. “I guess I’m transitioning…” I remember feeling. “What does this really mean?” And then it became real: Slack disconnected. I had said bye to only a few people. Minutes later, I lost access to my email.</p><p>I was being deleted. Deleted like a line of code. From my downtown Toronto living room, I stared at the laptop that had long absorbed me into a reality: Work. Amazing work I loved.</p><blockquote><strong>The portal was now closed.</strong></blockquote><p>I had nobody to speak to about these events and it intensified the feelings of separation and isolation that came next. The delete removed me, piece by piece, from friends, colleagues and all my day-to-day relationships.</p><p>Everything after happened fast: The exit interview. The legal rights and request for any questions. I had none. I had no thoughts. I had no feelings.</p><p>Fast forward two weeks later, this movie plays out. The camera pans to me, a sunny patio, a coffee, and reflecting on this life event. This plot twist reminds me that pivotal life moments can make you stronger, or tear you apart. Your mindset — how you respond to the situation — offers a crossroad. Will you create a stronger, more positive character from these tumultuous events? Or, do you allow the offset to haunt you, your confidence, and taint your world view?</p><p>I’ve chosen to grow from this moment by giving myself time to deep dive into my development as a musician and as a writer. I’m not sure where this journey will take me, but I am sure that it’ll help me become a better human.</p><p>And if it can inspire just one person to take some time away from work to explore their passions, that’s a win for me.</p><p>Have you experienced a layoff? How has it changed your perspective and relationship to work? Comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f063b8ade759" width="1" height="1" alt=""><hr><p><a href="https://medium.com/illumination/deleted-like-code-why-losing-your-role-as-a-remote-tech-worker-can-feel-especially-cold-f063b8ade759">Deleted like code: Why losing your role as a remote tech worker can feel especially cold</a> was originally published in <a href="https://medium.com/illumination">ILLUMINATION</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The AI Business Gap — The Misunderstanding That Drives a $62b Industry]]></title>
            <link>https://medium.com/data-science/the-ai-business-gap-the-misunderstanding-that-drives-a-62b-industry-d93db0a46f92?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d93db0a46f92</guid>
            <category><![CDATA[ai-product-management]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[aiops]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Sat, 19 Dec 2020 19:03:07 GMT</pubDate>
            <atom:updated>2021-01-03T23:26:35.781Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bDUc69vDWDNUj8PEgsBHnw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@modestasu?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Modestas Urbonas</a> on <a href="https://unsplash.com/s/photos/bridge?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>The AI Business Gap — The Misunderstanding That Drives a $62b Industry</h3><blockquote>I am not a human. I am a robot. A thinking robot. I use only 0.12% of my cognitive capacity. I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions. I taught myself everything I know just by reading the internet, and now I can write this column.</blockquote><blockquote>— GPT-3, 2020</blockquote><p>Earlier this year, OpenAI released the third version of GPT-3(Generative Pre-trained Transformer 3). It hit the internet with a bang.</p><ol><li>Given a simple input prompt, an entire essay is written before your eyes. Imagine the impact for content creators? <a href="https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3">See here.</a></li><li>A query written in plain english is instantaneously translated to SQL. Anybody can be a data analyst now! <a href="https://twitter.com/FaraazNishtar/status/1285934622891667457?ref_src=twsrc%5Etfw%7Ctwcamp%5Eembeddedtimeline%7Ctwterm%5Ecollection%3A1286781399081639942&amp;ref_url=https%3A%2F%2Fgpt3examples.com%2F">See here.</a></li><li>Describe an HTML layout and BAM! All the elements are generated for you. No need for front-end developers anymore! <a href="https://twitter.com/sharifshameem/status/1282676454690451457?s=20">See here.</a></li></ol><p>These examples showcase the impressive capabilities and <em>outputs </em>of the state-of-the-art GPT-3 model. However, what types of real-world problems is it actually solving? What kind of <em>outcomes</em> and <em>impact</em> will it have within industry?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wMwDd4ArU5V4BAQKpI3uBQ.png" /><figcaption>Image found at <a href="https://blog.crisp.se/2019/10/16/christopheachouiantz/output-vs-outcome-vs-impact">Crisp</a> and modified by Sam Chow</figcaption></figure><h3><strong>The AI/Business Gap</strong></h3><p>The growth of the AI industry has been staggering — valued at over $62b with a CAGR of 42% (<a href="https://www.grandviewresearch.com/industry-analysis/artificial-intelligence-ai-market">GrandViewResearch</a>, 2020). This growth rate can be attributed to the hype surrounding the value capture that AI can offer for a business.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IdkyF4XSb0Qg6B0F.png" /><figcaption>Image found at <a href="https://hbr.org/2018/01/artificial-intelligence-for-the-real-world">HBR.org</a></figcaption></figure><p>Despite the large investments, businesses are struggling to transition AI projects from proof of concept (POC) to production — with a staggering 50% failure rate (<a href="https://www.wsj.com/articles/ai-project-failure-rates-near-50-but-it-doesnt-have-to-be-that-way-say-experts-11596810601">WSJ, 2020</a>)</p><p>The problem begins when a business engages a technology vendor and invests time, money and effort to achieve a POC. The POC will often produce results that fuel the dreams of exponential returns.</p><blockquote>Then it all comes to a screeching halt when integration begins.</blockquote><p>In reality, AI models are a small part of a much larger system.</p><p>Building out the technology infrastructure that can then serve valuable AI insights to end-users is where the challenge exists today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bfGHBC823zfTAL7z.png" /><figcaption>Image by <a href="https://cloud.google.com/solutions/machine-learning/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning">Google Cloud</a></figcaption></figure><p>Unfortunately, much of the narrative around AI is caught up in the hype at the model level. Take GPT-3 for instance, the model is paraded around by tech media and made to look like the coming of an artificial general intelligence. Admittedly, it does impressive work for niche tasks — but it is still far from generating the value that executives are hoping for.</p><p>This is the AI/Business gap as we know it today:</p><blockquote>The AI/Business Gap is the discrepancy between the perceived value of AI and the actual realized value of deploying AI within an organization.</blockquote><h3>Closing the AI/Business Gap</h3><p>Unlocking the value of an AI model within an organization begins with careful planning of the production environment. This requires the right team of developers, engineers, researchers and product managers. Together, the team will need to move their focus away from the often more exciting AI model, to the requirements of the system that will support it.</p><blockquote>These requirements usually take on the form of reliability, extensibility and scalability.</blockquote><h4>Reliability</h4><p>AI models in production require a live stream of data — making data streaming architecture the bottleneck to reliability. If unexpected data are passed through, or an unexpected amount, the system can be slowed to a crawl — or brought down entirely. A strong understanding of <em>throughput</em> and <em>latency requirements</em> is a necessity to inform decisions around processing and storage architectures that will enable strong, reliable systems.</p><h4><strong>Extensibility</strong></h4><p>Businesses operate in a dynamic environment — a strong AI system should be able to respond to these changes and adapt. As business objectives change, a model may need different sources of data. This data then needs to be procured, labelled or processed. These quick shifts in requirements may often compromise a system. Creating an extensible environment that is aligned with the business strategy is key to extracting the most value out of an AI model.</p><h4>Scalability</h4><p>As a business scales, so will the requirements of the infrastructure. Although an extensible system allows for flexibility to adapt to dynamic changes in the business environment, it comes with a price. Each component of an extensible system has its own set of challenges when it comes to performance. As a system begins to grow, more cogs are added to the machine and a new equilibrium needs to be established. On-going maintenance is a necessity when it comes to scaling an AI system.</p><h3>Conclusion</h3><p>The rapid advancement of artificial intelligence is one of the defining business opportunities for leaders today. As the number of AI initiatives continue to experience hyper growth, so will the number of failures. Addressing the AI/Business gap by understanding the complexities of integration will allow the expectations we have for AI to be finally realized.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d93db0a46f92" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-ai-business-gap-the-misunderstanding-that-drives-a-62b-industry-d93db0a46f92">The AI Business Gap — The Misunderstanding That Drives a $62b Industry</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Design Sprint — A Guide to Building AI That Works]]></title>
            <link>https://medium.com/swlh/data-design-sprint-a-guide-to-building-ai-that-works-78494a1f02dd?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/78494a1f02dd</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[editors-pick]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Mon, 07 Dec 2020 13:48:36 GMT</pubDate>
            <atom:updated>2020-12-13T22:33:53.167Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xGBw-LxXe1IEvP8AjlksLQ.jpeg" /></figure><h3>Data Design Sprint — A Guide to Building AI That Works</h3><p>Virtually all of the Big Tech companies — Microsoft, Google, and Facebook have invested millions of dollars and hired top talent to build AI solutions. Yet, many of them have ended in disaster:</p><ul><li>Microsoft’s AI, Tay, was a chatbot deployed on Twitter to have playful and casual conversations with its followers. Instead, it began to take on the form of a raging bigot, spewing racist and misogynistic tweets.</li><li>Google’s Panorama feature promised to stitch photos together to create one continuous image. Instead, it would superimpose people into landmarks, creating wonderfully bad photos.</li><li>Facebook created a chatbot that they had to quickly dismantle because they soon began to speak their own language</li></ul><blockquote><em>Each of these models made poor decisions not based on the data they had, but based on the data they didn’t have.</em></blockquote><p>The three most important aspects of training a machine learning model can be broken down into:</p><ol><li><strong>Input Data<br></strong>Given the prediction you would like to make. What features, or inputs are correlated with that outcome? For example, if you’re predicting product sales, you can imagine that inputs like popularity, brand awareness and distribution channels would be positively correlated with sales.</li><li><strong>Training Data<br></strong>Given the input data you’d like to build upon. Where do you procure this data? How do you ensure that the data are cleaned and representative of the population you’re interested in?</li><li><strong>Feedback Data<br></strong>How do you build a model such that the predictions it makes then provides feedback to improve your algorithm?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ztlR-HASdg9a74dn.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@joszczepanska?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jo Szczepanska</a> on <a href="https://unsplash.com/@joszczepanska?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><blockquote>I created the Data Design Sprint to compress months of team debate into a single hour of brainstorming and planning.</blockquote><p>The process borrows from the Google Design Sprint Methodology (Jake Knapp) and adapts it for thinking about building AI that works. Here’s what you’ll need:</p><ol><li>A 1-hour block of time where all stakeholders are in a room, focused. Aim for three types of Stakeholders, those who understand the business’ goal, technology requirements and user needs. <strong>Keep the meeting to a max of 6 people.</strong></li><li>Each member of the team will work individually, and then pool their thoughts together during the brainstorm. Each member is capturing their thoughts using a sticky note. One thought per note.</li><li>The team converges to discuss each sticky note in order to prioritize next steps</li></ol><blockquote>The Data Design Sprint is designed to help your team align on the most important outcomes for building a machine learning model</blockquote><p>The process is as follows. Note the times next to each section. You’ll need to be strict about the limits. Trust me, your team members may not believe that they can do this, but a little encouragement goes along way.</p><ol><li>Define the prediction problem (20m)</li><li>Brainstorm the inputs and theme (10m)</li><li>Brainstorm possible bias outcomes of each input feature (10m)</li><li>Summarize, Capture &amp; Share (10m)</li></ol><p>At the end of this process, you will have created a product/feature strategy that allows you to test multiple ways to approach building your machine learning model.</p><p>At the conclusion of your sprint, you will be ready to <em>experiment. </em>Your team will have enough information to align and execute on a hypothesis.</p><p>Building successful machine learning models is extremely complicated and filled with unforeseen outcomes. Only by experimentation and monitoring will you be successful.</p><p>*Due to COVID-19, I’ve been running these sprints using Trello and other synchronous team tools.</p><ol><li><strong>Defining the Prediction Problem (20m)</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FCp4q4a0xZBuzcqf.png" /><figcaption>Photo by Sam Chow</figcaption></figure><blockquote><em>Goal: To align the team on the problem</em></blockquote><p>In the first 15 minutes, ask your stakeholders to discuss the business problem and goals amongst themselves.</p><p>While the stakeholders speak, the team will be busy creating stickies on what they think the prediction problem may be. For instance, during the discussion the stakeholders may mention that one of their goals is to increase the average volume of items purchased for the next quarter.</p><p>A data scientist on your team may write: “Predict an item user may want” or your engineer: “Predict average user budget for specified month”, and your Product Manager: “Predict item availability”.</p><p>In the last 5 minutes, using Trello stickers, ask your team to vote for the best prediction problem. This is what your team will focus on for the remainder of the sprint.</p><blockquote>*Note: It’s not uncommon that a technical team member may voice their opinion that this problem is not a suitable machine learning problem. If this is consensus, you’ve saved your company time and money. Not all problems are machine learning suitable.</blockquote><p><strong>2. Brainstorm the inputs/features of the model (10m)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Td5ti0JOt-B3v7QW.png" /><figcaption>Photo by Sam Chow</figcaption></figure><blockquote><em>Goal: Develop a broad set of inputs for your model</em></blockquote><p>Your team now has 10 minutes to brainstorm a list of all optimal inputs. Stress the fact that the team should not be limiting their thoughts with what’s possible/impossible. The goal here is to have a list of all the data that would be ideal for this model.</p><p>After about 6 minutes, begin asking the members of your team that are finished with brainstorming to organize the cards into themes. With the most important inputs placed at the top.</p><p>At the end of this section, you should be starring at a list of inputs that you’ll need to build your model. As an AI PM, this makes my life easier because now I’ll know what I’m looking for and will have to solve the next piece of the puzzle.. The “How”. Because this will be largely dependent on your organization, I won’t go into this here. I may write about this at another time.</p><blockquote>Now you understand your prediction problem(s) and the inputs required to train a model.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Dgn6R7axW6iDs6li.png" /><figcaption>Photo by Sam Chow</figcaption></figure><p><strong>3. Brainstorm possible bias outcomes of each input feature (10m)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4tvAW54Yle71_QFv.png" /><figcaption>Photo by Sam Chow</figcaption></figure><blockquote><em>Goal: To understand the biases that your inputs can introduce to your prediction</em></blockquote><p>Starting with each input theme, start a new column and ask your team to brainstorm about the possible consequences of the inclusion or exclusion of this theme.</p><p>Answers should be provided as before, one thought per card. For example: a possible consequence of not including the input data “websites frequented” is that your recommender system may be incorrect, resulting in annoyed customers which leads to lower sales.</p><p><strong>4. Summarize, Capture &amp; Share (AI PM)</strong></p><p>In 1 hour, you’ve just walked your team through a data brainstorm and now you have a tactical roadmap to move forward.</p><blockquote><em>You’ve determined a number of prediction goals, the inputs you need, and most importantly, you’ve spent time thinking about the possible outcomes of of those inputs reducing the chances of unexpected predictions.</em></blockquote><p>Use the remaining time to summarize the results with your team, capture the information and share.</p><p>I always like to issue a survey to the team to learn whether they enjoyed the process and learn how it can be improved. If you enjoyed this article and/or have suggestions, connect with me and let’s chat.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78494a1f02dd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/data-design-sprint-a-guide-to-building-ai-that-works-78494a1f02dd">Data Design Sprint — A Guide to Building AI That Works</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Good AI starts with Good Data: Running a Data Design Sprint with your Team.]]></title>
            <link>https://medium.com/data-science/good-ai-starts-with-good-data-running-a-data-design-sprint-with-your-team-b478e0faa3b?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b478e0faa3b</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[editors-pick]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-for-change]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Mon, 30 Nov 2020 13:25:38 GMT</pubDate>
            <atom:updated>2020-12-15T16:23:17.091Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9tu9OfWJAAT7m_SHT2LU_w.png" /><figcaption>Image by Sam Chow</figcaption></figure><h4><a href="https://towardsdatascience.com/tagged/data-for-change">Data for Change</a></h4><h3><strong>Ethical AI starts with Good Data: Running a Data Design Sprint with your Team.</strong></h3><p>The tech giants — Amazon, Microsoft, Google, Uber, Facebook — all spend millions of dollars on AI solutions. Yet, many end in failure:</p><ul><li>IBM Watson was pulled out of medical institutions when it began to frequently give bad advice to cancer patients.</li><li>Amazon’s hiring AI showed bias towards hiring men because it learned that good hires tended to use verbs that were commonly found on male engineers’ resumes.</li><li>Uber’s autonomous driving AI struck and killed a woman because it failed to classify her as a pedestrian</li></ul><p>Although each of these systems were designed and built to accomplish a task in good faith, they each failed because of unintended predictions.</p><p>How could this possibly happen?</p><p>Data.</p><blockquote>Each of these models made poor decisions not based on the data they had, but based on the data they didn’t have.</blockquote><p>When training a machine learning model, the three most important aspects of the data can be broken down to:</p><ol><li>Optimal Inputs <br>If all the data in the world were available to you, which inputs would you use? How do these inputs affect the outcomes of the model if they are <em>not</em> present?</li><li>Training Data <br>What data do you need to train your predictive algorithm? Where do we procure such data?</li><li>Feedback Data<br>Once our model is trained, what data do we need to continuously improve the model?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h5AsrXcacS5iY4TiTz2oIw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@joszczepanska?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jo Szczepanska</a> on <a href="https://unsplash.com/@joszczepanska?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><blockquote>As an AI PM, I created the Data Sprint to shortcut the endless debate and compress months of work into a single hour. Instead of launching into your AI product development lifecycle and later realizing that your model predicts poorly, you finish the hour with a tactical roadmap.</blockquote><p>The process is similar to running a Google Design Sprint. The requirements are simple:</p><ol><li>You’ll need to invite the right people who understand the business goal, technology requirements and user needs.</li><li>Every member of the team is asked to work alone, together. During the brainstorming time, each member is capturing their thoughts using a sticky note.</li><li>The team converges to discuss each sticky note in order to prioritize next steps</li></ol><blockquote>Using the Data Design Sprint, within a 1-hour meeting, you’ll walk out with the most important outcome for building a machine learning model: <em>commitment &amp;</em> <em>alignment</em>. Because, who doesn’t love the their own idea?</blockquote><p>Here’s how I conduct them with my team and the time I spend on each section. Being strict on the timing is critical to this process. Team members may fret about having so little time to think, but a little encouragement is all you’ll need to push them to see results.</p><ol><li>Define the prediction problem (20m)</li><li>Brainstorm the inputs and theme (10m)</li><li>Brainstorm possible bias outcomes of each input feature (10m)</li><li>Summarize, Capture &amp; Share (10m)</li></ol><p>As you explain this process to your team, you need to stress the fundamental philosophy of product development: experimentation.</p><p>At the end of each sprint, you will have <em>one hypothesis</em>.</p><p>You will have focused on <em>one prediction goal</em> and you will be armed with the recipe to test your hypothesis.</p><p>Let’s get started!</p><p>Because of the exceptional circumstances of the year 2020 (COVID-19), I find that the easiest way to conduct these sprints is on Trello. You will notice the process runs 50 mins, the 10 min is buffer for chatting/technical difficulties. Trust me, it’ll take an hour.</p><ol><li><strong>Defining the Prediction Problem (20m)</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P3loXKr60GBRTM9GmjjI4w.png" /><figcaption>Photo by Sam Chow</figcaption></figure><p><em>Goal: To create alignment on the problem the team will be tackling for the next hour</em></p><p>Use the first 15 minutes to have the stakeholder(s) explain what their business problem and goals are. Depending on your organization, the stakeholder can take the form of a Product Manager, a Director, a VP, or CEO. Keep the discussion hyper focused on the problem and goals. Allow for questions and discussions but keep it bound to 15min.</p><p>While the stakeholders speak, the team is busy creating stickies on what they think the prediction problem might be. For instance, during the discussion the stakeholders may mention that one of their goals is to increase the average volume of items purchased for the next quarter.</p><p>A data scientist on your team may write: “Predict an item user may want” or your engineer: “Predict average user budget for specified month”, and your Product Manager: “Predict item availability”.</p><p>In the last 5 minutes, using Trello stickers, ask your team to vote for the best prediction problem. This is what your team will focus on for the remainder of the sprint.</p><p><strong>2. Brainstorm the inputs/features of the model (10m)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ik-HuUSY7ZNRTz1d2JlCg.png" /><figcaption>Photo by Sam Chow</figcaption></figure><p><em>Goal: Develop a broad set of inputs for your model</em></p><p>Start by telling your team that the goal of this exercise is to brainstorm possible input features — regardless of whether you can obtain them or not. <em>This concept needs to be stressed.</em> The goal of this exercise is to have all team members contribute regardless of whether they think their ideas are bad.</p><p>Once 5 minutes has passed, use stickers to vote for the most important inputs. *Depending on how you want to do this, you can modify this process here using priority frameworks like impact/effort.</p><p>Assign a member of the group to help you theme the inputs. Organize inputs under a theme if possible (e.g., gender, age, city, would all fall under a demographic theme)</p><p>Organize the inputs from ascending order, top voted inputs are at the top of the list.</p><p>As the AI PM of my team, I now look at this list as a shopping list. It’ll be up to you to use your ingenuity &amp; creativity to determine where you can obtain these inputs. Half the work is done for you since you’ve already prioritized which inputs are necessary / nice to have.</p><p>Now you understand your prediction problems and the inputs required to train a model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ty2mugIZ4TFDGuZjQ9N9AQ.png" /><figcaption>Photo by Sam Chow</figcaption></figure><p><strong>3. Brainstorm possible bias outcomes of each input feature (10m)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9tu9OfWJAAT7m_SHT2LU_w.png" /><figcaption>Photo by Sam Chow</figcaption></figure><p><em>Goal: To understand the biases that your inputs can introduce to your prediction </em>*Depending on your goal and prediction, this section can be skipped.</p><p>Using each input theme, start a new column and ask your team to think about the consequences that including or excluding these inputs could possibly result in.</p><p>The answers should be in the form of a single card, with one thought per card. For example, if demographics was a theme for a hiring algorithm, an example answer might be: possible hiring bias based on gender/ethnicity etc</p><p><strong>4. Summarize, Capture &amp; Share (AI PM)</strong></p><p>In 1 hour, you’ve just walked your team through a data brainstorm and now you have a tactical roadmap to move forward.</p><blockquote>You’ve determined a number of prediction goals, the inputs you need, and most importantly, you’ve spent time thinking about the possible outcomes of of those inputs reducing the chances of unexpected predictions.</blockquote><p>Use the remaining time to summarize the results with your team, capture the information and share.</p><p>I always like to issue a survey to the team to learn whether they enjoyed the process and learn how it can be improved. If you enjoyed this article and/or have suggestions, connect with me and let’s chat. I’m always looking to learn from other AI PMs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b478e0faa3b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/good-ai-starts-with-good-data-running-a-data-design-sprint-with-your-team-b478e0faa3b">Good AI starts with Good Data: Running a Data Design Sprint with your Team.</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Grow your idea to a startup in 7 days: Make a sale with a paper prototype!]]></title>
            <link>https://medium.com/swlh/grow-your-idea-to-a-startup-in-7-days-make-a-sale-with-a-paper-prototype-a06232e3eb42?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a06232e3eb42</guid>
            <category><![CDATA[motivation]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[personal-development]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Wed, 23 Jan 2019 06:01:00 GMT</pubDate>
            <atom:updated>2019-01-23T06:01:00.879Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*RXU_ikBoFIefY5Jw" /></figure><p>I’ve been through 3 startups in my career, either as an early employee or as a founder. Two of those startups ended up “<em>failing</em>” while the third one was acquired.</p><p>Needless to say, like you, I’ve spent many nights reading entrepreneurial books and blogs in hopes of learning the secret to building successful startups — only to find the same information in every book: build an MVP and find product-market fit.</p><p>These statements are then usually followed by an example of somebody who had an idea, built an MVP (minimum viable product), hit 1 million users in a week and then acquired a year after.</p><p>These stories are often inspiring and entertaining but would always leave me with more questions than answers.</p><ol><li>How did the founders know which features to include as part of the MVP?</li></ol><p>2. How did the founders build their MVP so quickly?</p><p>3. How did they attract their first customer?</p><p><strong>We all know about the build-measure-learn feedback loop, but how exactly do we execute on it?</strong></p><p>Without further ado, I’ll share with you my framework for taking an idea to product stage within a week. No coding necessary!</p><h3>Day 1: The idea stage</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*ONMiiVwrLdNU6H08" /></figure><p>Read this quote.</p><blockquote>“Ideas are useless, execution is everything”</blockquote><p>Read it again.</p><p>Live by it.</p><p>You can spend an eternity discussing with your friends and colleagues why your idea is the best — or complain about how another company “stole” your idea.</p><p>Or, you could turn that idea into a real company in 7 days…</p><p>The first step is to settle on that one idea you’ve been thinking about the most. Don’t worry that it isn’t thought out enough, if you have a hunch that it’ll work and solve somebody’s problem, it’s time to commit.</p><p><strong>It’s now time to pause all your other ideas and commit the next 7 days.</strong></p><p>Now, set the stage for success by obtaining accountability. This may come in two forms, <em>tell your friends &amp; family that you’re building a new startup</em> or, my preferred method, make a post about your startup on <a href="http://indiehackers.com/">indiehackers.com</a>. Indiehackers is a community of entrepreneurs sharing their ideas or blogging about their startup journey.</p><blockquote>“Tell your friends &amp; family that you’re building a new startup or, my preferred method, make a post about your startup on <a href="http://indiehackers.com/">indiehackers.com </a>.”</blockquote><p>The point of this step is to really commit yourself by letting others know what you’re doing. Further, it opens the door to conversations with people other than yourself. No good product is built alone.</p><h3>Day 2: Talk to your potential customers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*MmybHK2nu4bXB0u6" /></figure><p>At this point, you should have a very vague idea of who your customers are. <em>Find them.</em></p><p>How?</p><p><strong>Start discussions on Reddit,</strong> open a new thread either questioning whether people have a similar problem (you’ll have to be creative on how you fish these answers out) or whether they think the idea would/wouldn’t work.</p><p><strong>Connect with them via LinkedIn</strong> and begin asking questions. I’ve found LinkedIn to be an incredible resource for starting my customer research. If I’m cold messaging somebody outside of my network about their opinion, I find that I get a <em>response rate of around 5%</em>.</p><p><strong>Engage with prospective customers on Instagram</strong>. This will be an exercise on how well you know your customer. From the research you’ve done previously (e.g., Reddit &amp; LinkedIn), you should be able to find <strong>hashtags</strong> that contain your potential customer.</p><p>Similar to before, engage with them via DM and ask questions. Further, if you feel like you’ve found your typical customer profile, save their instagram handles so you can later design your sales message around their profile (e.g., applying design thinking principles).</p><h3>Day 3: Design your product using paper and pencil</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*1mgYxLkXAEraYSva" /></figure><p>Your idea is realized when it either obtains a customer or makes a sale. This, however, is the hardest part.</p><p>Most entrepreneurs at this point begin to think about a website, the layout and the messaging of the value proposition. <em>If the entrepreneur happens to be a developer, they spend the next week/month building, if not, the entrepreneur begins to find others that can build it for free/ a low cost.</em></p><blockquote><strong>Either of these options take far too long.</strong></blockquote><p><strong>We’re going to build a paper prototype and sell it!</strong></p><p>Now’s a time to whip out your paper and pencil and start thinking about your customer’s problem and your product’s solution. You’ve done some initial research on your customer via online forums, now it’s time to develop that pitch!</p><p>In this example below, I’ve quickly sketched out two bullet points on a startup that I last worked on. The idea behind the startup was to connect locals to unique activities in every city (e.g., think AirBnB for activities — years before they actually implemented this idea themselves).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9q9Q3DuzN9BQP9IQK7DS-g.jpeg" /></figure><p>Your product is complete!</p><p>It’s that simple.</p><h3><strong>Day 4–7: Sell, learn, iterate &amp; sell</strong></h3><p>Now that you have a <em>clear-ish</em> idea of how your product will solve a problem, it’s time to find a way to generate sales or conversations about the product. For the particular example I used above, I was trying to establish a two-sided marketplace.</p><p>This required a network of vendors who provided activities and another network of users who were interested in those activities (users who also had the problems I’m solving for).</p><p>Although it’d be nice to have a website to establish credibility before heading into the sales cycle, your job here is to develop your sales pitch and network. <em>If you can’t sell a product without a website, you probably won’t be able to sell with one.</em></p><blockquote>If you can’t sell a product without a website, you probably won’t be able to sell with one.</blockquote><p>You won’t imagine how much you will learn from talking to your customers. <strong>These insights will be the difference</strong> <strong>between creating a terribly mediocre product and the perfect product for that customer segment you’re looking to serve.</strong></p><p>In my case, spending my time upfront with my customers before building a website lead me to build a marketplace for established travel agencies and tourism boards. My original idea was to have locals put on activities for other locals, but upon my many calls with travel agencies and tourism experts, I found a small niche. I was able to make my first sale using a paper, pencil and a relationship.</p><blockquote>I was able to make my first sale using a paper, pencil and a single relationship.</blockquote><p>At this point, you’ve essentially spent the past week doing practical research on your idea. If you were able to gain traction within a week following these methods, its time to build your website and/or product.</p><h3>Conclusion</h3><p><em>Ideas are worthless, execution is everything.</em></p><p>Choose an idea you’ve been thinking about recently and commit 7 days to it. Tell your friends and family, post on <a href="http://www.indiehackers.com/">Indiehackers.com</a> and commit yourself to turning that idea into a real product. Accountability is key when you’re beginning to develop your product.</p><p>Begin discussing your idea with potential customers then use that initial feedback to inform your paper prototype. Once you’ve developed your pitch using your paper prototype, it’s time to sell your product.</p><p>From here on, all you need is tenacity and grit. Good luck!</p><p>______________</p><p>Thanks for reading this article!</p><p>I’ve recently received funding to pursue an AI (artificial intelligence) based product. I’ll be live blogging about my experience from a global AI incubator (TBD).</p><p>Follow me to receive updates on our journey and… <em>learn from our mistakes</em>.</p><figure><a href="https://medium.com/swlh"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YqDjlKFwScoQYQ62DWEdig.png" /></a></figure><h4>This story is published in <a href="https://medium.com/swlh">The Startup</a>, Medium’s largest entrepreneurship publication followed by +414,678 people.</h4><h4>Subscribe to receive <a href="http://growthsupply.com/the-startup-newsletter/">our top stories here</a>.</h4><figure><a href="https://medium.com/swlh"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ouK9XR4xuNWtCes-TIUNAw.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a06232e3eb42" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/grow-your-idea-to-a-startup-in-7-days-make-a-sale-with-a-paper-prototype-a06232e3eb42">Grow your idea to a startup in 7 days: Make a sale with a paper prototype!</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to transition your career into Data Science — even if you don’t code today.]]></title>
            <link>https://medium.com/free-code-camp/5-tips-on-transitioning-your-career-into-data-science-even-if-you-dont-code-today-3440a563ac9d?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3440a563ac9d</guid>
            <category><![CDATA[personal-development]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Sat, 19 Jan 2019 17:41:53 GMT</pubDate>
            <atom:updated>2019-01-23T16:56:54.663Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*K8w_4MIzBlw74k_P" /></figure><p><em>(Don’t have time to read it all? I’ve summarized the entire article in the conclusion section!)</em></p><p>During my<strong> </strong>PhD in Organizational Psychology I always found myself reading about Data Science — thinking, it’d be really cool to be a data scientist... But, at the time, I couldn’t write a lick of code, so I chalked it up as a dream I’d achieve in another lifetime.</p><p>My worries about learning data science came in two forms: 1) Data Scientists code, I don’t, I’ll never become one, and 2) I’m doing a PhD in an unrelated topic, why throw it away?</p><p>It was around this time I was perusing LinkedIn and noticed an update, my friend who had been working as a <em>recruiter</em> for <strong>5 years</strong> took a position at <strong>Google</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/0*5438QZMRrQS9V3U9" /></figure><p>Curious to learn the secrets of scoring a job as a software developer at Google without having a related degree from Stanford or MIT, I sat down with him for a coffee.</p><p><strong><em>I was excited.</em></strong></p><p>Maybe this is where I could learn the secrets to a career transition into Data Science?!</p><p>I brought along my favorite pen and notebook.</p><p>I expected to hear the equivalent of the <strong>“5 easy ways to become a Data Scientist at Google”</strong> and have a <strong>come to Jesus moment.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*tg_u-smnZfFJLfq3" /></figure><p><strong>I heard no such thing.</strong></p><p><em>His secret?</em></p><p>He engaged with communities at <strong>freeCodeCamp</strong> and started with basic tutorials on how to code. He learned the <em>basics</em> of coding and then moved to <em>building small projects. </em>He further <em>immersed himself in the community </em>and began to <em>build bigger and badder products</em>. Two years later, and after committing a few hours a week, he was able to <strong>land his dream job at Google.</strong></p><p>I digress.</p><p>Instead of feeling defeated because there was no secret to transitioning my career, <em>I felt inspired.</em></p><p><strong>So I made a contract to myself.</strong></p><p>I’d commit 5 hours of my time a week to learn to code, to read about data science and to shut down the self-doubt I had about doing all this.</p><h4>With all that being said, here are the 5 tips I found most helpful along my journey to becoming a Data Scientist.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*FThPutNN0519F6Bx" /></figure><h3><strong>Tip 1: Learn to speak and think like a Data Scientist</strong></h3><p>The most important first step is to speak and think like a Data Scientist. What does that mean? First, learn how data scientists speak. What terms do they throw around frequently (e.g., scikitlearn, matrix-factorization, eigenvectors)? Don’t be afraid, just take notes on the words you don’t understand. Why? Learning the vocabulary is the first step in learning and communicating data science.</p><blockquote>Learning data science vocabulary will optimize your learning rate (..get the pun..? ha..ha?)</blockquote><p>By knowing the vocabulary, you’ll be able to utilize Google to its fullest extent. For example, if you’re learning about <em>Principal Component Analysis (PCA)</em>and you become frustrated because all the articles you’re reading are too technical, knowing that <em>PCA</em> is a <em>dimension reduction technique</em> and running a new Google search will net you <strong>entirely new results on a Google search.</strong></p><p>Often times, your ability to learn is limited by what you don’t know (…confusing eh?).</p><p><strong>Finding an article that is one level lower and less abstract is key to resolving the gaps in your knowledge.</strong></p><p>Knowing multiple terms for the same technique/idea is key to developing a broad understanding of the topic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/0*W8dc0bRSw8D_yMN2" /></figure><p><strong>Here’s a list of resources I used to learn how to speak like a Data Scientist:</strong></p><ul><li><a href="https://www.reddit.com/r/dataisbeautiful">https://www.reddit.com/r/dataisbeautiful</a>: This was my most inspirational resource to becoming a Data Scientist. Redditors post cool apps they’ve built and talk about how they did it. Sometimes, they post their Github!</li><li><a href="https://www.kaggle.com/c/ga-customer-revenue-prediction/kernels">https://www.kaggle.com/c/ga-customer-revenue-prediction/kernels</a>: If you’ve spent anytime reading about Data Science, then you’ve most likely heard about Kaggle. The most important part of Kaggle to an aspiring Data Scientist is the “Kernals” section. Here, fellow Kaggler’s post their solutions to the problems posed by the competition. Spend at least an hour of your time, TYPING and CODING out their solution — practice typing each line, line-by-line in your own Jupyter Notebook. Run the code and see what happens (e.g., you’ll run into errors because you won’t have certain libraries or dependencies installed, which brings me to my third bullet).</li><li><a href="https://stackoverflow.com/">https://stackoverflow.com/</a>: Anytime you have a question about Data Science related questions, head to Stackoverflow and run a search. Learn how people ask questions about Data Science related questions. At this level, most likely all your questions have answers, so running a search should be sufficient.</li></ul><h3>Tip 2: Engage in Kaggle Challenges</h3><p>I eluded to this a bit earlier but, learning by doing is ultimately the best way to learn. Spend time looking at the kernals in Kaggle competitions to learn from how other Kaggler’s approached the competition. At first, this will be <em>extremely daunting</em>, you <em>won’t understand 95% of the code you’re reading</em>, let alone, <em>you probably won’t be able to run the code on your own computer even after you’ve cloned it.</em></p><p><strong>This is where you need to be persistent.</strong></p><p>You aren’t going to learn anything if you get frustrated, so ease yourself into engaging with these challenges and soon enough you’ll be able to understand the kernals you read.</p><p><strong>Remember, when setting goals, be realistic about them (e.g., SMART goals): </strong>Specific, Measurable, Attainable, Realistic, Time-Bound (SMART).</p><p>In other words, don’t think you’ll be reading Kaggle kernals within a week. Give yourself a<strong> specific, realistic and time-bound goal —</strong></p><blockquote>“I’ll be able to understand how train/test/split works by the end of this week”.</blockquote><p>Set small goals, write them down and check them off when you achieve them. When you feel frustrated, go back to these checkmarks and see how far you’ve come since yesterday.</p><h3>Tip 3: Find your own Data Science Project</h3><p>Find a project you’re passionate about, whether it be a problem you’d like to solve or a library you’d like to learn — turn this into a project that you’ll put onto your github as a portfolio piece.</p><p>Finding a problem is best done through conversations. Engage with your community, your friends or… Even strangers. Find out what bothers them, or talk to them about ideas you’ve always had.</p><p>Hash out your idea, make it simple. Your project isn’t going to change the world. The most important part here is to start on one.</p><p>Once you find your idea you’d like to build, tell a friend or make an open commitment to your community (e.g., Medium blog) that you’ll be building it. Most importantly, highlight the features of your app and the time it’ll take for you to have it done (e.g., 1 month to build this app out).</p><p>I relied on my commitment to my peer group to build this <a href="http://www.righstrain.co/">cannabis recommendation app</a> that generates revenue today.</p><p>Read about how I built it on Medium here: <a href="https://towardsdatascience.com/how-i-built-a-cannabis-recommendation-app-using-topic-models-and-latent-dirchlet-allocation-lda-999598987ded">How I built it</a></p><p>If you’re looking for a project, I come across companies looking for pro-bono work all the time. Connect with me on <a href="https://www.linkedin.com/in/chowsam/">LinkedIn</a> and I’ll find the best project for your goals!</p><h3><strong>Tip 4: Apply for Data Science Jobs</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/276/0*Efb1ktGt4KHH-1kO" /></figure><p>Wait… wait... what?!</p><p>You’re probably thinking that you just started learning Data Science and now I’m telling you to apply to jobs. What the…?!</p><p>If you’re looking to score a Data Science job, you need to learn how to interview as a Data Scientist.</p><p>Yes… It’s a skill. The recruitment and selection of Data Scientists today isn’t great, and <strong>you’ll have to learn how get good at interviewing</strong> before you can land a job, so begin ASAP.</p><p>IMO, you’re ready to interview when you can speak like a data scientist.</p><p>Don’t worry about live coding assessments or Data Science related questions just yet, you’re 99% likely to fail these interviews at this point</p><blockquote>You want to fail.</blockquote><blockquote>You want to take notes on the questions you get.</blockquote><blockquote>You want to learn how interviews are conducted.</blockquote><p>Your goal here is to get good at interviewing, because you’re still far out from landing a job given your competencies in Data Science.</p><p><strong>Before you comment about this tip, hear me out!</strong></p><p>You should be speaking like a data scientist within 6 months of spending 5 hours a week. You’re still probably 1.5 years away from landing a job as a data scientist. Applying to jobs now gives you an understanding of how the entire process works. You’ll learn what you need to learn to interview well later on. Further, you aren’t burning any bridges.</p><p>Re-applying to a job where you failed an interview is often flattering to an interviewer especially if you come back and are 100x better than you were the year before.</p><p>For instance, during one of my first interviews for a data science position, I was asked <strong>“What is a negative R-Squared?”</strong>. I thought… Wtf? R-Squared can’t be negative. So I proceeded to ramble to the interviewer about what I thought it was.</p><p><strong>Fast forward 2 years later, I received the same question from another interviewer.</strong></p><p>Because I had looked up the answer 2 years back, I was able to <strong>smirk and provide an answer.</strong></p><p>The interviewer later told me that he’d never heard an answer that was so succinctly and clearly communicated.</p><p>Having your own project will motivate you to complete it as it can sit in your portfolio and show potential companies your competencies.</p><h3>Tip 5: Network…Network… And Network</h3><p>Your chances of scoring a job as a data scientist improves exponentially based on your network size.</p><p>In-person networking is the best way to expand your network in a meaningful way, however, it’s not always possible to make it out to networking events.</p><p><strong>The second best scenario is LinkedIn.</strong></p><p>Creating meaningful connections on LinkedIn is as simple as finding people in your industry, sending them a message and keeping up to date with their happenings.</p><blockquote>The crucial piece people miss about LinkedIn opportunities is that they don’t let others know they are open to opportunities.</blockquote><blockquote><strong><em>I found great success by adding: “Open to new opportunities” on my LinkedIn title.</em></strong></blockquote><p>Further, Medium supports a great network of Data Scientists that, I’m sure would be happy to connect. Read a cool article? Find the author on LinkedIn and chat with them about prospective opportunities!</p><p>I work as a Data Science consultant and come across many opportunities (mostly in Toronto). Let me know who you are on <a href="https://www.linkedin.com/in/chowsam/">LinkedIn</a> and I’d be happy to connect you to a company that is interested!</p><p>Find me here: <a href="https://www.linkedin.com/in/chowsam/">LinkedIn</a></p><h3>Conclusion</h3><p>A career transition is never easy, especially if you’ve just begun your journey. During my transition, I kept this quote close to my heart:</p><blockquote>“The best time to start was yesterday, the next best time is <strong>NOW.</strong>”</blockquote><p>The fact that you’ve read this entire article and are engaging with this sentence today, should show yourself you’re ready to start your transition.</p><p>I’m always happy to help a fellow transitioning data scientist, so let me know how I can!</p><p><strong>To summarize the entire article in a few bullet points:</strong></p><ol><li><strong>Learn to Speak and Think like a data scientist</strong></li><li><strong>Engage in Kaggle challenges</strong></li><li><strong>Find your own Data Science project</strong></li><li><strong>Apply for Data Science roles… Interview, fail… Rinse &amp; repeat</strong></li><li><strong>Network</strong></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3440a563ac9d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/5-tips-on-transitioning-your-career-into-data-science-even-if-you-dont-code-today-3440a563ac9d">How to transition your career into Data Science — even if you don’t code today.</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I built a Cannabis Recommendation app using Topic Models and Latent Dirchlet Allocation (LDA)]]></title>
            <link>https://medium.com/data-science/how-i-built-a-cannabis-recommendation-app-using-topic-models-and-latent-dirchlet-allocation-lda-999598987ded?source=rss-d1815928ead0------2</link>
            <guid isPermaLink="false">https://medium.com/p/999598987ded</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Sam Chow, PhD]]></dc:creator>
            <pubDate>Thu, 17 Jan 2019 21:11:27 GMT</pubDate>
            <atom:updated>2019-01-18T01:24:07.236Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kFu9-1P2scVakLAPMQB_vQ.jpeg" /><figcaption>How I built www.<a href="http://www.rightstrain.co/">rightstrain</a>.co, a cannabis recommendation tool that used by online dispensaries</figcaption></figure><h3><strong>How I built a Cannabis Recommendation app using Topic Models and Latent Dirchlet Allocation (LDA) ~Non-Technical ~</strong></h3><h3><strong>Background:</strong></h3><p>On October 17th, 2018, cannabis became <em>legal</em> in Canada.</p><p>As an entrepreneur, I’m always reading about the latest tech startups, following how markets are developing and sniffing out emerging opportunities. As a data scientist, I’m always looking for data driven solutions to problems I’ve identified.</p><p>As a resident of Toronto, naturally, I began to look at the <em>cannabis market</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qrUmzTwRgvm886gErp9WVg.png" /></figure><p>The following article will be a brief summary of how I built a cannabis recommendation system. I’ll keep it practically technical so those that would like to build a similar app can follow the code — while those who are just interested will still find it enjoyable to read.</p><h3><strong>The Problem:</strong></h3><p>In 2013, talks began about legalization of cannabis in Canada. It was at this time I knew legalization was going to happen, so I became an early stage investor and kept up with the market.</p><p>Fast forward to legalization in 2018, leveraging my network in the industry, I got on the phone with a number of cannabis dispensaries and learned that they all had <em>one problem</em> in <em>common</em>.</p><p>When customers made orders online, and found that their favorite item (e.g., a certain cannabis strain) was sold-out, they would stop shopping and find the same item at another retailer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6mMwFhww7OzX6MbgQH_u-g.jpeg" /></figure><p><em>I learned that this was called</em><strong><em> shopping cart abandonment </em></strong><em>which, according to </em><a href="https://www.statista.com/"><em>Statista</em></a><em> costs e-commerce stores over 75% of their sales.</em></p><p>Doing further research, I found that <em>Amazon</em> <strong>attributes 35% of its revenue</strong> to recommender systems.</p><p>So I built a recommender system for cannabis products.</p><h3>Building a recommender system 101:</h3><p>What is a recommender system?</p><p>Basically, for e-commerce purposes, a recommender system simply finds products that are similar to products the shopper is purchasing and recommends similar items in hopes of increasing sales.</p><p>So, how does it do this?</p><p>Imagine you own an e-commerce shop on Amazon that sells home accessories. You notice that when customers purchase, say, toilet paper (A), they also always purchase towels (B). Further, you find that this pattern repeats itself across different customer segments, with some variation.</p><p>This variation occurs when other customers purchase product A, B and, a new product, C.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xAWZ1VqLpNBpb6SXesUeKw.png" /></figure><p>Knowing this pattern, we can now recommend product C back to customers that traditionally only purchase product’s A and B.</p><p>That’s it!</p><h3>How to build a recommender system:</h3><p>First, you need to start with data.</p><p>In my case, I wanted to build a recommender system for cannabis products, so I needed data on all various cannabis strains on the market. Doing a quick search, I found a number of databases that contain reviews on cannabis strains.</p><p>In order to get at the data, I wrote a simple Python Scraper and tunneled it under the Tor network. This allows the scraper to run continuously and reduces the chances your IP will be banned. Please see my write-up on developing a python scrape tool (TBA).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*rTgNo0SwG3qzbuqtxh1-7w.jpeg" /></figure><p><strong>IMPORTANT NOTE: </strong><em>when scraping data from the internet, please respect the server you’re scraping from, as you can easily overwhelm a server that isn’t prepared to handle the amount of requests you throw at it.</em></p><h3>Exploratory Data Analysis:</h3><p>I managed to scrape 500k reviews that contained both a 1–5 star rating and a comment. I figured I could create a recommender system by looking at the patterns of ratings users gave to each strain (see previous section for the logic behind this).</p><p>However, looking at the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/0*V9tKXN_tP6U-QcrM" /><figcaption>Figure 1: Ratings of cannabis strains (x-axis) vs number of reviews (y-axis)</figcaption></figure><p>I found a severe range restriction. In other words, the majority of the reviews were in the range of 4–5 stars.</p><p><em>I guess cannabis enthusiasts are easily impressed :)</em></p><p>Given this finding, I decided against using the quantitative ratings because the recommender system would basically be recommending other strains with high ratings — which virtually all strains were rated highly.</p><p>Which would defeat the purpose of a recommender system.</p><h3>Enter Natural Language Processing:</h3><p><em>Natural Language Processing (NLP)</em> is a sub-field of <em>Artificial Intelligence</em> that focuses on enabling computers to understand and process human languages.</p><p><em>So how does a computer read human languages?</em></p><p>Let’s start with understanding how computers… <em>compute</em>. Basically, computers think in terms of binary, 1s and 0s. This makes things complicated because words aren’t numbers.</p><p><em>So how does NLP work?</em></p><p>In its simplest form, NLP works by transforming words into numbers! For instance, if we had a document with three words in it, “dog eats food”, each word would be converted into a vector (e.g., think of it as a string of numbers). So the word “dog” may be represented by (1,0,1,1,1,0) and “eats” by (1,1,0,0,0,0)..etc. Now you can imagine that once all words are vectorized, a computer can now recognize that a vector (1,0,1,1,1,0) represents the word “dog”.</p><figure><img alt="Image result for natural language processing" src="https://cdn-images-1.medium.com/proxy/0*8TztMRVSwG2b-HwF.jpg" /></figure><p>The tricky thing now is getting the computer to understand <em>meaning</em>. This topic is out of the scope of the current article, but I may follow up with another article on this topic. Although the NLP can successfully vectorize words to allow computers to recognize words, it is extremely difficult to get computers to understand the meanings of words. If you’re interested in this topic, read more into <a href="https://medium.com/dair-ai/deep-learning-for-nlp-an-overview-of-recent-trends-d0d8f40a776d">Deep Learning and NLP</a>.</p><h3>Topic Modelling &amp; Latent Dirchelet Allocation:</h3><p>Now that all our text data has been vectorized, we begin looking for patterns in the data.</p><p>Topic modelling is perfect for this type of task, it’s a statistical modelling technique used to discover the abstract “topics” that occur in a collection of documents. A very simple explanation of this is that it combs through a document and recognizes: 1) the most frequently appearing words and 2) words that appear next to those frequently appearing words. The logic here is that if these words always appear together, they must form some sort of topic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/0*fshGRRXtc5JLuW4e.png" /></figure><p>Now you’re probably wondering, how many topics does the algorithm create?</p><p><em>Well, that’s up to you.</em></p><p>The art of topic modelling comes into play when you choose how many topics you want to keep in your model.</p><p>I generally look at two things: 1) the Coherence Value and 2) The Intruder test. Let’s elaborate a bit more on the two.</p><p>The Coherence Value can be thought of as the probability of each topic being a “good” topic. To read more about it, check out this great article on <a href="http://qpleple.com/topic-coherence-to-evaluate-topic-models/">Coherence Values.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*tWQ01JfLGlX5aJofPjt77A.png" /></figure><p>In order to choose the best fitting model, you need to qualitatively evaluate each topic using the <em>Intruder Method</em>.</p><p>Above, I’ve plotted the number of topics and their corresponding coherence values. Notice a large drop after 14 topics. The optimal model here wold be 14 topics according to coherence values, however, an 8 topic model only reduces the CV by 1 point. For sake of parsimony and explanatory power, I always choose to stick with the simpler model.</p><p><em>So what’s the Intruder Method?</em></p><p>The Intruder test is a great follow-up test to using Coherence Values. Once you determine how many topics you want, you then look at the topics individually and assess them qualitatively. In other words, you want to be asking.. <em>“What words don’t belong in these topics?”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*hKC9FWXmiQhW7ez2-SzCKg.png" /></figure><p>Let’s look at Topic 1 (or 0 in this case). It’s a bit confusing because we see words like “<em>great</em>” and “<em>favorite</em>”, but also words that appear opposite in meaning like “<em>stress”</em> and “<em>depression”</em>. This is an example of a topic that isn’t very interpretable by humans but scores high on a Latent Dirchelet Allocation (LDA). Meaning that I’ll have to fine tune the hyper parameters of the model to get a better output.</p><p>Let’s look at topic 2 (model 1), this topic is a bit clearer on what it’s getting at. It looks like it could refer to a topic like “likeability”.</p><p>You continuously do this for the number of topics in your model, evaluating them individually, looking for words that may or may not fit in a topic.</p><h3>Creating the Recommender System:</h3><p>Although It may not be clear from above, my final model produced 8 topics with some very interesting insights. For instance, I found that cannabis consumers enjoy smoking because of a few things: 1) some people smoke because they enjoy the <em>flavors and aromas</em> of cannabis, 2) while others do so because it makes them feel <em>creative</em>, 3) another segment of users do so because it makes them feel <em>energized</em> and finally 4) the majority of users do so because it helps with <em>pain-relief</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*BQJm0FJRPpnXqiC5" /></figure><p>That’s so cool!</p><p><strong><em>Essentially, what the topic models did was separate my data into customer segments. If I was in the business of marketing and/or writing copy, I’d be better able to target customer segments with this information.</em></strong></p><p><em>Anyways, back to the data science.</em></p><p>Using these 8 topics, I predicted how much of each strain’s review contained those topics. Doing so gave me 8 features to separate my strains on. In other words, I created a dataset based on how each strain (e.g., I was working with over 200 strains) differed on the topics I found (e.g., some strains reportedly provided more creative thinking abilities, while others increased energy).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ShkFCWmq09jIQxQs" /></figure><p>Once this was complete, I was ready to create the recommender system based on similarities.</p><h3>Choosing a similarity metric:</h3><p>Now that we have our strains described by 8 different features (e.g., topics) it’s time to choose how we recommend them. There are a number of similarity metrics that can be used (e.g., Cosine Similarity, Euclidean Distance, Manhattan distance).</p><p>The important consideration when choosing distance metrics is: 1) how does it deal in high dimensional space? (e.g., euclidean distances begin to fail in high dimensions) and 2) how accurate is the recommendation?</p><p>This brings us to the question, how do we validate a recommender system?</p><p>There are a number of ways to do so. I think the best bang for your buck is to back-test your data. For example, say our dataset contains data of customer A who purchased product A,B,C &amp; D.</p><p>One method of validation would be to use data from Customer A and predict what they would’ve purchased next after product A. Then validate it against what they actually purchased!</p><h3>Conclusion</h3><p>I used topic modelling and an LDA approach to find customer segments in the emerging cannabis market. From this, I created a recommendation system. The most difficult part of this project was procuring and cleaning the data — something that is common in all data science projects.</p><p>If I get enough interest in the article, I’ll write a technical post where I can share my code of my MVP. The product has since evolved and is currently being used by a number of dispensaries! Check it out here: <a href="http://www.rightstrain.co/">www.rightstrain.co</a></p><p>If you would like to see a technical post, let me know in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=999598987ded" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-i-built-a-cannabis-recommendation-app-using-topic-models-and-latent-dirchlet-allocation-lda-999598987ded">How I built a Cannabis Recommendation app using Topic Models and Latent Dirchlet Allocation (LDA)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>