<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kai Xin Thia on Medium]]></title>
        <description><![CDATA[Stories by Kai Xin Thia on Medium]]></description>
        <link>https://medium.com/@thiakx?source=rss-38556f69915d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OEVQUyN3djWiv7CcG90RIg.jpeg</url>
            <title>Stories by Kai Xin Thia on Medium</title>
            <link>https://medium.com/@thiakx?source=rss-38556f69915d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:46:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thiakx/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques]]></title>
            <link>https://medium.com/data-science/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/bae9f3e4c27f</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Fri, 10 Apr 2020 15:48:33 GMT</pubDate>
            <atom:updated>2020-04-13T15:34:41.484Z</atom:updated>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>The future of visual RecSys is an exciting one. Let us explore some of the most cutting edge techniques and ideas that we can incorporate into our recommenders.</h4><h3>Style2Vec (2017) — Combining Multiple Convolutional Neural Networks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7mGagmth2IieSLhB_Dc4aA.png" /><figcaption>Lee, H., Seol, J., &amp; Lee, S. (2017). Style2Vec: Representation Learning for Fashion Items from Style Sets. Open Access From <a href="https://arxiv.org/abs/1708.04014">https://arxiv.org/abs/1708.04014</a></figcaption></figure><blockquote>The authors “propose Style2Vec, a vector representation model for fashion items. Based on the intuition of distributional semantics used in word embeddings, Style2Vec learns the representation of a fashion item using other items in matching outfits as context. Two different convolutional neural networks are trained to maximize the probability of item co-occurrences” (Lee et al., 2017).</blockquote><p>The paper is short, but the idea is fascinating. The use of two CNN models, where one generates embeddings <strong>for the target image</strong> while the other generates embeddings for <strong>similar context items</strong>, is a novel application of the techniques derived from <a href="https://towardsdatascience.com/introduction-to-word-embedding-and-word2vec-652d0c2060fa">Word2Vec</a> to images. If we take a look at the image above, for example (a) we have<strong> pink skirt</strong> — <strong>black skirt +</strong> <strong>black jacket</strong> = <strong>pink jacket</strong>. It is important to note that we are adding/subtracting <strong>traits</strong> that are not usually expressed as a product attribute, such as the holes in example (c). Thus, Style2Vec allows us to combine/select styles in a different manner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vrs6KeYQSDSzBpTkVaRinQ.png" /><figcaption><a href="https://github.com/DavidNepozitek/Style2Vec">Style2Vec</a> implemented by <a href="https://github.com/DavidNepozitek">David Nepožitek</a>. Open source on <a href="https://github.com/DavidNepozitek/Style2Vec">Github</a>.</figcaption></figure><p><a href="http://david.nepozitek.cz/">David Nepožitek</a> implemented the ideas proposed in the Style2Vec paper and open source the code on Github. The <a href="https://github.com/DavidNepozitek/Style2Vec/blob/master/docs/Report.md">results from his report are impressive</a>, as you can see from the image above, he manages to group items similar to each other in terms of colors, patterns, and shapes. We can leverage results like this to generate whole outfits and collections based on users’ preferences.</p><h3>Generative Image Models (2017) with GAN</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kKFWrsbdliI91Xeb5oYLiA.png" /><figcaption>Kang, W.-C., Fang, C., Wang, Z., &amp; McAuley, J. (2017). Visually-Aware Fashion Recommendation and Design with Generative Image Models. Open Access From <a href="https://arxiv.org/abs/1711.02231">https://arxiv.org/abs/1711.02231</a> | <a href="https://github.com/kang205/DVBPR">Code</a></figcaption></figure><blockquote>The authors used <strong>Generative Adversarial Network (GAN)</strong>, “an unsupervised learning framework in which two components ‘compete’ to generate realistic looking outputs... One component (a generator) is trained to generate images, while another (a discriminator) is trained to distinguish real versus generated images. Thus the generated images are trained to look ‘realistic’ in the sense that they are indistinguishable from those in the dataset” (Kang et al., 2017).</blockquote><p>GAN allows us to generate <strong>the ideal product</strong> for our users, even if it does not exist in the product catalog. The image above is the output. We have the real images on the left and generated images on the right, and we can see that the images on the right generally have a higher preference score. There are two use cases for generating the ideal product.</p><p>Firstly, <strong>GAN helps us build an awareness of the users’ preferences</strong> and represents that awareness concretely with an image. We can leverage this image with techniques like CNN + nearest neighbors to<strong> select items that are visually most similar to the ideal state.</strong></p><p>Furthermore, <strong>GAN helps in product procurement and design</strong>. If we know that green colored shirts with purple strips are a growing trend among the ideal products, we can bring in/create such products to fulfill the market need, before customers voice them out to us. Even if green colored shirts with purple strips are not exactly what the customer wants now, GAN allows us to be <strong>proactive instead of reactive</strong> to market demands and changes.</p><h3>Explainable Outfit Recommendation with Comment Generation (2019)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w-by_vxNFpfapY7GRUhUyA.png" /><figcaption>Lin, Y., Ren, P., Chen, Z., Ren, Z., Ma, J., &amp; de Rijke, M. (2019). Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation. Open Access From <a href="https://arxiv.org/abs/1806.08977v3">https://arxiv.org/abs/1806.08977v3</a> | <a href="https://bitbucket.org/Jay_Ren/fashion_recommendation_tkde2018_code_dataset/src/master/">Code</a></figcaption></figure><blockquote>The authors “propose a novel neural network framework,neural outfit recommendation(NOR), that simultaneously provides outfit recommendations and generates abstractive comments. NOR consists of two parts: outfit matching and comment generation. For outfit matching, we propose a convolutional neural network with a mutual attention mechanism to extract visual features…For abstractive comment generation, we propose a gated recurrent neural network with a cross-modality attention mechanism to transform visual features into a concise sentence”.</blockquote><p><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">RecSys based on CNN is powerful</a>, but it can be hard to interpret the output. There have been separate attempts to <a href="https://github.com/utkuozbulak/pytorch-cnn-visualizations">visualize the CNN gradients by Utku Ozbulak,</a> <a href="https://github.com/ekinakyurek/Long-Term-Recurrent-Convolutional-NN">generate image comments by Donahue, J. et al</a>. Still, it is not easy to combine both techniques and apply it to the context of RecSys. If we look at the resultant image above, we see that it is the first step in understanding the recommendations, with <strong>“great denim look”, “love the red and white”</strong> being good examples of explaining why the outfits are recommended. However, in the negative cases in the last row, we can see that the <strong>comment generation itself is not perfect</strong>; sometimes it is describing something not found within the image or is completely out of context. Note that the authors <strong>build the comment generating model using data from Polyvore, </strong>a community-powered social commerce website.</p><p><strong>Nonetheless, explainable AI (XAI) is a critical piece in understanding, evaluating, and deploying deep learning solutions in production. </strong>For more on XAI, <a href="https://github.com/feifeife">Feifeife</a> has gathered <a href="https://github.com/feifeife/All-about-XAI">an impressive collection of XAI materials</a>.</p><h3>Private Personalized RecSys (2020)</h3><p>Increasingly, RecSys are being deployed in privacy-sensitive domains like <a href="https://www.mdpi.com/2079-3197/7/2/25/htm">healthcare</a>, <a href="https://www.sciencedirect.com/science/article/pii/S1877050910003194">education</a>, and <a href="https://www.sciencedirect.com/science/article/pii/S0167923615001153">finance</a>. We want the benefits of personalized healthcare/education/financial plans, but at the same time, the fear of giving up our data and then losing them to a hack is real. It seems oxymoronic — <strong>how can we build a personalized RecSys while maintaining user privacy?</strong></p><p>Back in 2009, McSherry &amp; Mironov from Microsoft Research explored this issue in their paper <a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2009/06/NetflixPrivacy.pdf">Differentially Private Recommender Systems</a> with a simple idea. In essence,<strong> we can add noise to the item ratings and the item-item covariance matrix in line with </strong><a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2009/06/NetflixPrivacy.pdf"><strong>ε-Differential Privacy</strong></a><strong> </strong>(the actual mathematics behind this is non-trivial). In other words:</p><ul><li>We mask away the <strong>identifying traits</strong> of any particular user (user A buys pink shirts on the first Monday of every month).</li><li>To obtain <strong>general trends</strong> (segment X of users likes to buy pink shirts)</li><li><strong>The privacy loss is mathematically proven to be bounded by a factor of ε.</strong></li><li>While differential privacy is a <strong>useful metric to measure risk internally when designing a RecSys, it is not intuitive to explain to users, nor does it guarantees that data is secured.</strong></li></ul><p>A new paper by Ribero et al. (2020) extends the idea with <a href="https://arxiv.org/pdf/2003.00602.pdf">Federating Recommendations Using Differentially Private Prototypes</a>. Federating learning is a modern approach to distributed machine learning.</p><ul><li>Instead of training massive models on centralized servers, <strong>we send out small (megabyte sized) models to users’ devices.</strong></li><li>The models are <strong>trained on the user’s device with their data</strong> during the device idle time.</li><li>We <strong>only send</strong> <strong>training results back </strong>to a centralized server.</li></ul><p>You can see <a href="https://federated.withgoogle.com/">an illustration of this process illustrated by Google’s comic strip</a>. By combining differential privacy and federated learning, Ribero et al. propose a novel approach to tackling the issue of private, personalized RecSys.</p><blockquote>“Most federated learning methods require <strong>multiple rounds</strong> of communication between entities and a central server, which poses a problem for <strong>differential privacy requirements</strong>. Specifically, we can think of each round of communication from the entities to the server as a query sent to the individual entities, which has<strong> potential to leak information…</strong>(hence) we constrain the communication to <strong>only two rounds</strong>, back and forth” (Ribero et al., 2020).</blockquote><p>Of course, the problem is a challenging one. To cut down the number of rounds to only two, the team needs to come up with a novel way to compress the data and then save it in an accessible form. They name these data structures as “prototypes”:</p><blockquote>These prototypes are designed to: a) contain similar information as Xh, thus allowing construction of an accurate item representation; b) be of low dimension relative to Xh, hence minimizing communication load; and c) maintain differential privacy with respect to the individual users.</blockquote><p>The paper is a challenging read on a rapidly evolving and important topic. If you are interested to learn more about federated learning and differential policy, <a href="https://github.com/lee-man">lee-man</a> is collecting <strong>a list of readings, tools, and code on their Github post.</strong></p><h3>Further Readings</h3><p>Visual RecSys is an exciting field, and I hope you enjoyed the various techniques we discussed today. For more cutting edge stuff on RecSys, you can explore</p><ul><li>The collection on <a href="https://recsys.acm.org/">ACM RecSys</a> and their <a href="https://www.youtube.com/channel/UC2nEn-yNA1BtdDNWziphPGA/videos">YouTube channel</a>, they are the biggest event in the world of RecSys.</li><li>Spotify’s engineering blog is impressive. <a href="https://labs.spotify.com/2020/01/16/for-your-ears-only-personalizing-spotify-home-with-machine-learning/">Check out their recent post on personalization.</a></li><li><a href="https://netflixtechblog.com/">Netflix’s tech blog</a> is, of course, another excellent read, covering many of the real-world deployment and scalability challenges.</li><li>Zalando is an eCommerce firm with a solid research team. <a href="https://research.zalando.com/welcome/mission/publications/">Their publications are worth a read</a>.</li><li>Recombee provides recommendation-as-a-service and <a href="https://medium.com/recombee-blog">has a pretty good Medium blog</a> covering the technologies employed in their company.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bae9f3e4c27f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Temporal Fashion Recommender]]></title>
            <link>https://medium.com/data-science/temporal-fashion-recommender-59c26313fa25?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/59c26313fa25</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Sun, 05 Apr 2020 15:34:04 GMT</pubDate>
            <atom:updated>2020-04-06T01:33:06.888Z</atom:updated>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>Building a Recommender That Evolves with Seasons</h4><p>One of the biggest challenges in the design of any recommender system (RecSys) is in handling temporal shifts. Since the world of fashion evolves with time, recommenders we design must adjust to the changing tides as well.</p><p>In this article, we will consider how we can <strong>factor in the seasonal outfit subtleties into our Convolutional Neural Network (CNN) model using temporal weights.</strong></p><h3>The Business Problem</h3><p>Our imaginary eCommerce company, <strong>HappyPanda Co., </strong>is expanding globally and will like to launch <strong>Seasonal Collections</strong>. The product requirements are:</p><ul><li>Given collections of outfits, how can we adjust them to suit the various seasons across different countries? Note that each country will have different fashion sense and weather conditions for different seasons.</li></ul><h3>The Data</h3><ul><li>We will be using a subset of<strong> </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>DeepFashion</strong></a><strong> data open-sourced by Liu Z. et al., </strong>The Chinese University of Hong Kong. Our data consists of <strong>280K fashion images across 46 categories. You can </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>download the data from their website.</strong></a><strong> </strong>Furthermore, the team has released an <a href="https://github.com/switchablenorms/DeepFashion2">updated version with additional data</a>. You will need to <a href="https://docs.google.com/forms/d/e/1FAIpQLSeIoGaFfCQILrtIZPykkr8q_h9qQ5BoTYbjvf95aXbid0v2Bw/viewform?usp=sf_link">fill up a Google form</a> to gain access to the data.</li><li>The <strong>temperature data</strong> is from <a href="https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data"><strong>Kaggle — Climate Change: Earth Surface Temperature</strong></a> open-sourced by <a href="http://berkeleyearth.org/">Berkeley Earth</a>. The data consists of global temperatures across 3,400 cities from 1750 to 2013. <strong>We will only be using a subset of the data, the average temperature of Madrid and Melbourne</strong> <strong>across 10 years (2004–2013).</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WOUWtYGWaNJFp6eNjluIhg.jpeg" /><figcaption>Selected Spring, Summer, Autumn, Winter Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>From the deep fashion data, I select outfits <strong>suitable for each of the four seasons</strong>. For example, in winter, most people will don on a heavier jacket for warmth. On the other hand, spring and summer is the time to enjoy some time under the sun with lighter clothes. <strong>Temperature thus plays a significant role</strong> in influencing what people wear. For instance, if we consider the average temperature across the year in a particular city,<strong> Madrid (Spain)</strong>, we can place our selected outfits across time to observe the relationship between outfits and temperature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TH1TIka0DGK6mC4RmCwiMw.jpeg" /><figcaption>Madrid(Spain) temperature from <a href="https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data">Kaggle — Climate Change: Earth Surface Temperature</a> (<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC BY-NC-SA 4.0</a>) by <a href="http://berkeleyearth.org/">Berkeley Earth</a></figcaption></figure><blockquote>More importantly, the <strong>transition between the seasons takes place over weeks, </strong>and our model should be able to <strong>adjust the recommendations over time, helping our users transit into the next season gradually.</strong></blockquote><blockquote>Of course, in the real world, we will have a<strong>ccess to a richer set of time series data</strong>. For example, customers’ view/purchase trends over time, hot sellers, or fads like Korean Kpop Fashion. Weather data here is an illustration of the potential of incorporating time series data and not the definitive nor sole data solution for the problem.</blockquote><h3>Reviewing Results from <strong>Seasonal</strong> Collection</h3><p>Before we dive into the code, let us take a look at the input/output of the model to gain intuition on the problem and proposed solution.</p><p>Input:</p><ul><li>I select<strong> 24 outfits for </strong>each season. They will act as the seed images for Seasonal Collections.</li><li><strong>I will only pass images into the model; the model does not ingest additional attributes/descriptions/product details; </strong>this is because we want the model to automatically learn and detect the style of fashion images passed into it without further human/machine labeling.</li></ul><p>Output:</p><ul><li>The goal is for Seasonal Collections to <strong>blend in the influence of each season, weighted by how deep into the season we are.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*we4RLTq5v8mnc7SXnpyxAQ.jpeg" /><figcaption>Outfits selected by Seasonal<strong> </strong>Collections for Madrid (Spain). Left: 100% Winter. Right: 100% Spring. Middle: 50–50 blend of Winter and Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>Madrid’s weather <strong>transits in a gradual manner </strong>from Winter to Spring from January to May. Seasonal Collections can generate a new collection for the transition period by <strong>blending the features of winter </strong>(coat, long sleeves, full-length wear) <strong>with features from Spring </strong>(lighter materials, colorful wear, less layering) into a new “Spring is coming collection”. <strong>The ratio we use here is 50–50</strong>, so both Winter and Spring have equal weight in the final blend (we will explore this concept in the code section).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHVszVciWWBaIuEorFGvqw.jpeg" /><figcaption>Outfits selected by Seasonal<strong> </strong>Collections for Melbourne (Australia). Left: 100% Summer. Right: 100% Autumn. Middle: 25–75 blend of Summer and Autumn. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w95hLm4XeXSXFkuhI1AsMg.png" /></figure><p>In contrast to Madrid, <strong>Melbourne’s Summer transits to Autumn rapidly</strong>, with temperatures falling several °C every week. Thus, we need to <strong>aggressively introduce elements from the Autumn collection</strong> to match our customers’ requirements for warmth. <strong>Using a blending ratio of 25–75, we assign a higher weight to the Autumn collection</strong>, allowing our model to pull out essential attributes such as layering, long sleeves, and thicker materials to keep our customers warm.</p><blockquote>We can see the importance of <strong>localization; </strong>to tune the model for different cities across different periods.<strong> There is no one size fits all solution for RecSys, especially when we want to recommend across temporal data.</strong></blockquote><h3>The Code</h3><p><a href="https://colab.research.google.com/gist/thiakx/336f04cfc4d56d903ebc17cf67b599fa/modeling-temporal-fashion-recommender-across-season.ipynb"><strong>Link to code on Colab</strong></a><strong> </strong>(you just need a free Google Account to run the code on GPU in the cloud).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2a1c671fb36887cac2338946fbaae3f/href">https://medium.com/media/b2a1c671fb36887cac2338946fbaae3f/href</a></iframe><p>Seasonal Collections is built on<strong> Convolutional Neural Networks, with transfer learning from ResNet and approximate nearest neighbors</strong>. I will not dive into the individual theories as they have been well covered in many articles:</p><p><strong>Explanation of Overall Setup:</strong></p><ul><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/transfer-learning-with-convolutional-neural-networks-in-pytorch-dd09190245ce">Transfer Learning with Convolutional Neural Networks in PyTorch</a></li></ul><p><strong>Convolutional Neural Networks:</strong></p><ul><li><a href="https://youtu.be/YRhxdVk_sIs">Convolutional Neural Networks (CNNs) explained by deeplizard (Video)</a></li><li><a href="https://algobeans.com/2016/01/26/introduction-to-convolutional-neural-network/">Convolutional Neural Networks (CNN) Introduction</a></li><li><a href="https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/">A Beginner’s Guide To Understanding Convolutional Neural Networks</a></li></ul><p><strong>ResNet and Transfer Learning:</strong></p><ul><li><a href="https://shuzhanfan.github.io/2018/11/ResNet/">Understanding Deep Residual Networks</a></li><li><a href="https://towardsdatascience.com/an-overview-of-resnet-and-its-variants-5281e2f56035">An Overview of ResNet and its Variants</a></li><li><a href="https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a">A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning</a></li></ul><p><strong>Approximate Nearest Neighbors:</strong></p><ul><li><a href="https://erikbern.com/2015/10/01/nearest-neighbors-and-vector-models-part-2-how-to-search-in-high-dimensional-spaces.html">Erik: Annoy: Nearest neighbors and vector models — part 2 — algorithms and data structures</a></li></ul><p>Instead, I will focus on the core logic of Seasonal Collections:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8fb6e49b4c8b0f209395a69d54e2d410/href">https://medium.com/media/8fb6e49b4c8b0f209395a69d54e2d410/href</a></iframe><p>The command to get the list of outfits for the Melbourne example above is this line of code:</p><pre>get_similar_image(generate_image_mix(summer_ids, autumn_ids, number_of_items=24, ratio=0.25))</pre><ul><li>In essence, we are taking two collections (<em>summer_ids</em> and <em>autumn_ids</em>) and mixing them up based on the <em>ratio (0.25) </em>and selecting <em>24</em> items from the mix.</li><li>This means that <strong>6 outfits from summer and 18 outfits from autumn</strong> will be used as the seed to generate the new collection.</li><li><strong>The <em>ratio</em> is thus the critical hyperparameter that can be tuned based on the time-series pattern.</strong> If we want to increase the influence of a collection, we tune the ratio in order to <strong>include more outfits of that collection in the mix.</strong></li><li>Centroid embeddings will then <strong>merge all the seed images into one representation</strong> by averaging the values across all dimensions. Finally, approximate nearest neighbors is applied to return outfits closest to the representation.</li><li>We can place the get_similar_image() code fragment above in a loop and generate a mix of collections based on different ratios:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8S6fUFc7-L-IoIV58g_0bw.png" /><figcaption>Ratio 1.00 Winter vs. Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aZlmH0luyGPsGSCAzETvoQ.png" /><figcaption>Ratio 0.75 Winter vs. Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DHhQ1KgIdALMP5Tj7S7i6w.png" /><figcaption>Ratio 0.50 Winter vs. Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJsGlXdtO6DbY1tNtZON8A.png" /><figcaption>Ratio 0.25 Winter vs. Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PPbW_YKe3wfpTmYPZy-dYQ.png" /><figcaption>Ratio 0.00 Winter vs. Spring. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><h3>What have we learned</h3><p>We can intuitively incorporate seasonality and time-series data into RecSys by tweaking the ratio to blend the different collections. However, it is not easy to determine this ratio in the real world. We will need to conduct a lot of AB testing, and gather feedback from users, domain experts, product teams. Furthermore, we will have to localize the solution across different geographies and markets to cater for the difference in fashion trends and even the weather.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59c26313fa25" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Personalized Real-Time Fashion Collection Recommender]]></title>
            <link>https://medium.com/data-science/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/22dc90c150cb</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Sat, 28 Mar 2020 15:43:39 GMT</pubDate>
            <atom:updated>2020-04-01T04:25:45.807Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5na703b68Hm_Zdfz3A_Llg.jpeg" /><figcaption>Outfits selected by our RecSys. Left: “Anything with Shorts”. Right top: “Urban Lifestyle”. Bottom Right: “Flowery Dreams”. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><h4>Modern RecSys</h4><h4>We will make use of transfer learning, approximate nearest neighbors, and embeddings centroid detection in PyTorch to build our recommender.</h4><p>I have worked in the data industry for over seven years and had the privilege of designing, building, and deploying two recommender systems (RecSys) that went on to serve millions of customers. In this article, I will introduce modern approaches to visual recommender by walking through a case study with code and sharing some of my experience designing RecSys.</p><h3>The Business Problem</h3><p>Imagine you are hired by an eCommerce, <strong>HappyPanda Co.,</strong> to work on their Fashion Collection feature. The product team has outlined the requirements:</p><ul><li>The model should be able to <strong>scan across all 280,000+ product images</strong> and automatically <strong>generate a group of recommended products </strong>that are <strong>customized to what the user has viewed/bought.</strong></li><li>Speed is of the essence in an eCommerce firm. The group of recommended <strong>products should be generated within 2ms</strong> to meet the “real-time” requirements of the pipeline.</li><li>The model should be able to<strong> adapt to new products and new users</strong>.</li></ul><p>Sounds fun? Let’s build this.</p><h3>The Data</h3><p>We will be using a subset of<strong> </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>DeepFashion</strong></a><strong> data open-sourced by Liu Z. et al., </strong>The Chinese University of Hong Kong. Our data consists of <strong>280K fashion images across 46 categories. You can </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>download the data from their website.</strong></a></p><p>Furthermore, the team has released an <a href="https://github.com/switchablenorms/DeepFashion2">updated version with additional data</a>. You will need to <a href="https://docs.google.com/forms/d/e/1FAIpQLSeIoGaFfCQILrtIZPykkr8q_h9qQ5BoTYbjvf95aXbid0v2Bw/viewform?usp=sf_link">fill up a Google form</a> to gain access to the data.</p><h3>Fashion Collection — Steps to build it</h3><p>After understanding the business requirements, we break down the steps needed to construct Fashion Collection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9z1jN9Gbif3PxO_c9GH0Ug.jpeg" /><figcaption>My steps to build Fashion Collection</figcaption></figure><h3>Reviewing Results from Fashion Collection</h3><p>Before we dive into the code, let us take a look at the input/output of the model to gain intuition on the problem and proposed solution.</p><p>Input:</p><ul><li>I simulate outfits viewed/bought by users by selecting <strong>12 outfits of similar styles</strong>, in consultation with my wife. We will focus on three users: Alice, Bella and Cathy, each with different taste preference.</li><li><strong>I will only pass images into the model; the model does not ingest additional attributes/descriptions/product details; </strong>this is because we want the model to automatically learn and detect the style of fashion images passed into it without further human/machine labeling.</li></ul><p>Output:</p><ul><li>The goal is for collections generated by the model to<strong> contain some aspects of the input but with serendipity to pleasantly surprise users.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BM-hymK_WMzpqvtkEtSqLQ.jpeg" /><figcaption>Left: Outfits viewed/bought by Alice. Right: Outfits recommended by Fashion Collection. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p><strong>Alice</strong> is from sunny Singapore and loves shorts with outfits suitable for the hot weather. Based on her selected outfits, Fashion Collection recommends a variety of new outfits, all with shorts and seemingly ideal for life under the sun. <strong>Note that we did not explicitly tell the model to filter the outfits by shorts. </strong>Fashion Collection can pick up the signal that Alice likes outfits with shorts and make the appropriate recommendations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i3_9hio_RV2Z13Dv6muMTQ.jpeg" /><figcaption>Left: Outfits viewed/bought by Bella. Right: Outfits recommended by Fashion Collection. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>Bella is from artful Barcelona and loves flowery patterns and colorful designs in her outfit. Based on her selected outfits, Fashion Collection recommends a variety of new outfits across a variety of colors, patterns, and cuts similar to her taste.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H2Pp8uwMypwlOW1UfJ1ghA.jpeg" /><figcaption>Left: Outfits viewed/bought by Cathy. Right: Outfits recommended by Fashion Collection. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>Cathy is from cosmopolitan New York and dresses in her familiar urban lifestyle. The urban lifestyle is a <strong>much more complicated style</strong>, with layering, torn jeans, accessories, boots, graphic t-shirts as some of the defining features. It is thus exciting to see that Fashion Collection has recognized some of these critical features of the style and recommends a mix of outfits with layering, torn jeans, accessories, boots, graphic t-shirts!</p><blockquote>This is also why we do not want to hard code filters / limit the outfits by categories. <strong>We want Fashion Collection to be responsive to different users and changing user preferences — even for a style as diverse as urban lifestyle.</strong></blockquote><p>For the next section, I will walk through some of the major concepts, alongside the code fragments. Do not worry if you cannot understand all the theories in one reading. In essence, these are dense topics that are continually evolving; I will provide a high-level overview with additional resources under further readings for your review. We will follow the stages outlined in our flow above (data, prep, model, search).</p><h3>The Code</h3><p><a href="https://colab.research.google.com/gist/thiakx/c850d08ab37b7e57b8dae9ecfa1a3b8b/building-a-personalized-real-time-fashion-collection-recommender.ipynb"><strong>Link to code on Colab</strong></a><strong> </strong>(you just need a free Google Account to run the code on GPU in the cloud). We will make use of the Fastai package by Jeremy Howard (and the famed <a href="https://course.fast.ai/">Practical Deep Learning for Coders</a> course).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9903f6d2ccfa4570c3ec71ec01c1ac3/href">https://medium.com/media/d9903f6d2ccfa4570c3ec71ec01c1ac3/href</a></iframe><h3>[Prep] Code: Convert Images to Embeddings</h3><p>This conversion is the first step in the processing of data for our model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/77d6e81cf19bf9343222c96d22f01271/href">https://medium.com/media/77d6e81cf19bf9343222c96d22f01271/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*BbGUB0Pid9wqjVR03bN5yg.png" /><figcaption>Results from our code — image data processed and ready for training. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><h3>[Prep] Theory: What are Embeddings and Why Do We Need Them?</h3><p>To explain and illustrate the concept of embeddings, we will make use of a smaller open-source data set known as <a href="https://github.com/zalandoresearch/fashion-mnist">FashionMNIST</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u7Vv2lxf3MNKyo4qFR0u1g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*sFhj_AV8HB7NRri3ojDCSA.png" /><figcaption>Left: Sample images. Right: The numerical representation of images. Sample images from <a href="https://github.com/zalandoresearch/fashion-mnist">FashionMNIST,</a> open-source by <a href="https://hanxiao.io/2018/09/28/Fashion-MNIST-Year-In-Review/">Xiao H.</a></figcaption></figure><p>Traditionally, we represent images as a massive array of integers (3D array for RGB images and 1D array for grayscale images). These arrays are unwieldy and grow exponentially — we will need to keep track of millions of numbers to analyze hundreds of high-resolution images! It will be impossible to scale any modeling using arrays of integers; hence the modern approach of embeddings is created.</p><p>One excellent illustration of the embeddings concept is the example in “<a href="https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526">Neural Network Embeddings Explained</a>” by Will Koehrsen. Traditionally, we made use of techniques like one-hot encoding to represent items in a matrix. For example, given three books, we will have a 3x3 matrix, where each item is represented by an array of <strong>discrete</strong> numbers that<strong> grows with every new item added</strong> (4x4 with 4 items, 5x5 with 5 items…). Furthermore,<strong> there is no sense of similarity nor distance</strong> as the items are not connected with any form of relationship.</p><pre><strong># One Hot Encoding Categoricals</strong></pre><pre><strong>books = [&quot;Harry Potter and The Philosopher&#39;s Stone&quot;,<br>         &quot;Harry Potter and The Chamber of Secrets&quot;, <br>         &quot;The Lean Startup&quot;]</strong></pre><pre><strong>books_encoded = [[1, 0, 0],<br>                 [0, 1, 0],<br>                 [0, 0, 1]]</strong></pre><pre><strong>Similarity (dot product) between First and Second = 0<br>Similarity (dot product) between Second and Third = 0<br>Similarity (dot product) between First and Third = 0</strong></pre><p>Once we apply a transformation to convert the objects into embeddings, we now <strong>limit the number of elements in the array representing each item (the limit is 2 in this example) </strong>on a continuous scale, and the values have a relationship-based meaning. Objects close to each other based on similarity (dot product) are highly related.</p><pre><strong># Idealized Representation of Embedding</strong></pre><pre><strong>books = [&quot;Harry Potter and The Philosopher&#39;s Stone&quot;,<br>         &quot;Harry Potter and The Chamber of Secrets&quot;, <br>         &quot;The Lean Startup&quot;]</strong></pre><pre><strong>books_encoded_ideal = [[0.53,  0.85],<br>                       [0.60,  0.80],<br>                       [-0.78, -0.62]]</strong></pre><pre><strong>Similarity (dot product) between First and Second = 0.99<br>Similarity (dot product) between Second and Third = -0.94<br>Similarity (dot product) between First and Third = -0.97</strong></pre><blockquote>Instead of analyzing millions of discrete variables for each image, embeddings are vector representations of discrete variables. For deep learning, we usually make use of neural network embeddings to reduce the dimensionality of categorical variables into something managable</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*p4EuVzpy2Aojdj1ZxZ55tw.gif" /><figcaption>FashionMNIST embeddings projection. Images from <a href="https://github.com/zalandoresearch/fashion-mnist">FashionMNIST,</a> animated with Tensorboard</figcaption></figure><p>Because we can control the size of the vector representations, we can scale down a huge image array into a small vector made up of far fewer numbers. The result can be seen from the FashionMNIST objects in the image above, where the objects are projected onto the 3D vector space. Through the process of embeddings, <strong>image vectors that are similar will be projected close to each other</strong>. Thus, when we have the embeddings, we can project the objects into vector space and formulate the concept of distance and neighbors for visualization and simple recommendations. <strong>You can explore the concept of embeddings with the notebook below and read more about them under further readings section:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e188c4e77ae70cc6374323cfa57bbc91/href">https://medium.com/media/e188c4e77ae70cc6374323cfa57bbc91/href</a></iframe><h3>[Model] Code: Transfer Learning from Deep Convolutional Neural Networks (CNN)</h3><p>Fastai abstracts away a lot of the code logic in setting up the neural networks, allow us to do transfer learning with a few lines of code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/407ab708e49d9d2bb6c3217f6616b5b7/href">https://medium.com/media/407ab708e49d9d2bb6c3217f6616b5b7/href</a></iframe><p>It is also fairly straightforward to run the model and generate embeddings for each image with the Fastai hook. Note that we want the second last linear layer as that layer stores the embedding for the image representations; the last linear layer is the output layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d1duIRzV7PzFqyUH6pAv2g.png" /><figcaption>Note that the second to last linear layer is (4)</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e26208dad7901032eb0704fc906f37f2/href">https://medium.com/media/e26208dad7901032eb0704fc906f37f2/href</a></iframe><p>Hence when we call learner.model.module[1][4] to get the layer with embeddings.</p><h3>[Model] Theory: What are CNN?</h3><p>Convolution is not a new technique. In essence, we are applying a kernel to every pixel in the image to achieve a goal, usually to blur, sharpen or detect edges/objects. For each pixel, we will do an elementwise product with the kernel, followed by summing the result to a get a single number.</p><blockquote>Since each kernel specializes in detecting one aspect of the image, you can imagine us stacking up different kernels to formulate a comprehensive strategy. That is indeed the case and a collection of kernels is called a filter. In CNN, we can even stack multiple layers of filters, with each filter designated with a specific task.</blockquote><p><a href="https://setosa.io/ev/image-kernels/">If you are interested, you should try out the tool developed by Victor Powell with different types of kernels</a>. CNN is a fascinating model as it combines the power of convolution and neural networks. There are many different architectures but generally consist of <a href="https://algobeans.com/2016/01/26/introduction-to-convolutional-neural-network/">a combination of convolution, subsampling, activation and full connectedness, as noted by Algobeans</a>. <strong>You can learn more about kernels and CNN under the further readings section.</strong></p><h3>[Model] Theory: How Does Transfer Learning Works?</h3><blockquote>For most real-world deployment, we do not train a CNN from scratch. Organizations like Microsoft Research has released state-of-the-art, large scale, pre-trained deep CNN (DCNN) models over the years, and we should leverage on their work by training on top of their baseline models. This is known as transfer learning.</blockquote><p>Concretely, we take a large pre-trained DCNN like <a href="https://arxiv.org/abs/1512.03385">ResNet50 (He K., et al.)</a>, freezes most of the layers, and train on the last few layers. The intuition is that for most of the DCNN, the knowledge gained by the filters is transferable components such as detection of edges, patterns, gradients, colors, etc. Thus, we fine-tune the last few layers to our problem, and we can have a working solution. <strong>You can explore the concept of transfer learning under further readings section.</strong></p><h3>[Search] Code: Approximate Nearest Neighbors with Annoy &amp; Embeddings Centroid Detection</h3><p>We run Annoy through each row in the dataframe and find the nearest neighbors with the get_nss_by_vector function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fd78a0b488dcec7a8da88bcc5367ec0f/href">https://medium.com/media/fd78a0b488dcec7a8da88bcc5367ec0f/href</a></iframe><p>Finally, we insert the embeddings of all 12 (or more) user-selected items into a list and <strong>average the embeddings values across each of the dimensions</strong>; <strong>This creates a phantom object that represents the aggregated value across all the selected items.</strong> We can then find the nearest neighbor to this phantom item, which can be done in ~2ms when we call Annoy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f641bcd15e4f0210af36414c4d4ffec7/href">https://medium.com/media/f641bcd15e4f0210af36414c4d4ffec7/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*up9auyBRepCtLfqPONJXqQ.png" /><figcaption>The time it took to generate Anything with Shorts Collection</figcaption></figure><h3>[Search] Theory: Approximate Nearest Neighbors</h3><p>If we only have a small corpus of images to search, simple distance metrics like cosine similarity will work. In real-world deployments, such as e-commerce, we usually have millions of images to compare with each other, and it will be impractical for the API to execute pair-wise comparisons across every single image. <a href="https://github.com/spotify/annoy">Annoy</a> (Approximate Nearest Neighbors Oh Yeah) is by Erik Bernhardsson from Spotify, with an easy to use API that can be integrated into our PyTorch workflow.</p><blockquote>More importantly, it helps us find closest neighbors without the need to calculate pair-wise distance across every single image.</blockquote><p>If you are interested to learn more about annoy, do check out Erik’s article under further readings.</p><h3>What have we learned</h3><p>We walk through the entire process of building a personalized real-time Fashion Collection recommender. With modern tools like PyTorch, Fastai, and Annoy, we constructed a powerful product that can generate relevant recommendations based on the style of our users.</p><h3>Further Readings</h3><ul><li><a href="https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526">Neural Network Embeddings Explained</a></li><li><a href="https://towardsdatascience.com/types-of-convolution-kernels-simplified-f040cb307c37">Types of Convolution Kernels : Simplified</a></li><li><a href="https://aishack.in/tutorials/image-convolution-examples/">Image convolution examples</a></li><li><a href="https://youtu.be/YRhxdVk_sIs">Convolutional Neural Networks (CNNs) explained by deeplizard (Video)</a></li><li><a href="https://algobeans.com/2016/01/26/introduction-to-convolutional-neural-network/">Convolutional Neural Networks (CNN) Introduction</a></li><li><a href="https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/">A Beginner’s Guide To Understanding Convolutional Neural Networks</a></li><li><a href="https://towardsdatascience.com/transfer-learning-with-convolutional-neural-networks-in-pytorch-dd09190245ce">Transfer Learning with Convolutional Neural Networks in PyTorch</a></li><li><a href="https://shuzhanfan.github.io/2018/11/ResNet/">Understanding Deep Residual Networks</a></li><li><a href="https://towardsdatascience.com/an-overview-of-resnet-and-its-variants-5281e2f56035">An Overview of ResNet and its Variants</a></li><li><a href="https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a">A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning</a></li><li><a href="https://machinelearningmastery.com/transfer-learning-for-deep-learning/">A Gentle Introduction to Transfer Learning for Deep Learning</a></li><li><a href="https://erikbern.com/2015/10/01/nearest-neighbors-and-vector-models-part-2-how-to-search-in-high-dimensional-spaces.html">Erik: Annoy: Nearest neighbors and vector models — part 2 — algorithms and data structures</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22dc90c150cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[COVID-19 Case Study with CNN]]></title>
            <link>https://medium.com/analytics-vidhya/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f07fd93a11a</guid>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Sat, 21 Mar 2020 09:29:11 GMT</pubDate>
            <atom:updated>2020-04-10T16:49:19.768Z</atom:updated>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>We will cluster COVID-19 X-ray images based on severity with our CNN RecSys flow using transfer learning, Spotify’s Annoy, and PyTorch</h4><p>This work is meant as a proof-of-concept of how we can apply the same framework we set up in <a href="https://medium.com/@thiakx/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">the previous CNN chapter</a> onto a completely different domain.</p><p><strong>We will swap out the training data and employ a more powerful pre-trained model (Resnet152); the rest of the code remains identical to the one we used for DeepFashion images. </strong>We aim to identify clusters of X-ray images with similar severity in infection using Approximate Nearest Neighbors.</p><blockquote>This work is not intended as medical research nor representative of how we can use CNN to detect COVID-19.</blockquote><p>This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.</p><h3>The COVID-19 Data</h3><ul><li>Joseph Cohen from the University of Montreal is <a href="https://github.com/ieee8023/covid-chestxray-dataset">building a database of COVID-19 cases with chest X-ray or CT images</a>. Currently, the data set consists of <strong>124</strong> COVID-19 x-ray images, including cases across Asia, Europe, and the US.</li><li>To enrich the data, I merged another <strong>1.5k X-ray images</strong> of healthy patients from the <a href="https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia">Kaggle’s Chest X-Ray Images (Pneumonia) dataset</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*K8ob658ty9NPENdXifb4_g.jpeg" /><figcaption>From left to right: Healthy, infected, seriously-infected X-ray images of patients. Source: <a href="https://github.com/ieee8023/covid-chestxray-dataset">COVID-19 image data collection</a> by <a href="https://josephpcohen.com/w/">Joseph Cohen</a></figcaption></figure><p>Intuition of why CNN will be able to work well on this data set:</p><blockquote>As outlined in the previous chapter, the strength of CNN is in the convolutional filters. These filters are very good at detecting shapes, lines, boundaries within the image. From the X-ray images, we see that as the infection worsens, the image blurs with more white areas and the rib cage becomes less visible; these are visual cues that CNN will be able to pick up and learn.</blockquote><h3>Cleaning the data</h3><ul><li>As there are less than 25 samples of ARDS, Pneumocystis, SARS &amp; Streptococcus in total, I decided to remove those samples and only keep COVID and healthy samples.</li><li>As there are less than 25 samples of CT scans and only 1 CT scan for healthy patients, I decided to remove CT scans and only keep X-rays.</li><li>After the cleaning, we have 102 COVID X-rays and 1,584 healthy X-rays.</li></ul><h3>Model training</h3><p>We will follow the exact same steps outlined in the previous Convolutional Neural Networks RecSys chapter (you can refer back to that chapter for more details):</p><ul><li>Convert images to embeddings</li><li>Conduct Transfer Learning from ResNet152</li><li>Use Fastai hooks to retrieve image embeddings from step 2</li><li>Use Approximate Nearest Neighbors to obtain the most similar images based on the embeddings from step 3.</li></ul><h3>Analysis</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*cxLg-lxRVuhO8IFSunTrCg.jpeg" /><figcaption>Healthy X-ray scan with 36 most similar scans generated by our model. Source: <a href="https://github.com/ieee8023/covid-chestxray-dataset">COVID-19 image data collection</a> by <a href="https://josephpcohen.com/w/">Joseph Cohen</a></figcaption></figure><p>For healthy X-ray scans, our model can pick up 36 most similar X-rays that are all healthy. The model can identify and cluster healthy scans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*qIQ-wvC0fe5qi2mfISmyig.jpeg" /><figcaption>Infected X-ray scan with 36 most similar scans generated by our model. Source: <a href="https://github.com/ieee8023/covid-chestxray-dataset">COVID-19 image data collection</a> by <a href="https://josephpcohen.com/w/">Joseph Cohen</a></figcaption></figure><p>For infected X-ray scans, our model usually picks up a mix of 80% infected X-ray scans and 20% healthy scans. Depending on the degree of infection, the model finds it challenging to differentiate between the lightly infected scans and healthy scans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*vs0lmF-hgOy0L7zZYghqLw.jpeg" /><figcaption>Seriously-Infected X-ray scan with 36 most similar scans generated by our model. Source: <a href="https://github.com/ieee8023/covid-chestxray-dataset">COVID-19 image data collection</a> by <a href="https://josephpcohen.com/w/">Joseph Cohen</a></figcaption></figure><p>For the seriously infected X-ray scans, our model can pick up 36 most similar X-rays that are all infected. The model can identify and cluster seriously infected scans.</p><h3>Potential use case of this work</h3><p>We can use this model to track the change in scan severity over time. If the scan today has fewer healthy neighboring scans and is drifting towards the seriously-infected cluster, this is a sign that the condition of the patient has worsened over time.</p><h3>The Code</h3><p><a href="https://colab.research.google.com/gist/thiakx/79a247354065c4428d084271fabf2b73/covid-19-case-study-with-cnn.ipynb">Link to Colab</a> (you just need a free Google Account to run the code on GPU in the cloud)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40d53cf02b96580fa93cc89d60a53cd6/href">https://medium.com/media/40d53cf02b96580fa93cc89d60a53cd6/href</a></iframe><h3>What have we learned</h3><p>In this chapter, we explore the use of our previously developed CNN Recsys flow in the healthcare domain. We observed how we can train a powerful model with minimum changes to our code, showcasing the flexibility of our flow.</p><h3>Explore the rest of Modern Visual RecSys Series</h3><ul><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part1-introduction-1241c02f76d6">Modern Visual RecSys: How does a recommender work?</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34">Modern Visual RecSys: How to Design a Recommender?</a></li><li><a href="https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521">Modern Visual RecSys: Intro to Visual RecSys</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">Modern Visual RecSys: Convolutional Neural Networks Recommender</a></li><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a></li><li><a href="https://medium.com/@thiakx/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a></li></ul><p>Series labels:</p><ul><li>Foundational: general knowledge and theories, minimum coding experience needed.</li><li>Core: more challenging materials with code.</li><li>Pro: Difficult materials and code, with production-grade tools.</li></ul><h3>Further Readings</h3><ul><li><a href="https://www.pyimagesearch.com/2020/03/16/detecting-covid-19-in-x-ray-images-with-keras-tensorflow-and-deep-learning/?fbclid=IwAR0DE2Boiy7JxlHBV8DC91ggEJ7TWTEPKK9K9TFqp0tUUEakHyueSo8OFw4">Detecting COVID-19 in X-ray images with Keras, TensorFlow, and Deep Learning</a></li><li><a href="https://towardsdatascience.com/using-deep-learning-to-detect-ncov-19-from-x-ray-images-1a89701d1acd">Using Deep Learning to detect Pneumonia caused by NCOV-19 from X-Ray Images</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f07fd93a11a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">COVID-19 Case Study with CNN</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Convolutional Neural Networks Recommender]]></title>
            <link>https://medium.com/analytics-vidhya/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4174644ab0d</guid>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Fri, 20 Mar 2020 12:09:16 GMT</pubDate>
            <atom:updated>2020-05-03T11:52:09.784Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tDEPgijshutIAZ0IsO_twQ.jpeg" /><figcaption>Target image on left, recommendations generated by our model on the right. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><h4>Modern RecSys</h4><h4>We will build a recommender with transfer learning, Spotify’s Annoy, PyTorch, and return visually similar products across 240K images in 2ms</h4><p>This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.</p><h3>The Data</h3><p>We will be using a subset of<strong> </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>DeepFashion</strong></a><strong> data open-sourced by Liu Z. et al., </strong>The Chinese University of Hong Kong. Our data consists of <strong>280K fashion images across 46 categories. You can </strong><a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html"><strong>download the data from their website.</strong></a></p><p>Furthermore, the team has released an <a href="https://github.com/switchablenorms/DeepFashion2">updated version with additional data</a>. You will need to <a href="https://docs.google.com/forms/d/e/1FAIpQLSeIoGaFfCQILrtIZPykkr8q_h9qQ5BoTYbjvf95aXbid0v2Bw/viewform?usp=sf_link">fill up a Google form</a> to gain access to the data.</p><h3>What is Convolution?</h3><p>Convolution is not a new technique. In essence, we are applying a kernel to every pixel in the image to achieve a goal, usually to blur, sharpen or detect edges/objects. For each pixel, we will do an elementwise product with the kernel, followed by summing the result to a get a single number.</p><p>Let us walk through an example with the image <a href="https://setosa.io/ev/image-kernels/">kernels tool developed by Victor Powell</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*1ls32N8k6ZbjVpl5l2oc1A.png" /><figcaption>The base image. Source: <a href="https://setosa.io/ev/image-kernels/">Image Kernels</a> by <a href="http://twitter.com/vicapow">Victor Powell</a></figcaption></figure><p>We can see that every pixel of the image has a color value associated with it, where white =255 and black = 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*oja_6Az96lTJFP8Hm4ydZA.png" /><figcaption>Picking the kernel. Source: <a href="https://setosa.io/ev/image-kernels/">Image Kernels</a> by <a href="http://twitter.com/vicapow">Victor Powell</a></figcaption></figure><p>Next, we will pick the kernel. The kernel can be of any size, but of course, it will take a longer time for a small kernel to scan through a large image. <a href="https://en.wikipedia.org/wiki/Sobel_operator">Sobel</a> is a ubiquitous edge detection algorithm with smoothing so that it is less susceptible to noise. Notice how there are different kinds of Sobel (top, bottom, left, right) and like their namesake, these kernels are designed to pick up specific components of the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*63CHendCrhrDJOOlx4COtQ.gif" /><figcaption>This is convolution. Source: <a href="https://setosa.io/ev/image-kernels/">Image Kernels</a> by <a href="http://twitter.com/vicapow">Victor Powell</a></figcaption></figure><p>As you can see from the animation, we are essentially moving a 3x3 kernel across the image, generating new scores and assigning them to the output image. You will notice that after applying the bottom Sobel, only parts of the output image is highlighted in white; these white sections are the bottom edges detected by bottom Sobel.</p><blockquote>Since each kernel specializes in detecting one aspect of the image, you can imagine us stacking up different kernels to formulate a comprehensive strategy. That is indeed the case and a collection of kernels is called a filter. In CNN, we can even stack multiple layers of filters, with each filter designated with a specific task.</blockquote><p><a href="https://setosa.io/ev/image-kernels/">If you are interested, you should try out the tool yourself with different types of kernels</a>. CNN is a fascinating model as it combines the power of convolution and neural networks. There are many different architectures but generally consist of <a href="https://algobeans.com/2016/01/26/introduction-to-convolutional-neural-network/">a combination of convolution, subsampling, activation and full connectedness, as noted by Algobeans</a>. You can learn more about kernels and CNN under the additional resources section.</p><h3>Why CNN for Visual Recommendations?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*CrEn0opfPGlYrIgzB-GduA.png" /><figcaption>Closeup of filters 14, 242, 250 and 253. Source: <a href="https://benanne.github.io/2014/08/05/spotify-cnns.html#">Recommending music on Spotify slides</a> by <a href="https://benanne.github.io/about/">Sander Dieleman</a></figcaption></figure><p>Now is an excellent time to review what we learned back in <a href="https://medium.com/@thiakx/modern-visual-recsys-part1-introduction-1241c02f76d6">part1 of this series.</a> Sander from Spotify designed a CNN with filters to detect the different types of music based on their frequency patterns. CNN opens a brand new way of recommending music that is <strong>intuitive, </strong>as it is based on analyzing and understanding the structure of music. Machines lacked the natural ability to comprehend and appreciate music; CNN help bridge the gap.</p><blockquote>The power of CNN is its ability to break down a complex visual problem into layers of filters — quite often we can visualize these filters to gain an intuition of what the model is trying to learn.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*avh0eZ6nF8DFLVCXgoPjVQ.png" /><figcaption>CNN based recommendation. Source: <a href="https://symj07.wordpress.com/2018/03/19/poshnet/">PoshNet</a> by <a href="https://symj07.wordpress.com/author/windflower715hotmailcom/">Summer Yuan</a></figcaption></figure><p>Thus, our goal is to build a CNN that can recommend items based on <strong>visual similarity with the input image</strong>. CNN can be applied across a wide variety of visual problems, and I have collected a list of great articles below. Note that in <a href="https://medium.com/@thiakx/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">the next chapter</a> we will adapt our CNN flow to identify clusters of X-ray images with similar severity in infection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*f1-CfVAOaw1KuQAGp8W8cA.jpeg" /></figure><p><em>Seriously-Infected X-ray scan with 36 most similar scans generated by our model. Source: </em><a href="https://github.com/ieee8023/covid-chestxray-dataset"><em>COVID-19 image data collection</em></a><em> by </em><a href="https://josephpcohen.com/w/"><em>Joseph Cohen</em></a></p><ul><li><a href="http://ai.sensilab.monash.edu/2018/09/17/similarity-search-engine/">Searching for Visually Similar Artworks by SensiLab, Monash</a></li><li><a href="https://towardsdatascience.com/finding-familiar-faces-with-a-tensorflow-object-detector-pytorch-feature-extractor-and-spotifys-3e78858a8148">Finding Familiar Faces (in Anime Characters) with a Tensorflow Object Detector, Pytorch Feature Extractor, and Spotify’s Annoy by Michael Sugimura</a></li><li><a href="https://blog.usejournal.com/fastai-image-similarity-search-pytorch-hooks-spotifys-annoy-9161bf517aaf">Fastai — Image Similarity Search — Pytorch Hooks &amp; Spotify’s Annoy by Abhik Jha</a></li><li><a href="https://towardsdatascience.com/similar-images-recommendations-using-fastai-and-annoy-16d6ceb3b809">Similar Images Recommendations using FastAi and Annoy by Gautham Kumaran</a></li></ul><h3>Transfer Learning: Leverage Pre-Trained Deep CNN</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*CnMaVu8eWoD9nnM6rBqMgw.png" /><figcaption><a href="https://arxiv.org/pdf/1810.00736.pdf">Benchmark Analysis of Representative Deep Neural Network Architectures (2018)</a>. Source: <a href="https://arxiv.org/">arxiv</a> by <a href="https://www.researchgate.net/profile/Simone_Bianco">S. Bianco</a> et al.</figcaption></figure><blockquote><strong>For most real-world deployment, we do not train a CNN from scratch. </strong>Organizations like Microsoft Research has released state-of-the-art, large scale, pre-trained deep CNN (DCNN) models over the years, and we should leverage on their work by training on top of their baseline models. This is known as transfer learning.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8vDlURGKX711IUrogIu9uw.png" /><figcaption>ResNet Architectures. 18-layer ResNet is an excellent baseline to test model while 152-layer is a great general-purpose model. Source: <a href="https://arxiv.org/abs/1512.03385">Deep Residual Learning for Image Recognition</a> by <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=He%2C+K">He K. et al</a>.</figcaption></figure><p>One of the standard pre-trained DCNN is ResNet. Deeper networks have the potential of better representing the input function. The problem with deep networks is the vanishing gradient problem, as we will need to multiply small numbers repeatedly to conduct the backpropagation. ResNet solves this problem with the <strong>identity shortcut connection that skips one or more layers</strong>, allow us to construct very deep networks that generalize well over a variety of problems. See further readings section for more details on ResNet.</p><h3>Approximate Nearest Neighbors with Annoy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z3AtogfYPXWM_GdZAcuQKw.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*IbgjQFZW4tHjmTG-wKKr8A.png" /><figcaption><a href="https://erikbern.com/2015/10/01/nearest-neighbors-and-vector-models-part-2-how-to-search-in-high-dimensional-spaces.html">Who says binary trees are only useful for Leetcode interviews?</a> Source <a href="https://github.com/spotify/annoy">Annoy</a> by <a href="https://erikbern.com/about.html">Erik</a></figcaption></figure><p>If we only have a small corpus of images to search, simple distance metrics like cosine similarity will work. In real-world deployments, such as e-commerce, we usually have millions of images to compare with each other, and it will be impractical for the API to execute pair-wise comparisons across every single image. <a href="https://github.com/spotify/annoy">Annoy</a> (Approximate Nearest Neighbors Oh Yeah) is by Erik Bernhardsson from Spotify, with an easy to use API that can be integrated into our PyTorch workflow.</p><blockquote>More importantly, it helps us find closest neighbors without the need to calculate pair-wise distance across every single image.</blockquote><p>If you are interested to learn more about annoy, do checkout Erik’s article under further readings.</p><h3>The Code</h3><p>Please refer to <a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb</a></p><h3>Reviewing Results from the recommender</h3><p>Let us take a look at the results of the recommender. For example, we observe that some items are easier to recommend, such as this striped sweater.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0teTMPPgA9ZojwedCdlVCw.jpeg" /><figcaption>Target image on left, recommendations generated by our model on the right. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>White jeans are a little harder; we seem to end with up with a mix of leggings, black pants and blue jeans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fG9xEqMyOWbGiPB9EpVMGg.jpeg" /><figcaption>Target image on left, recommendations generated by our model on the right. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><p>While this…interesting outfit results in very diverse set of recommendations. It seems challenging to matching complex colors, layers, and outfits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lvqsxnEHYZLqCF_q9yWOPQ.jpeg" /><figcaption>Target image on left, recommendations generated by our model on the right. Outfits from <a href="http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html">DeepFashion,</a> open-source by<a href="https://liuziwei7.github.io/"> <strong>Liu Z. et al.</strong></a></figcaption></figure><h3>What have we learned</h3><p>In this chapter, we explore the use of CNN in recommendations. We are using a couple of advanced techniques here, but with modern tools like ResNet, Fastai, Annoy, we can deploy a powerful recommender that generates new recommendations instantly.</p><h3>Explore the rest of Modern Visual RecSys Series</h3><ul><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part1-introduction-1241c02f76d6">Modern Visual RecSys: How does a recommender work?</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34">Modern Visual RecSys: How to Design a Recommender?</a></li><li><a href="https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521">Modern Visual RecSys: Intro to Visual RecSys</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">Modern Visual RecSys: COVID-19 Case Study with CNN</a></li><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a></li><li><a href="https://medium.com/@thiakx/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a></li></ul><p>Series labels:</p><ul><li>Foundational: general knowledge and theories, minimum coding experience needed.</li><li>Core: more challenging materials with code.</li><li>Pro: Difficult materials and code, with production-grade tools.</li></ul><h3>Further Readings</h3><ul><li><a href="https://towardsdatascience.com/types-of-convolution-kernels-simplified-f040cb307c37">Types of Convolution Kernels : Simplified</a></li><li><a href="https://aishack.in/tutorials/image-convolution-examples/">Image convolution examples</a></li><li><a href="https://youtu.be/YRhxdVk_sIs">Convolutional Neural Networks (CNNs) explained by deeplizard (Video)</a></li><li><a href="https://algobeans.com/2016/01/26/introduction-to-convolutional-neural-network/">Convolutional Neural Networks (CNN) Introduction</a></li><li><a href="https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/">A Beginner’s Guide To Understanding Convolutional Neural Networks</a></li><li><a href="https://towardsdatascience.com/build-a-fashion-mnist-cnn-pytorch-style-efb297e22582">Let’s Build a Fashion-MNIST CNN, PyTorch Style</a></li><li><a href="https://symj07.wordpress.com/2018/03/19/poshnet/">PoshNet: You personal virtual closet</a></li><li><a href="https://towardsdatascience.com/an-overview-of-resnet-and-its-variants-5281e2f56035">An Overview of ResNet and its Variants</a></li><li><a href="https://towardsdatascience.com/review-inception-v4-evolved-from-googlenet-merged-with-resnet-idea-image-classification-5e8c339d18bc">Review: Inception-v4 — Evolved From GoogLeNet, Merged with ResNet Idea (Image Classification)</a></li><li><a href="https://towardsdatascience.com/a-simple-guide-to-the-versions-of-the-inception-network-7fc52b863202">A Simple Guide to the Versions of the Inception Network</a></li><li><a href="https://erikbern.com/2015/10/01/nearest-neighbors-and-vector-models-part-2-how-to-search-in-high-dimensional-spaces.html">Erik: Annoy: Nearest neighbors and vector models — part 2 — algorithms and data structures</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4174644ab0d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">Convolutional Neural Networks Recommender</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intro to Visual RecSys]]></title>
            <link>https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/12d54976c521</guid>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Thu, 12 Mar 2020 15:30:12 GMT</pubDate>
            <atom:updated>2020-04-10T16:48:16.751Z</atom:updated>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>In this chapter, we will explore the “hello world” data for visual models, the <a href="https://github.com/zalandoresearch/fashion-mnist">FashionMNIST</a> dataset from Zalando with PyTorch, Tensorboard and Colab.</h4><p>This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.</p><h3>FashionMNIST and the visual challenge</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*0HbqSFOBcx0qwd-Y3__-CA.png" /><figcaption>Source: <a href="https://research.zalando.com/welcome/mission/research-projects/fashion-mnist/">FashionMNIST</a> by by <a href="https://research.zalando.com/welcome/team/kashif-rasul/">Kashif Rasul</a> &amp; <a href="https://hanxiao.io/about/">Han Xiao</a></figcaption></figure><p>The data consists of:</p><ul><li>Training set of 60,000 images and a test set of 10,000 images.</li><li>Each image is 28x28 grayscale, across 10 classes: T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot.</li></ul><p>Our goal is to gain an intuitive understanding of two critical concepts: <strong>embeddings</strong> and <strong>distance</strong>, as they are fundmental building blocks to the following chapter on <em>Convolutional Neural Networks (CNN).</em></p><h3>What are embeddings and why we need them?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8IF7PIXdoEBt83JoKaCc_g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*rzVK9c5Z2s0k0sqvdxD3qQ.png" /><figcaption>Left: The images from FashionMNIST. Right: The numerical representation of images</figcaption></figure><p>Traditionally, we represent images as a massive array of integers (3D array for RGB images and 1D array for grayscale images). These arrays are unwieldy and grow exponentially — we will need to keep track of millions of numbers to analyze hundreds of high-resolution images! It will be impossible to scale any modeling using arrays of integers; hence the modern approach of embeddings is created.</p><p>One excellent illustration of the embeddings concept is the example in “<a href="https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526">Neural Network Embeddings Explained</a>” by Will Koehrsen. Traditionally, we made use of techniques like one-hot encoding to represent items in a matrix. For example, given three books, we will have a 3x3 matrix, where each item is represented by an array of <strong>discrete</strong> numbers that<strong> grows with every new item added</strong> (4x4 with 4 items, 5x5 with 5 items…). Furthermore,<strong> there is no sense of similarity nor distance</strong> as the items are not connected with any form of relationship.</p><pre><strong># One Hot Encoding Categoricals</strong></pre><pre><strong>books = [&quot;Harry Potter and The Philosopher&#39;s Stone&quot;,<br>         &quot;Harry Potter and The Chamber of Secrets&quot;, <br>         &quot;The Lean Startup&quot;]</strong></pre><pre><strong>books_encoded = [[1, 0, 0],<br>                 [0, 1, 0],<br>                 [0, 0, 1]]</strong></pre><pre><strong>Similarity (dot product) between First and Second = 0<br>Similarity (dot product) between Second and Third = 0<br>Similarity (dot product) between First and Third = 0</strong></pre><p>Once we apply a transformation to convert the objects into embeddings, we now <strong>limit the number of elements in the array representing each item (the limit is 2 in this example) </strong>on a continuous scale, and the values have a relationship-based meaning. Objects close to each other based on similarity (dot product) are highly related.</p><pre><strong># Idealized Representation of Embedding</strong></pre><pre><strong>books = [&quot;Harry Potter and The Philosopher&#39;s Stone&quot;,<br>         &quot;Harry Potter and The Chamber of Secrets&quot;, <br>         &quot;The Lean Startup&quot;]</strong></pre><pre><strong>books_encoded_ideal = [[0.53,  0.85],<br>                       [0.60,  0.80],<br>                       [-0.78, -0.62]]</strong></pre><pre><strong>Similarity (dot product) between First and Second = 0.99<br>Similarity (dot product) between Second and Third = -0.94<br>Similarity (dot product) between First and Third = -0.97</strong></pre><blockquote>Instead of analyzing millions of discrete variables for each image, embeddings are vector representations of discrete variables. For deep learning, we usually make use of neural network embeddings to reduce the dimensionality of categorical variables into something managable</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*p4EuVzpy2Aojdj1ZxZ55tw.gif" /><figcaption>FashionMNIST embeddings projection on Tensorboard — for details, see “The Code” section</figcaption></figure><p>Because we can control the size of the vector representations, we can scale down a huge image array into a small vector made up of far fewer numbers. The result can be seen from the FashionMNIST objects in the image above, where the objects are projected onto the 3D vector space. Through the process of embeddings, <strong>image vectors that are similar will be projected close to each other</strong>. Thus, when we have the embeddings, we can:</p><ul><li>Project the objects into vector space and formulate the concept of distance and neighbors for visualization and simple recommendations (this chapter).</li><li>Make use of embeddings to train deep learning models (next chapter)</li></ul><p>Let’s take a look at how we can build such embeddings.</p><h3>The tools</h3><p>A quick overview of the tools that we will be using for the coding sections:</p><ul><li><a href="https://pytorch.org/"><strong>PyTorch</strong></a> vs. Tensorflow: They are the two dominating frameworks in the deep learning space. PyTorch <a href="https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research-tensorflow-dominates-industry/">is gaining momentum in research </a>due to its simplicity and pythonic nature that makes it easier to integrate into python workflows. Even <a href="https://openai.com/blog/openai-pytorch/">organizations like OpenAI is switching to PyTorch</a>. I find Tensorflow verbose and confusing to use due to changes in Tensorflow 2.0 that break old code with new function names and parameters. PyTorch’s simplicity fits our workflow. We will use PyTorch in all our code examples. I have added PyTorch learning materials (book and tutorials) under the further readings section.</li><li><a href="https://www.tensorflow.org/tensorboard"><strong>Tensorboard</strong></a>: Tensorboard used to be one key differentiating factor for Tensorflow. Now that <a href="https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html">Tensorboard supports PyTorch natively</a>, we can code in PyTorch and visualize it in Tensorboard.</li><li><a href="https://colab.research.google.com/"><strong>Colab</strong></a>: Google’s Colab hosts Jupyter notebooks in the cloud with GPU access for free. This is an excellent way for us to share code and explore deep learning frameworks without the hassle and cost of setting up the GPU environment (you just need a free Google Account).</li></ul><h3>The Code</h3><p>You can preview most of the code below but you will need to start Colab in order to interact with the Tensorboard (the most important part of this chapter).</p><p><a href="https://colab.research.google.com/gist/thiakx/cd734300abe9dbf3db425ec026f80ffe/intro-to-visual-recsys.ipynb">Start Colab</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e188c4e77ae70cc6374323cfa57bbc91/href">https://medium.com/media/e188c4e77ae70cc6374323cfa57bbc91/href</a></iframe><h3>What have we learned</h3><p>In this chapter, we learn about embeddings, how they work and why they matter. We also explored the code in PyTorch, experimented with the visualization in Tensorboard to have an intuitive understanding of how recommendations can be done via embeddings.</p><p>In the next chapter, we will build on all these understanding to develop a Convolutional Neural Networks (CNN) based recommender.</p><h3>Explore the rest of Modern Visual RecSys Series</h3><ul><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part1-introduction-1241c02f76d6">Modern Visual RecSys: How does a recommender work?</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34">Modern Visual RecSys: How to Design a Recommender?</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">Modern Visual RecSys: Convolutional Neural Networks Recommender</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">Modern Visual RecSys: COVID-19 Case Study with CNN</a></li><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a></li><li><a href="https://medium.com/@thiakx/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a></li></ul><p>Series labels:</p><ul><li>Foundational: general knowledge and theories, minimum coding experience needed.</li><li>Core: more challenging materials with code.</li><li>Pro: Difficult materials and code, with production grade tools.</li></ul><h3>Further Readings</h3><ul><li><a href="https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526">Neural Network Embeddings Explained</a></li><li><a href="https://pytorch.org/tutorials/index.html">PyTorch Tutorials</a></li><li><a href="https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html">Visualizing Models, Data, and Training with TensorBoard</a></li><li><a href="https://pytorch.org/deep-learning-with-pytorch">Deep Learning with PyTorch Book (Official, Free)</a></li><li><a href="https://course.fast.ai/">fast.ai Practical Deep Learning for Coders, v3</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12d54976c521" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521">Intro to Visual RecSys</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Design a Recommender?]]></title>
            <link>https://medium.com/analytics-vidhya/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/30d2352fff34</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Wed, 04 Mar 2020 15:57:20 GMT</pubDate>
            <atom:updated>2020-04-10T16:48:28.245Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>For this chapter, I will introduce the RecSys Design Framework with a case study of Amazon.</h4><p>This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.</p><h3>RecSys Framework — Amazon case study</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OAWl5DIM6iMeSUc3hhGqRA.png" /><figcaption>Recommendations on my <a href="https://www.amazon.com/">Amazon</a> homepage</figcaption></figure><p>An eCommerce website like Amazon is heavily reliant on having a good RecSys. After all, <strong>users cannot be expected to browse through millions of products </strong>that are on the platform, while<strong> sellers will like exposure </strong>for their products. Furthermore, there is <strong>limited space</strong> on the website/app; a good RecSys should be able to match the user’s preferences with the most relevant product and display the results in a certain order that encourages the user to click or purchase the item.</p><p>How do we go about designing a RecSys from scratch? Let me share with you a framework I use.</p><p><strong>Step 1: Define The business case</strong></p><p>There is no clear “correct answer” for RecSys. From the Amazon recommendations above, how do you tell if the RecSys is doing a great job? If I bought socks and monitors in the past, does it still make sense to recommend me more socks and monitors? RecSys is all about improving user experience and business KPIs. It is thus vital to understand the metrics involve:</p><p><strong><em>RecSys metrics:</em></strong></p><ul><li>“How well it matches historical trends”: <a href="https://towardsdatascience.com/evaluation-metrics-for-recommender-systems-df56c6611093">accuracy / relevancy / coverage</a></li><li>“How diverse are the recommendations”: <a href="https://gab41.lab41.org/recommender-systems-its-not-all-about-the-accuracy-562c7dceeaff#.5exl13wqv">diversity / serendipity / novelty</a></li><li><a href="https://www.youtube.com/watch?v=Yz7fKIHfFW8">Respecting user Privacy</a></li><li><a href="https://medium.com/@InDataLabs/approaching-the-cold-start-problem-in-recommender-systems-e225e0084970">Cold start problem</a> (the challenge of recommending to new users)</li></ul><p><strong><em>Business metrics:</em></strong></p><ul><li>Clickthrough rate (CTR)</li><li>Purchase rate</li><li>Introduce new product / seller</li></ul><p>While accuracy/relevancy gives as a ballpark figure to understand how well our RecSys matches up with historical trends, <strong>they do not show the full picture.</strong></p><blockquote>Users cannot judge items they have not yet seen. In essence, a RecSys can perform poorly on historical measures while introducing lots of new products to users and increase the overall purchase rate. It is <strong>crucial to have a mix of historical, diversity, and business metrics measured through </strong><a href="https://www.coursera.org/lecture/recommender-metrics/a-b-studies-field-experiments-W8dHo"><strong>AB testing with different demographics of users.</strong></a></blockquote><p><strong>Step 2: Prepare the Data</strong></p><p>Let us take a look at the patent filed by Amazon in 1998 that contains the foundation of RecSys, relevant even till today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y5F4rHNraPvqpv5vu1qDgQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUov88HT-0Ge8z_neunfLA.png" /><figcaption>Source: Patent US 6,266,649 B1: <a href="https://insight.rpxcorp.com/pat/US6266649B1">Collaborative recommendations using item-to-item similarity mappings</a> by Amazon</figcaption></figure><p>From the patent image above, we can outline the various pieces of data that are found in all modern RecSys:</p><ul><li>User interactions (shown as web server — html in image). These are usually clickstream data that are stored as log files and accessed via a <a href="https://www.confluent.io/blog/using-apache-kafka-drive-cutting-edge-machine-learning/">publish-subscribe messaging system like Kafka</a>. A common technique to track users is <a href="https://pixelprivacy.com/resources/browser-fingerprinting/">browser fingerprinting,</a> which comes with issues of <a href="https://blog.mozilla.org/internetcitizen/2018/07/26/this-is-your-digital-fingerprint/">user privacy</a>.</li><li>User profiles (with purchase history, ratings, shopping cart, wishlist, comments, recently browse, etc.). These are usually transactional and structured data stored in most customer databases. A baseline data for <a href="https://buffer.com/library/marketing-personas-beginners-guide"><strong><em>persona creation</em></strong></a><strong><em>, </em></strong><a href="https://mixpanel.com/blog/2018/05/09/user-segmentation-guide-understanding-customers/"><strong><em>segmentation</em></strong></a><strong><em>, </em></strong><a href="https://business.twitter.com/en/resources/holiday-guide/remarketing-tips.html"><strong><em>targeting</em></strong></a><strong><em>, </em></strong><a href="https://blog.hubspot.com/service/customer-retention"><strong><em>retention and reactivation</em></strong></a>. A lot of work is in <a href="https://www.printful.com/blog/the-basics-of-ecommerce-fraud-what-is-it-and-how-to-manage-it/"><strong>user fraud</strong></a> as well.</li><li>Seller profiles (ratings, sales, price range, promotions, partnerships, novelty, catalog, etc.). These are missing in the image above but are essential in any modern RecSys design. Sellers are the platform’s lifeblood and content partner. The data will be critical in product <strong>promotions</strong>, launching <strong>new products</strong>, and<strong> ranking</strong>/placement of products on the site.</li><li>Item attributes (price history, category, sales, quantity in stock, consolidate across multiple-sellers, images, fake product check, etc.). These data are vital to the item similarity models. Product image similarity will be a core part of this series of articles.</li></ul><blockquote>Use data in combination — the power of the data and insights grow exponentially when you combine them. The image outline three key relationships underlying most RecSys: <a href="https://es.coursera.org/lecture/machine-learning-applications-big-data/collaborative-filtering-recsys-user-user-and-item-item-X7tMz">popular item + similar item</a>, customer + item, <a href="https://towardsdatascience.com/a-gentle-introduction-on-market-basket-analysis-association-rules-fa4b986a40ce">market basket analysis</a>.</blockquote><p><strong>Step 3: Design a suitable architecture</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DOzDuX2KgJPHfN9vKw1NXw.png" /><figcaption>Source: <a href="https://netflixtechblog.com/system-architectures-for-personalization-and-recommendation-e081aa94b5d8">System Architectures for Personalization and Recommendation</a> <em>by </em><a href="http://www.twitter.com/xamat"><em>Xavier Amatriain</em></a><em> and </em><a href="http://www.twitter.com/justinBasilico"><em>Justin Basilico</em></a></figcaption></figure><p>The system architecture of RecSys is usually a guarded secret and differs based on the scale and requirements. One good baseline architecture is the one from Netflix (see the link under further readings for more details). Netflix hosts their RecSys on AWS and divides the system into three time horizons:</p><ul><li>Offline (batch processing), these are models that are computationally heavy and takes a long time to run. Models that need to generate <strong>long term relationship pairs across all users and products</strong>, such as collaborative filtering models will fall under this category. Usually, we will run these models at scheduled intervals throughout the day, <strong>every few hours</strong>. These are often the most complex and accurate models.</li><li>Nearline (semi real-time / micro-batch processing). <a href="https://databricks.com/blog/2018/03/20/low-latency-continuous-processing-mode-in-structured-streaming-in-apache-spark-2-3-0.html">Modern frameworks like Spark</a> allows for processing of data <strong>every 5–10s, </strong>and this is great for <strong>modeling short term behavioral patterns, such as browsing patterns</strong>. We can update the recommendations based on what the user is looking at and recommend complementary items for what they added to the cart.</li><li>Online (real-time) models can be costly and unnecessary for most use cases that can be handled in semi real-time. Because real-time processing occurs in the ballpark of <strong>~10ms</strong>, <strong>little modeling</strong> can be completed at this speed. Most likely, we will be <strong>pre-generating</strong> the results (such as the most popular items) and serving results via <strong>an efficient data structure such as </strong><a href="https://redislabs.com/docs/quick-guide-recommendations-using-redis/"><strong>Redis</strong></a><strong> (in-memory database) or cache. </strong>Real time testing in the form of <a href="https://databricks.com/session/fact-store-scale-for-netflix-recommendations">AB testing</a> or <a href="https://towardsdatascience.com/bandits-for-recommender-system-optimization-1d702662346e">multi armed bandits</a> are also critical for the success of any RecSys.</li></ul><blockquote>Designing a good RecSys architecture takes experience, time, and an understanding of stakeholder requirements (metrics, data, budget, time, etc.).</blockquote><blockquote><strong>The key is to start small with offline models, scale towards semi real-time models, and always be testing in real-time.</strong></blockquote><h3>What have we learned</h3><p>It is not easy to design a RecSys. We should always start with the business problem. Do we have a current baseline solution? What are the expectations/goals? How much resources are we willing to put into this project? Next, we should evaluate our data sources. Are we tracking the user interactions? Do we have access to the user and item data? How much historical data do we have to work with? The answer to all these questions will determine the kinds of architecture we design to tackle the requirements.</p><h3>Reflections</h3><ul><li>Imagine your RecSys team consists of business stakeholders, designers, front end &amp; back end engineers, data engineers, and fellow data scientists. What questions will you ask the various team members?</li><li>Below, we have the patent filings from Amazon regarding how they generate shopping cart and instant recommendations. Do the workflows make sense? Are there any clarifications that you will like from the data scientists that designed the workflow? How will you improve the workflow with modern tools/processes?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8cIf-jeIyDOrc3IoLThEUw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MqR3sLg6mD0zUo0PMEDrhw.png" /><figcaption>Source: Patent US 6,266,649 B1: <a href="https://insight.rpxcorp.com/pat/US6266649B1">Collaborative recommendations using item-to-item similarity mappings</a> by Amazon</figcaption></figure><h3>Explore the rest of Modern Visual RecSys Series</h3><ul><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part1-introduction-1241c02f76d6">Modern Visual RecSys: How does a recommender work?</a></li><li><a href="https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521">Modern Visual RecSys: Intro to Visual RecSys</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">Modern Visual RecSys: Convolutional Neural Networks Recommender</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">Modern Visual RecSys: COVID-19 Case Study with CNN</a></li><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a></li><li><a href="https://medium.com/@thiakx/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a></li></ul><p>Series labels:</p><ul><li>Foundational: general knowledge and theories, minimum coding experience needed.</li><li>Core: more challenging materials with code.</li><li>Pro: Difficult materials and code, with production grade tools.</li></ul><h3>Further Readings</h3><ul><li><a href="https://towardsdatascience.com/evaluation-metrics-for-recommender-systems-df56c6611093">Evaluation Metrics for Recommender Systems</a></li><li><a href="https://gab41.lab41.org/recommender-systems-its-not-all-about-the-accuracy-562c7dceeaff#.5exl13wqv">Recommender Systems — It’s Not All About the Accuracy</a></li><li><a href="https://pixelprivacy.com/resources/browser-fingerprinting/">Browser Fingerprinting <em>What Is It and What Should You Do About It?</em></a></li><li>[Video] Slack — <a href="https://www.youtube.com/watch?v=Yz7fKIHfFW8">Building recommender systems with strict privacy boundaries</a></li><li><a href="https://www.confluent.io/blog/using-apache-kafka-drive-cutting-edge-machine-learning/">Using Apache Kafka to Drive Cutting-Edge Machine Learning</a></li><li><a href="https://towardsdatascience.com/a-gentle-introduction-on-market-basket-analysis-association-rules-fa4b986a40ce">A Gentle Introduction on Market Basket Analysis — Association Rules</a></li><li><a href="https://es.coursera.org/lecture/machine-learning-applications-big-data/collaborative-filtering-recsys-user-user-and-item-item-X7tMz">Collaborative Filtering RecSys: User-User and Item-Item</a></li><li><a href="https://netflixtechblog.com/system-architectures-for-personalization-and-recommendation-e081aa94b5d8">Netflix: System Architectures for Personalization and Recommendation</a></li><li><a href="https://databricks.com/blog/2018/03/20/low-latency-continuous-processing-mode-in-structured-streaming-in-apache-spark-2-3-0.html">Introducing Low-latency Continuous Processing Mode in Structured Streaming in Apache Spark 2.3</a></li><li><a href="https://databricks.com/session/fact-store-scale-for-netflix-recommendations">Fact Store (for ML and A/B tests) at Scale for Netflix Recommendations</a></li><li><a href="https://towardsdatascience.com/bandits-for-recommender-system-optimization-1d702662346e">Bandits for Recommender System Optimization</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=30d2352fff34" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34">How to Design a Recommender?</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How does a recommender work?]]></title>
            <link>https://ai.plainenglish.io/modern-visual-recsys-part1-introduction-1241c02f76d6?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1241c02f76d6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Wed, 26 Feb 2020 10:04:25 GMT</pubDate>
            <atom:updated>2020-04-15T06:17:35.669Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h4>Modern RecSys</h4><h4>In this series of articles, I will introduce modern approaches to visual recommender systems. We begin with a case study of Spotify.</h4><p>I have worked in the data industry for over seven years and had the privilege of designing, building, and deploying two recommender systems (RecSys) that went on to serve millions of customers. For each chapter, I will walk through case studies and share my experience designing RecSys.</p><p>This is part of my Modern Visual RecSys series; feel free to check out the rest of the series at the end of the article.</p><h3>RecSys Basics — Spotify Case Study</h3><p>We begin with a case study of Spotify to understand how RecSys works and introduce several key concepts, including a modern approach called convolutional neural networks (CNN), applied to music.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XB2NE83REXveKe1SQTOk9Q.png" /><figcaption>My “discover weekly” recommendations from <a href="https://www.spotify.com/">Spotify</a></figcaption></figure><p>Let us take a look at my personalized music recommendations from Spotify. It contains a mix of Chinese/Japanese/ English pop music with new music and old tunes going back some 20 years. A few observations as I scroll through the recommendations:</p><ul><li>None of the music is from artists that I “save to liked”.</li><li>The <strong>genre</strong> is similar to what I usually listen to.</li><li>The <strong>tunes</strong> are similar to what I usually listen to.</li><li>There is a mix of new and old songs.</li></ul><p>It seems that this recommendation product is trying to help me discover new music that is <strong>familiar yet different</strong> from my usual listening habits. But how is this achieved? Chris Jonson from Spotify has the following slide on the architecture of Discover Weekly:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QMwLFmQapiBQ3b8prBGH0g.png" /><figcaption>Source: slide from presentation: <a href="https://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-spotifys-discover-weekly">From Idea to Execution: Spotify’s Discover Weekly</a> by <a href="https://www.slideshare.net/MrChrisJohnson">Chris Johnson</a></figcaption></figure><p>We see that there are three main methods employed (the red box):</p><ol><li>Collaborative filtering (CF) using user behavior (play logs) and music content (track metadata).</li><li>Natural Language Processing (NLP) and text mining/scraping of news/blogs/text and music content (track metadata).</li><li>Audio models that analyze the raw audio data.</li></ol><blockquote>The result is a “Spotify blob” for each user, with an ever-shifting musical preference based on user interactions and an expanding library of music from Spotify. Visually, the goal of Discover Weekly is to find these white <strong>contour lines </strong>that cut across the user’s musical preferences.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*7eScBDPiUtZ3buoe4CjpRw.png" /><figcaption>Source: <a href="https://qz.com/571007/the-magic-that-makes-spotifys-discover-weekly-playlists-so-damn-good/">The magic that makes Spotify’s Discover Weekly playlists so damn good</a> by <a href="https://qz.com/">Quartz</a> &amp; Spotify</figcaption></figure><p>Let us dive deeper into each of these methods.</p><h3>Collaborative filtering (CF)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/1*ToqwIa_yyXG1rmtipYaRRQ.png" /><figcaption>Source: <a href="https://qz.com/571007/the-magic-that-makes-spotifys-discover-weekly-playlists-so-damn-good/">The magic that makes Spotify’s Discover Weekly playlists so damn good</a> by <a href="https://qz.com/">Quartz</a> &amp; Spotify</figcaption></figure><p>CF is the classic method employed across different RecSys. It simply takes user interactions (your clicks, saves, likes, purchases, etc.) and matches them with other users in the system with similar tastes (in music, films, fashion, etc.).</p><blockquote>CF assumes that <strong>users with similar tastes will appreciate content from others within the same community</strong>.</blockquote><p>There are drawbacks such as:</p><ul><li>Echo chambers (Facebook showing you left/right-wing posts over and over again based on your reading behavior)</li><li>Safe but boring recommendations (recommending another Artist A song when you know that I am a big fan of Artist A)</li><li>Cold start problem where CF cannot match new items/users due to a lack of data — CF will always need to be paired with a backup plan in deployment (top/most popular products for example)</li></ul><p>There are various implementations of CF. For example, <a href="https://spark.apache.org/docs/2.2.0/ml-collaborative-filtering.html">Spark’s alternating least squares (ALS)</a>, <a href="https://docs.fast.ai/collab.html">FastAI’s collab</a>, <a href="http://surpriselib.com/">Surprise</a> (for explicit/user rating data). Check out the further readings section for more tutorials.</p><h3>Natural Language Processing (NLP)</h3><p>One way to handle the cold start problem, especially for new releases, is to scrape the internet news/blogs and fill in metadata information about the song (artist, title, mood {happy, love…}, genre {pop, Korean}, etc.) with web scrapers like <a href="https://realpython.com/beautiful-soup-web-scraper-python/">Beautiful Soup</a>, <a href="https://scrapy.org/">Scrapy</a>, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XjQFPhBBi0SVItzLwubDew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LaqL0Ld_theITJqx3LQ6qg.png" /><figcaption>Source: slide from presentation: <a href="https://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-spotifys-discover-weekly">From Idea to Execution: Spotify’s Discover Weekly</a> by <a href="https://www.slideshare.net/MrChrisJohnson">Chris Johnson</a></figcaption></figure><p>With the scrapped textual data, together with details from the playlist, it is possible to associate keywords with individual artists/playlists.</p><blockquote>Modern approaches make use of <strong>word embeddings</strong> to construct sentence/document vectors; mathematical representations that allow for comparison across the vector space.</blockquote><p>Common techniques are word2vec, doc2vec, and Latent Dirichlet Allocation (LDA). Vectorization is key to the <a href="https://www.techinasia.com/introducing-tech-asias-unique-content-recommender">content-based recommender my team built at Tech in Asia</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*0r4tTTouOfx8Qv-11moFQw.png" /><figcaption>Source: <a href="https://www.techinasia.com/introducing-tech-asias-unique-content-recommender">Introducing Tech in Asia’s unique content recommender</a> by By <a href="https://www.linkedin.com/in/ikaruswill/">Will Ho</a> &amp; <a href="https://www.linkedin.com/in/joshua-rallen-lim-49813129/">Joshua Lim</a></figcaption></figure><h3>Audio Models</h3><p>Sander Dieleman (Research Scientist at DeepMind) once interned at Spotify and wrote a great article on <a href="https://benanne.github.io/2014/08/05/spotify-cnns.html#">Recommending music on Spotify with deep learning</a>. He used a technique called convolutional neural networks (CNN) that we will cover in the later chapters. Intuitively, our goal is for each filter (shown as columns in the image below) picks up a distinct musical feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CiV6bvg33VhsRgIj2JEiag.png" /><figcaption>Visualization of the filters learned in the first convolutional layer. The time axis is horizontal, and the frequency axis is vertical. Source: <a href="https://benanne.github.io/2014/08/05/spotify-cnns.html#">Recommending music on Spotify slides</a> by <a href="https://benanne.github.io/about/">Sander Dieleman</a></figcaption></figure><p>If we zoom in to take a look at the specific filters, we can pick up trends as noted by Sander:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*A4TaW25oyTTmzRDQYHn-GQ.png" /><figcaption>Closeup of filters 14, 242, 250 and 253. Source: <a href="https://benanne.github.io/2014/08/05/spotify-cnns.html#">Recommending music on Spotify slides</a> by <a href="https://benanne.github.io/about/">Sander Dieleman</a></figcaption></figure><blockquote>Note that the time axis is horizontal, the frequency axis is vertical (Frequency increases from top to bottom). Negative values are red, positive values are blue, and white is zero.</blockquote><blockquote>Filter 14 seems to pick up <strong>vibrato singing</strong>. [Notice the recurring blue shades for column 14 across different frequencies]</blockquote><blockquote>Filter 242 picks up some kind of <strong>ringing ambience</strong>. [Notice the blue stripe +red base]</blockquote><blockquote>Filter 250 picks up <strong>vocal thirds</strong>, i.e., multiple singers singing the same thing, but the notes are a major third (4 semitones) apart. [Notice the neat recurring alternation between red and blue rows]</blockquote><blockquote>Filter 253 picks up various types of <strong>bass drum sounds</strong>. [Notice that most of the music exists within a small range of frequencies at the top].</blockquote><blockquote>These musical patterns act as a <strong>musical signature</strong>, allowing Spotify to mix and match songs of similar signatures to generate playlists <strong>that sounds familiar but with degrees of controlled novelty</strong> for the user.</blockquote><p>The data scientists can always dial up or dial down the novelty mix to the musical signature based on user response to the recommendations. Such is the power of modern tools like CNN.</p><h3>What have we learned</h3><p>RecSys are very interesting models to explore. Even seemingly simple music playlist recommendation can involve a diverse array of models that brings together the user interactions, content, external data, and domain-specific techniques such as audio models in this Spotify case study.</p><p>In the next chapter, we will learn how to design a recommender.</p><h3>Reflections</h3><p>Take a look at your recommendations on Spotify (or Amazon/ Netflix/ YouTube/ any other services you used with personalization).</p><ul><li>Are they relevant to you? What % of the recommendations are spot on? What % is terrible?</li><li>How will you improve the recommendations?</li><li>Will you put more weight on recent behaviors vs. historical?</li><li>How will you introduce new products?</li><li>What will you show new users?</li><li>How will you design a recommender that keeps up with the latest trends?</li></ul><h3>Explore the rest of Modern Visual RecSys Series</h3><ul><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part2-the-recsys-design-framework-30d2352fff34">Modern Visual RecSys: How to Design a Recommender?</a></li><li><a href="https://medium.com/analytics-vidhya/intro-to-visual-recsys-12d54976c521">Modern Visual RecSys: Intro to Visual RecSys</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4-convolutional-neural-networks-recommender-c4174644ab0d">Modern Visual RecSys: Convolutional Neural Networks Recommender</a></li><li><a href="https://medium.com/@thiakx/modern-visual-recsys-part4b-covid-19-case-study-with-cnn-5f07fd93a11a">Modern Visual RecSys: COVID-19 Case Study with CNN</a></li><li><a href="https://towardsdatascience.com/building-a-personalized-real-time-fashion-collection-recommender-22dc90c150cb">Building a Personalized Real-Time Fashion Collection Recommender</a></li><li><a href="https://towardsdatascience.com/temporal-fashion-recommender-59c26313fa25">Temporal Fashion Recommender</a></li><li><a href="https://medium.com/@thiakx/the-future-of-visual-recommender-systems-four-practical-state-of-the-art-techniques-bae9f3e4c27f">The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques</a></li></ul><p>Series labels:</p><ul><li>Foundational: general knowledge and theories, minimum coding experience needed.</li><li>Core: more challenging materials with code.</li><li>Pro: Difficult materials and code, with production grade tools.</li></ul><h3>Further Readings</h3><ul><li><a href="https://benanne.github.io/2014/08/05/spotify-cnns.html">Recommending music on Spotify with deep learning</a></li><li><a href="https://medium.com/s/story/spotifys-discover-weekly-how-machine-learning-finds-your-new-music-19a41ab76efe">How Does Spotify Know You So Well?</a></li><li><a href="https://www.techinasia.com/introducing-tech-asias-unique-content-recommender">Introducing Tech in Asia’s unique content recommender</a></li><li><a href="https://medium.com/recombee-blog/machine-learning-for-recommender-systems-part-1-algorithms-evaluation-and-cold-start-6f696683d0ed">Machine Learning for Recommender systems by Pavel Kordík</a> | <a href="https://medium.com/recombee-blog/machine-learning-for-recommender-systems-part-2-deep-recommendation-sequence-prediction-automl-f134bc79d66b">Part2</a></li><li><a href="https://medium.com/hackernoon/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75">Introduction to RecSys by Kung-Hsiang, Huang (Steeve)</a> | <a href="https://towardsdatascience.com/introduction-to-recommender-system-part-2-adoption-of-neural-network-831972c4cbf7">Part2</a></li><li><a href="https://towardsdatascience.com/introduction-to-recommender-systems-6c66cf15ada">Introduction to recommender systems by Baptiste Rocca</a></li><li><a href="https://medium.com/@jonathan_hui/machine-learning-recommender-system-e3237b9df14a">Machine Learning — Recommender System by Jonathan Hui</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1241c02f76d6" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/modern-visual-recsys-part1-introduction-1241c02f76d6">How does a recommender work?</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create differentiating products in the face of AI democratization]]></title>
            <link>https://medium.com/@thiakx/how-to-create-differentiating-products-in-the-face-of-ai-democratization-e34d77a9048a?source=rss-38556f69915d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e34d77a9048a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Kai Xin Thia]]></dc:creator>
            <pubDate>Wed, 30 May 2018 03:39:20 GMT</pubDate>
            <atom:updated>2018-05-30T03:39:36.663Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AyAfi-miXu3_vvHn0vU62A.jpeg" /></figure><p>I have the privilege of hosting <a href="https://www.linkedin.com/in/avi-patchava-b49a012/">Avi</a> (VP — Data Sciences, AI &amp; ML, InMobi), <a href="https://www.linkedin.com/in/hongtingwong/">Hong Ting</a> (CEO and Cofounder, Botbot.AI) and <a href="https://www.linkedin.com/in/kenny-chua-04ab1bb/">Kenny</a> (Head of R&amp;D Singapore, DataRobot) during the recent <a href="https://www.techinasia.com/events/singapore">Tech in Asia Singapore 2018</a> conference. Here are the key takeaways.</p><p>Q: What is AI and does the definition matter?</p><ul><li>From computer science angle, what we deem as “AI” in recent news is just Narrow AI — machines that can do a single task well like <a href="https://www.smartnation.sg/initiatives/Mobility/self-driving-vehicles-sdvs--future-of-mobility-in-singapore">self-driving cars</a>, <a href="https://www.telegraph.co.uk/science/2017/10/18/alphago-zero-google-deepmind-supercomputer-learns-3000-years/">AlphaGo</a>, <a href="https://www.cnet.com/how-to/everything-you-need-to-know-about-google-home/">Google Home</a> and so on. We are still very far away from General AI — the likes of J.A.R.V.I.S. from Ironman movies that can think, communicate and act in a wide range of activities, just like an average person. <a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">Artificial Superintelligence</a> — AI that goes beyond the limits of human thoughts is just a <a href="https://www.technologyreview.com/s/602410/no-the-experts-dont-think-superintelligent-ai-is-a-threat-to-humanity/">pipe dream at the moment</a>. Ultimately, the definition might not matter as much as the fantastic technologies we build in pursuit of next level developments in AI.</li></ul><p>Q: Are AI tools only for the big companies with big budgets? Why should startups or SMEs care about AI?</p><ul><li><a href="https://botbot.ai/appearances/tiasg2018-pr/">As noted by Hong Ting</a>, “The advantage that we have, as SMEs and start-ups, is that we can zoom in on specific areas and get really, really good at it. We can also use all the tools that the big players have built and add on the datasets to create something unique for us”. <a href="https://www.techinasia.com/empath-wins-tech-asia-singapore-2018-arena">Staying focus</a>, <a href="https://www.oreilly.com/ideas/data-science-startups-focus-on-ai-enabled-efficiency">moving fast</a>, making use of <a href="https://github.com/awesomedata/awesome-public-datasets">public open data</a> and <a href="https://medium.com/swlh/ai-in-2018-for-developers-2f01250d17c">open source tools</a> are an essential competitive advantage of smaller firms.</li><li>It is <a href="https://www.thoughtspot.com/thoughtspot-blog/no-luck-recruiting-ai-talent-youre-not-doing-it-right">difficult to hire good AI talents</a>, especially for startups who might not <a href="https://www.cio.com/article/3252338/artificial-intelligence/winning-the-war-for-ai-talent.html">be able to compete</a> on pay alone. A definite <a href="https://nurture.ai/">purpose</a> of the company and <a href="https://www.inc.com/michelle-cheng/how-startups-are-grappling-with-artificial-intelligence-talent-hiring-frenzy.html">showing up at a hospital with flowers and balloons</a> for your potential AI engineer might be edge needed to secure the talent you need.</li></ul><p>Q: Will self-service AI tools like DataRobot, Google Cloud AI, AWS ML replace the need for data scientists?</p><ul><li><a href="http://customerthink.com/if-the-computer-is-a-bicycle-for-our-minds-artificial-intelligence-is-a-harley-davidson/">Computers are like a bicycle for the mind</a> — they allow us to go further and do more with our lives. Photoshop did not kill the photography industry; it created new possibilities in image editing. Similarly, self-service AI tools will help automate a lot of the<a href="https://www.infoworld.com/article/3234465/data-science/even-data-scientists-are-facing-ai-automation.html"> repetitive, structured expects of machine learning</a>, freeing up the human mind to focus on analysis, design and productionize their domain knowledge. By using standardized, well-tested methodologies, it also lowers the risk of development and makes brings the <a href="https://www.technologyreview.com/s/608921/you-could-become-an-ai-master-before-you-know-it-heres-how/">powerful tools to the public</a>. It is far more important to push for <a href="https://www.oreilly.com/ideas/the-state-of-ai-adoption">AI adoption in companies</a> at this stage.</li></ul><p>Q: Lots of companies are going with the mobile-first strategy, what are some of the unique opportunities and challenges that mobile offers to building AI products?</p><ul><li>Asia, especially Southeast Asia, will witness an entire generation of people <a href="https://www.techinasia.com/talk/full-guide-southeast-asia-digital-landscape-2017">coming online via mobile devices </a>within the next few years. Even mobile device makers are <a href="https://www.weforum.org/agenda/2018/02/ai-smartphones-are-coming">producing AI chips</a>. <a href="https://clearbridgemobile.com/artificial-intelligence-driving-future-mobile-app-personalization/#Alexa_Pushing_AI_to_Mobile">Mobile-first</a> provides companies with huge opportunities for <a href="https://www.forbes.com/sites/briansolis/2017/11/30/extreme-personalization-is-the-new-personalization-how-to-use-ai-to-personalize-consumer-engagement/#171c151b829a">personalization</a> and<a href="https://dzone.com/articles/your-users-crave-instant-gratification-so-how-do-y-1"> instant gratification</a>, though <a href="https://www.theverge.com/2018/5/25/17393766/facebook-google-gdpr-lawsuit-max-schrems-europe">privacy is a growing concern</a>.</li><li>The biggest challenge in developing AI products is usually in scoping the problem and gathering the right data. <a href="https://www.nimdzi.com/ai-meets-localization/">Language localization</a> is a huge challenge in Southeast Asia due to the large number of spoken languages. <a href="https://www.techinasia.com/talk/artificial-intelligence-holds-promise-southeast-asia">Building an AI startup in Southeast Asia </a>that can handle the mix of English, Chinese, Bahasa, Thai, Vietnamese, Tagalog and Singlish is both science and art.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e34d77a9048a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>