<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kung-Hsiang, Huang (Steeve) on Medium]]></title>
        <description><![CDATA[Stories by Kung-Hsiang, Huang (Steeve) on Medium]]></description>
        <link>https://medium.com/@huangkh19951228?source=rss-2fc7b9c3f02a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*D36_E45qZHNBD7TvFgiutQ.jpeg</url>
            <title>Stories by Kung-Hsiang, Huang (Steeve) on Medium</title>
            <link>https://medium.com/@huangkh19951228?source=rss-2fc7b9c3f02a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 16:50:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@huangkh19951228/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hesitant Customers and Where to Find Them  —  Reduce Shopping Cart Abandonment with Hesitant…]]></title>
            <link>https://medium.com/rosetta-ai/hesitant-customers-and-where-to-find-them-reduce-shopping-cart-abandonment-with-hesitant-279e41f7ef7e?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/279e41f7ef7e</guid>
            <category><![CDATA[e-commerce-business]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ecommerce-solution]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Mon, 21 Sep 2020 06:01:35 GMT</pubDate>
            <atom:updated>2020-09-24T03:10:53.068Z</atom:updated>
            <content:encoded><![CDATA[<h3>Hesitant Customers and Where to Find Them — Reduce Shopping Cart Abandonment with Hesitant Customer Detection</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*9RdwJtU6Z1zL1Roj" /><figcaption>Image from Unsplash (<a href="https://images.unsplash.com/photo-1563013544-824ae1b704d3?ixlib=rb-1.2.1&amp;ixid=eyJhcHBfaWQiOjEyMDd9&amp;auto=format&amp;fit=crop&amp;w=1350&amp;q=80">https://images.unsplash.com/photo-1563013544-824ae1b704d3?ixlib=rb-1.2.1&amp;ixid=eyJhcHBfaWQiOjEyMDd9&amp;auto=format&amp;fit=crop&amp;w=1350&amp;q=80</a>)</figcaption></figure><p>Shopping cart abandonment has long been an issue for E-Commerce. According to a <a href="https://99firms.com/blog/shopping-cart-abandonment-stats/">survey</a>, 75% of online shoppers tend to add things in their shopping cart and never check out the cart. To address this challenge, one of the solution is to identify hesitant customers and provide them deals or coupons that encourage them to checkout their cart. In this article, we focus on the data and the modeling perspectives of this challenge. We will discuss how this challenge can be formulated as Machine Learning problems, and talk about data collection and the modeling components.</p><h3>Task Definition</h3><p>What is hesitant customer? Under the scope of our problem, hesitant customers are those <strong>having a hard time deciding</strong> <em>which items to buy</em> or <em>whether they should buy</em> <em>or not</em>. When given <strong>correct incentives</strong>, they will be able finalize buying decision and willing to checkout their carts. To narrow down the problem for easier modeling, we define hesitant customers as:</p><blockquote>Customers who <strong>hit a popped up deal </strong>AND eventually <strong>lead to transaction with the deal</strong>.</blockquote><p>The bigger problem of reducing shopping cart abandonment rate can be boiled down to two tasks:</p><ol><li>Hesitant Behavior Identification</li><li>Deals Matching</li></ol><h4>Hesitant Behavior Identification</h4><p>The first task is to recognize hesitant behavior given a sequence of interactions between end users and browsers. These interactions include actions that typical recommender systems rely on, such as clicking on items, viewing items, and selecting items to cart. More advanced actions that are crucial for our purpose, like switching between tabs, should also be taken into account. The task is formalized as a binary classification, where each sequence of interactions are associated with a binary label that indicate whether the action sequence is considered hesitant.</p><h4>Deals Matching</h4><p>Each identified hesitant customers is matched with a deal that are most relevant to their interest. These deals are composed of images and descriptive information about the deals. Intuitively, this task can be viewed as a ranking problem, where the most relevant deals should be given the highest score. Yet, in some situations, pair-wise ranking losses that aim to ensure positive samples are scored higher than the negative samples could be hard to trained. A proxy loss function, such as binary cross entropy, can help ease the training process.</p><h3>Evaluation Metrics</h3><h4>Hesitant Behavior Identification</h4><p>Due to the high imbalance in the collected data (significant amount of negative data), the most commonly used metric, accuracy, is not suitable for our current task. Alternatively, these are some of the metrics that are reasonable for this task:</p><ol><li><strong>Area under</strong> <strong>the ROC Curve</strong>.<strong> (AUC)</strong>: AUC only cares whether you rank rather than the actual predicted score of it. An AUC score at 0.5 indicates the performance is as good as random guess.</li><li><a href="https://en.wikipedia.org/wiki/F1_score"><strong>F-1 Score</strong></a>: F-1 takes into account both recall and precision, which makes it also a great candidate for unbalanced data.</li></ol><h4>Deals Matching</h4><p>As a task related information retrieval, rank-based metrics are suitable for this task. Below we list a few of them:</p><ol><li><strong>Hit@k</strong>: Hit@k computes the number of hits from the top k highest scored/ ranked items for each instance and divided by the total number of instances.</li><li><strong>Mean Reciprocal Rank (MRR)</strong>: MRR can be viewed as an extension of Hit@k, where the quality of the “hit” is considered. Intuitively, the hit at the higher rank, should be given more credits than hit at the lower rank. MRR assigns a score for each hit 1/(2^<em>h</em>), where <em>h</em> is the rank of the hit item.</li></ol><h4>Business Side Consideration</h4><p>From the business perspective, we tend to look at conversion rate and revenue with A/B testing when comparing new and old algorithms . It’s always a good idea to collaborate closely with the business department and make sure that the new technology helps the company work towards the long-term goals.</p><h3>Data Collection</h3><p>In the beginning, we do not have annotated data both tasks. Adapting reinforcement learning may be a solution, but the effect (successful identification of hesitant customers) should be much worse compared with expert-defined rules. Thus, we collaborated with E-Commerce experts to devise a set of rules for determining when to provide deals. With these rules, our front-end system would pop up deals on the web page of our customers if any of the rules are triggered. We are be able to collect data about whether a customer click a deal and later on checkout their cart with the deal. After we have collected enough data (~10K positive click-deal interaction), we started switching to a ML-based identification model, and continue to collect the same data.</p><p>Intuitively, we want to model continuous behavior of end users. Yet, tracking too granular interactions would result in too noisy data that can harm the performance of models. We discretized and carefully selected a salient set of additional data to collect, including:</p><ol><li>The amount of time each customer spent on each web page.</li><li>The timestamp of customers switching between tabs.</li><li>The interactions between customers and the web page (click, select to cart, checkout, and etc).</li><li>The items that each customer viewed in each session.</li></ol><h3>Modeling</h3><h4>Hesitant Behavior Identification</h4><p>As we discussed earlier, at the beginning of the data collection process, we utilize expert knowledge to identify customers who might be interested in clicking pop-up deals. In the next stage, we model it as a temporal prediction problem with user behavior data in the recent sessions taken into account. Sequential models, such as RNN and attention-based models, are naturally strong baselines for this task.</p><h4>Deals Matching</h4><p>Although in the early stage there is not much training data for finding the best deal to pop up, we develop content-based model that largely leverages content data (image, deal information, and etc) for retrieving the most relevant deals for each customer. Later on, we gradually refine our model towards a hybrid fashion, which balances between content data and collected user behavioral data.</p><h3>Conclusion</h3><p>This blog post has highlighted the importance of hesitant customer detection and how we boil it down to two sub-tasks. We talked how our model is evaluated, how the data collection is done, and the modeling component . In the next blog post of this series, I plan to release the evaluation results and some analysis. Till next time :)</p><p>For more information about Rosetta.ai, please visit our <a href="https://www.rosetta.ai/">website</a>, or our <a href="https://www.facebook.com/RosettaAI">FB fan page</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=279e41f7ef7e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/hesitant-customers-and-where-to-find-them-reduce-shopping-cart-abandonment-with-hesitant-279e41f7ef7e">Hesitant Customers and Where to Find Them  —  Reduce Shopping Cart Abandonment with Hesitant…</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learn Hugging Face Transformers & BERT with PyTorch in 5 Minutes]]></title>
            <link>https://medium.com/rosetta-ai/learn-hugging-face-transformers-bert-with-pytorch-in-5-minutes-acee1e3be63d?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/acee1e3be63d</guid>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Mon, 16 Mar 2020 18:03:24 GMT</pubDate>
            <atom:updated>2020-03-16T18:03:24.239Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z4mGaMsu34LfyE76QAi9qA.png" /><figcaption><a href="https://huggingface.co/transformers/">https://huggingface.co/transformers/</a></figcaption></figure><p><strong>Bidirectional Encoder Representations from Transformers (BERT)</strong> marked a new era for Natural Language Processing last year. By the time the paper was published, it achieved the <strong>state-of-the-art results over 11 Natural Language Understanding</strong> tasks. It proved the capabilities of a Language Model properly trained on huge corpus to largely improve downstream tasks.</p><p>The <strong>Transformers</strong> package developed by <strong>HuggingFace</strong> unifies the implementation of different BERT-based models. It provides an easy-to-use interface and a wide variety of BERT-based models as shown in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VfDscAz6Nd1uRrOxijVL2w.png" /><figcaption><a href="https://github.com/huggingface/transformers">The various BERT-based models supported by HuggingFace Transformers package</a></figcaption></figure><p>This blog post will use BERT as an example. The usage of the other models are more or less the same. This rest of the article will be split into three parts, tokenizer, directly using BERT and fine-tuning BERT. You will learn how to implement BERT-based models in 5 minutes.</p><h3>Tokenizer</h3><p>Most of the BERT-based models use similar with little variations. For instance, BERT use ‘[CLS]’ as the starting token, and ‘[SEP]’ to denote the end of sentence, while RoBERTa use &lt;s&gt; and &lt;/s&gt; to enclose the entire sentence.</p><p>In the transformers package, we only need three lines of code to do to tokenize a sentence.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/29dffa0c7b4af6f65d6f603fbae981d9/href">https://medium.com/media/29dffa0c7b4af6f65d6f603fbae981d9/href</a></iframe><p>The <em>tokens</em> variable should contain a list of tokens:</p><pre>[&#39;[CLS]&#39;, &#39;learn&#39;, &#39;hugging&#39;, &#39;face&#39;, &#39;transformers&#39;, &#39;&amp;&#39;, &#39;bert&#39;, &#39;with&#39;, &#39;p&#39;, &#39;##yt&#39;, &#39;##or&#39;, &#39;##ch&#39;, &#39;in&#39;, &#39;5&#39;, &#39;minutes&#39;, &#39;[SEP]&#39;]</pre><p>Then, we can simply call</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a89132cdca79d48945d5898dc69a216c/href">https://medium.com/media/a89132cdca79d48945d5898dc69a216c/href</a></iframe><p>to convert these tokens to integers that represent the sequence of ids in the vocabulary.</p><pre>[101, 4553, 17662, 2227, 19081, 1004, 14324, 2007, 1052, 22123, 2953, 2818, 1999, 1019, 2781, 102]</pre><h4>How to map the original word position to tokenized position?</h4><p>In some application, such as named entity recognition or event extraction, we need to know the tokenized position for an original word. Since the tokenizer may break a single word into several sub-words, you may wonder how do we know we know the tokenized position of a given original word?</p><p><a href="https://github.com/huggingface/transformers/blob/cc4c37952a961f2d13e83f3d5ba6dab811d0bbfd/src/transformers/data/processors/squad.py">Following the Hugging Face’s practice</a>, we basically loop over each word in the sentence and create a mapping from original word position to the tokenized position.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/915fc2bf44db24ed5868697565472eee/href">https://medium.com/media/915fc2bf44db24ed5868697565472eee/href</a></iframe><p>The mapping is stored in the variable <em>orig_to_tok_index</em> where the element <em>e</em> at position <em>i </em>corresponds to the mapping (<em>i</em>, <em>e</em>).</p><h3>Directly Using Pre-Trained BERT</h3><p>Getting the pre-trained BERT is straightforward. Thanks to this transformers package. We only need one line of code to do so.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd6b09a1db0a38283736c586b8bb8391/href">https://medium.com/media/bd6b09a1db0a38283736c586b8bb8391/href</a></iframe><p>This line of code will automatically fetch the pre-trained weights of a PyTorch BERT and download it to a cache directory for future use.</p><h3>Fine-tuning Pre-Trained BERT</h3><p>To fine-tune BERT, the suggested way is to inherit the class <em>BertPreTrainedModel</em>. When we call <em>from_pretrained</em> on our custom classifier class, it will automatically pass in an extra config variable to the model. As shown in the following.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e98f8149426781be786a9ebb350048fa/href">https://medium.com/media/e98f8149426781be786a9ebb350048fa/href</a></iframe><p>This way, the BERT will be trained jointly with the Linear layer.</p><h3>Conclusion</h3><p>In this article, you have learned the three most common usages of the transformers package, tokenization, directly using BERT, and fine-tuning BERT. In the next article, I will demonstrate how I use BERT-based model on one of my current research projects, event extraction in biomedical domains. Stay tuned 🤗</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=acee1e3be63d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/learn-hugging-face-transformers-bert-with-pytorch-in-5-minutes-acee1e3be63d">Learn Hugging Face Transformers &amp; BERT with PyTorch in 5 Minutes</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[深入探討 Latent Dirichlet Allocation (LDA) 與在推薦系統上的應用]]></title>
            <link>https://medium.com/rosetta-ai/%E6%B7%B1%E5%85%A5%E6%8E%A2%E8%A8%8E-latent-dirichlet-allocation-lda-%E8%88%87%E5%9C%A8%E6%8E%A8%E8%96%A6%E7%B3%BB%E7%B5%B1%E4%B8%8A%E7%9A%84%E6%87%89%E7%94%A8-2441d57ecc8a?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2441d57ecc8a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Wed, 12 Feb 2020 00:59:01 GMT</pubDate>
            <atom:updated>2020-02-12T08:15:50.399Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*N8TDydMy6vEeq0dF" /><figcaption>Image from Unsplash (<a href="https://unsplash.com/photos/5mZ_M06Fc9g">https://unsplash.com/photos/5mZ_M06Fc9g</a>)</figcaption></figure><p>此篇文翻譯自筆者英文文章：<a href="https://medium.com/p/e2e8ea5e661c?source=post_stats_page---------------------------">A Deep Dive into Latent Dirichlet Allocation (LDA) and Its Applications on Recommender System</a></p><p><strong>Latent Dirichelet Allocation（LDA）</strong>是一種主題建模演算法（topic modeling），廣泛使用於以無監督學習，探索語料庫中的隱含主題（latent topic）。它已被廣泛應用於各個領域，尤其是在自然語言處理和推薦系統中。這篇文章將帶你探討LDA，從概論到詳細技術說明，最後我們將討論LDA在推薦器系統上的應用！</p><h3>概論</h3><p>LDA是語料/文檔的生成概率模型。它基於“詞袋” （bag-of-words）的假設，就是單詞和文檔都存在可替換性。也就是說，我們並不考慮文檔中的單詞順序或文檔的順序。</p><blockquote>每個文檔由不同含量的主題所組成，而每個主題則是以不同字詞的分佈來表示。</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tzA508AIEIZdwDzx.png" /><figcaption>每個文檔由不同含量的主題所組成 (<a href="https://www.youtube.com/watch?v=DWJYZq_fQ2A">https://www.youtube.com/watch?v=DWJYZq_fQ2A</a>)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/0*1MV5GjeLQwEaFNvn.png" /><figcaption>每個主題則是以不同字詞的分佈來表示。 (<a href="https://www.youtube.com/watch?v=DWJYZq_fQ2A">https://www.youtube.com/watch?v=DWJYZq_fQ2A</a>)</figcaption></figure><p>在LDA的假設中，每個文檔中的主題進行採樣，然後對每個採樣主題中的單詞進行採樣來生成每個文檔。為了得到字詞和主題的正確機率分佈，我們可以使用 Gibbs Sampling, Maximum a Posteriori (MAP)或 Expectation Maximization (EM)訓練LDA。</p><h3>板符號（Plate Notation）</h3><p>為了更深入，讓我們談談LDA的板符號（Plate Notation）。在貝氏推論（Bayesian inference）中，Plate Notation是一種圖形表示表達對隨機變數採樣的重複過程的方式。每個板塊都可以看作是一個“迴圈”，其中，如板塊右下角所表示的符號代表該迴圈的次數。下圖表示LDA的板符號。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/0*tmy3mgMDWiYhNS23.png" /></figure><p>上圖可以分為兩個部分，上面的圖板指出，有K個主題，其字詞的狄利克雷分布 (Dirichlet distribution) 在單詞上由超參數β控制。同樣，下面的圖板描述了有M個文檔，每個文檔包含N個字詞。灰色的圓圈w代表觀察到的單詞，而白色圓圈代表不同的潛在變數。 z表示字詞w對應的主題，θ是主題在文檔上的狄利克雷分布，而θ被超參數⍺所控制。</p><h3>生成過程 （Generative Process）</h3><p>我們現在大致了解瞭了如何通過標牌符號生成文檔，讓我們用數學式子來表達它。<br>1. 對於i從1到M， 從狄利克雷分布抽取樣本θ <em>θ_i〜Dir(⍺)</em><br>2. 對於k從1到K，從另一個抽取樣本φ狄利克雷分布<em>φ_k</em>〜<em>Dir(β)</em><br>3. 對於i從1到M以及j從1到N，從多項式分佈中抽取樣本<em>z_ij</em>〜<em>Multinomial(θ_i)</em>和樣本<em>w_ij</em>〜<em>Multinomial(φ_z_ij)</em><br>以《紐約時報》為例。首先，對於每個新聞文章，我們對整個文檔的主題分佈θ_i以及每個主題上的單詞分佈<em>φ_k</em>進行抽樣。然後，對於文檔中的每個字詞j，我們從給定的主題分佈<em>Multinomial（θ_i）</em>抽樣主題<em>z_ij</em>，接著，在對單詞分佈<em>Multinomial（φ_z_ij）</em>對字詞進行抽樣得到<em>w_ij</em>。下圖顯示了這個過程。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oTp5-bKC_z0fPd6t.png" /></figure><h3>狄利克雷分布</h3><p>我們剛剛都在把狄利克雷分布當作黑匣子，而沒有說明他是怎麼樣的一個函數。讓我們簡要地簡要介紹一下狄利克雷分布。 k維狄利克雷分布是由k維參數向量Alpha控制。下面我們顯示狄利克雷分布的3維范例。基本上alpha越大，分佈越趨向中心。這種分佈讓模型能夠有非常大的自由度決定詞/主題相對於主題/文檔相關的比例，因為某些主題/文檔可能與許多組詞/主題相關聯，而其他主題/文檔可能只和少部分詞/主題有關係。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/0*Bu7h5VxNSRYpcfsY.png" /></figure><h3>訓練模型</h3><p>學習LDA模型的問題我們稱之為“推理”問題。也就是說，給定觀察變數w以及超參數⍺和β，我們如何估計潛在變數的後驗機率（posterior probability）。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A7HfMuj0esxO9TDW.png" /></figure><p>然而分母的部分因為計算他所需要的積分並沒有閉合解（closed form solution），簡單來說就是太複雜無法計算，如以下所示：</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gJd9ediX5yr-cCRY.png" /></figure><p>因此我們必須用估計的方式得到此解，而吉布斯抽樣（gibbs sampling）是最常用的方法之一。</p><h3>吉布斯抽樣</h3><p>使用吉布斯抽樣，我們可以避免直接計算難解積分。簡單來說，我們想對<em>p(w|⍺,β)</em>進行抽樣來估計分佈，但是我們不能直接這樣做。所以利用吉布斯抽樣，在每一次循環迭代中，我們可以分別對每一個潛在變數抽樣，同時固定所有其他變數。這樣，我們可以得到後驗分佈<em>p(θ, z, φ|w, ⍺, β)</em>。 每次迭代中，我們對<em>θ, z </em>和<em>φ</em>進行抽樣，並且固定為所有其他變量，如以下所示：</p><p>對於i從1到<em>MaxIter </em>（最大迴圈數）：</p><ol><li>抽樣<em>θ_{i}〜p（θ| z = z_ {i-1}, φ=φ_{i-1} w, ⍺, β）</em></li><li>抽樣<em>z_ {i}〜p（z |θ=θ_{i}, φ=φ_{i-1} w, ⍺, β）</em></li><li>抽樣<em>φ_{i}〜p（φ|θ=θ_{i}, z = z_ {i} w, ⍺, β）</em></li></ol><p>由於早期迭代中的樣本分佈比較不穩定，因此我們捨棄前面幾輪的抽樣，這些被捨棄的樣本我們稱之為「burn-in」。</p><h3>LDA on Recommender System</h3><p>LDA 常用在推薦系統的兩個子領域：</p><ol><li>協同過濾 (collaborative filtering)</li><li>基於內容推薦（content-based recommender system）</li></ol><h3>協同過濾</h3><p>LDA應用在基於物品的協同過濾 (item-based CF)時, <strong>物品和使用者</strong> 可以被比擬為前面所述之 <strong>文檔和字詞</strong> (基於使用者的協同過濾則會相反). 也就是說，每個物品對應到一個使用者群組的分佈，而每個使用者群組則由不同比例的使用者所組成。利用LDA，我們可以找出使用者和物品的潛在關係。</p><h3>基於內容推薦</h3><p>第二個LDA在推薦系統上的應用是基於內容推薦，而這個應用還蠻直觀的。有別於以往直接使用TF-IDF所得到的個物品的表徵，我們可以利用LDA來抽取潛在物品特徵向量。底下我們示範一段如何用文檔訓練LDA以及如何利用LDA預測。</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b4b58c7fc9ea39800aa80c5802f6c41e/href">https://medium.com/media/b4b58c7fc9ea39800aa80c5802f6c41e/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de9b1cdfe10b2935d54643128aef6a15/href">https://medium.com/media/de9b1cdfe10b2935d54643128aef6a15/href</a></iframe><h3>總結</h3><p>在這篇文章中，我們從討論了LDA的概論以及LDA背後數學的細節，我們也討論了LDA如何應用在推薦系統中，並且示範了一些程式碼該如何使用它，我希望這篇文章是對你有幫助的，我們下次見：）</p><h3>引用</h3><ol><li>Blei, D. M., Ng, A. Y., &amp; Jordan, M. I. (2003). Latent dirichlet allocation. <em>Journal of machine Learning research</em>, <em>3</em>(Jan), 993–1022.</li><li><a href="https://docs.google.com/presentation/d/17vZ5kWGS8beROHHEKvXIcJN_d7KEBJ5TgAJCmKL1bkQ/edit#slide=id.p">Alexander, S. Latent-Variable Modeling for Natural Language</a></li><li><a href="https://www.youtube.com/watch?v=DWJYZq_fQ2A&amp;t=1s">Scott, S LDA Algorithm Description</a></li><li><a href="https://towardsdatascience.com/light-on-math-machine-learning-intuitive-guide-to-latent-dirichlet-allocation-437c81220158">Thushan, G. Intuitive Guide to Latent Dirichlet Allocation</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2441d57ecc8a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/%E6%B7%B1%E5%85%A5%E6%8E%A2%E8%A8%8E-latent-dirichlet-allocation-lda-%E8%88%87%E5%9C%A8%E6%8E%A8%E8%96%A6%E7%B3%BB%E7%B5%B1%E4%B8%8A%E7%9A%84%E6%87%89%E7%94%A8-2441d57ecc8a">深入探討 Latent Dirichlet Allocation (LDA) 與在推薦系統上的應用</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Deep Dive into Latent Dirichlet Allocation (LDA) and Its Applications on Recommender System]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/rosetta-ai/a-deep-dive-into-latent-dirichlet-allocation-lda-and-its-applications-on-recommender-system-e2e8ea5e661c?source=rss-2fc7b9c3f02a------2"><img src="https://cdn-images-1.medium.com/max/1000/0*LKGHxDP7q0oA2nAS" width="1000"></a></p><p class="medium-feed-snippet">Latent Dirichlet Allocation (LDA) is a topic modeling algorithm for discovering  the underlying topics in corpora in an unsupervised&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/rosetta-ai/a-deep-dive-into-latent-dirichlet-allocation-lda-and-its-applications-on-recommender-system-e2e8ea5e661c?source=rss-2fc7b9c3f02a------2">Continue reading on Rosetta.ai Taiwan Blog »</a></p></div>]]></description>
            <link>https://medium.com/rosetta-ai/a-deep-dive-into-latent-dirichlet-allocation-lda-and-its-applications-on-recommender-system-e2e8ea5e661c?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2e8ea5e661c</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Sun, 02 Feb 2020 07:42:40 GMT</pubDate>
            <atom:updated>2020-02-10T16:14:22.080Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 2)]]></title>
            <link>https://medium.com/rosetta-ai/when-martech-meets-artificial-intelligence-the-good-the-bad-and-the-ugly-part-2-d6edc1b5d419?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/d6edc1b5d419</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Mon, 13 Jan 2020 01:28:19 GMT</pubDate>
            <atom:updated>2020-01-18T00:29:34.184Z</atom:updated>
            <content:encoded><![CDATA[<h3>When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 2)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*RKrZtYHMUre95iMu" /><figcaption>Image from Unsplash (<a href="https://unsplash.com/photos/unRkg2jH1j0">https://unsplash.com/photos/unRkg2jH1j0</a>)</figcaption></figure><p>In the last <a href="https://medium.com/@huangkh19951228/when-martech-meets-artificial-intelligence-the-good-the-bad-the-ugly-part-1-fe21fe428485?">episode</a>, we discussed the applications of Artificial Intelligence (AI) on MarTech. By delivering contents more related to your users, you can increase your conversion rate and click-through-rates as well as your customer loyalty. In addition, accurate sales forecasting will enable you to minimize your warehousing costs. The potential of AI applying to MarTech, undeniably, can bring tremendous profit to e-commerces. Yet, they are not without flaws. There are limitations and pitfalls that we must be aware of. In this blog post, we will focus on what are some of the issues related to the applications of AI on MarTech.</p><h3>Limitations &amp; Pitfalls</h3><h4>Data</h4><p>The power of AI is subject to the quality of the data it is trained on. That is to say, the richer and cleaner the data, the higher performance the AI. Unfortunately, the majority of e-commerces, regardless of their scale, often poorly maintain their product feed structures. For instance, there was a fashion-related e-commerce I met at <em>Rosetta.ai</em>. In their dataset, initially, there was an attribute called “category”, which literally records which category the product belongs to. On a holiday, the planned to launch a campaign where some of the products will be on sales. Instead of adding a campaign tag for each of these on-sale product, they duplicated the original ones, changing the price, and modifying their “category” attributes. Although this may be one the easiest way for merchants to put products on sale, this makes their product feeds extremely messy. Aside from the fact that there exists duplicated items on their sites, this may befuddle the AI systems since they have no clue that some items are actually the same.</p><h4>Law</h4><p>As of May 2018, the <strong>General Data Protection Regulation (GDPR) </strong>has already took effect. It applies to all citizens of European Union (EU) and European Economic Area (EEA), roughly covering the entire Europe. It places a tight constraint on the collection and usage of personal data for these citizens. Although it successfully protects their privacy to a certain degree, given only the obfuscated data or partial data of the users, <strong>personalized marketing would not be optimal</strong>. Recommender system (RS), for instance, can only work sub-optimally if contextual and user-related data are not available. While we can still leverage session-based recommender system technique to deal with this situation, RS would be undoubtedly more powerful with the support of user information. From MarTech companies’ point of view, in addition to efficiently utilizing the limited data they can collect, it is also challenging to carefully read the GDPR legal codes and make sure they do not violate and of them.</p><h4>Fraudulence</h4><p>The majority of MarTech companies claim that they are using AI. Unfortunately, some of them are frauds, faking AI with conventional programming, some are not even workable, and only a small portion of them are legit AI MarTech companies. One example would be iFlytek, a speech recognition AI company from China. They once stated that their product can achieve highly accurate real-time machine translation for speech. In their demos, it seems that their product was truly amazing with fast and accurate translation. However, the truth is that there were actual human behind the screen typing and doing the translation themselves. This is completely absurd. While iFlytek is not a MarTech company, the reason behind this incident applies to all the AI companies. Aside from the fundamental <strong>integrity issue</strong> of the fake AI companies, another main factor causing this incident is that <strong>people are too overwhelmed by AI</strong> and that they believe that <strong>AI has unlimited superpower</strong>. This belief can be positive as more companies may be willing to adopt AI, but also can be very dangerous. Too much expectation on AI resulted from it could lead to unreasonable demands on the AI products from the client companies or customers. Therefore, in order to meet those requirements, some AI companies decide to cheat rather than discussing the technology infeasibility with their clients.</p><h3>What can we do?</h3><p>To avoid adoptions of inferior MarTech, it is essential to keep in mind three things:</p><h4>Read the Law</h4><p>As I mentioned, due to GDPR and other privacy-related laws, data collection and transfer now becomes a serious issue that everyone should pay attention to. Company could easily be charged huge amount of money for violating any of these laws. For example, violators of GDPR alone can be fined up to €20 million, which could result in bankrupt for the majority of the violated companies. Before adopting a new AI MarTech, make sure you understand what data they are collecting, can these personal data be deleted by their owner, and how they are using these data. Otherwise, your company will be at great risk.</p><h4>Keep Descent Expectation on AI</h4><p>AIs are good, but they do not have superpower as you might think. Neither will them be killing us, at least not in the foreseeable future. We should definitely be positive about AI MarTech because this industry is booming. Yet, we should only keep a reasonable expectation on them so that we can maintain a healthy AI ecosystem. In fact, one of the major reasons of the last AI winter was that AI fails the unreasonably high expectation projected by humans.</p><h4><strong>Always Validate the New Technology</strong></h4><p>Before adopting a new technology, make sure you can try it before actually using it. Although there is no guarantee that the product will still works after the trial periods, it is better than never seeing it work ever. At <em>Rosetta.ai</em>, our recommender system service provides 14-day free trial. In the meantime, we help you conduct A/B testing between your original recommender system and ours. You can easily compare our products with other competitors’. In addition to profitability, make sure your company also evaluates a new technology in terms of feasibility and applicability. If any of these three components are not met, perhaps it is not a good idea to adopt this technology.</p><p>All in all, AI-powered MarTech has great potential in generating more revenue. As long as the above threes are checked, you are good to go! Let’s embrace MarTech and AI!</p><h3>Reference</h3><ol><li><a href="http://technews.tw/2018/09/25/iflytek-fake-ai/">http://technews.tw/2018/09/25/iflytek-fake-ai/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6edc1b5d419" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/when-martech-meets-artificial-intelligence-the-good-the-bad-and-the-ugly-part-2-d6edc1b5d419">When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 2)</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[推薦系統之路 — 從自學到CTO]]></title>
            <link>https://medium.com/@huangkh19951228/%E6%8E%A8%E8%96%A6%E7%B3%BB%E7%B5%B1%E4%B9%8B%E8%B7%AF-%E5%BE%9E%E8%87%AA%E5%AD%B8%E5%88%B0cto-46ace329af38?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/46ace329af38</guid>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Fri, 10 Jan 2020 04:45:48 GMT</pubDate>
            <atom:updated>2020-01-10T04:46:06.098Z</atom:updated>
            <content:encoded><![CDATA[<h3>推薦系統之路 — 從自學到CTO</h3><p>從一個連推薦系統都沒聽過的大學生，到成為行銷公司Rosetta.ai的CTO的。</p><p>我，都經歷了些什麼？</p><h3>序章</h3><p>第一次接觸推薦系統是在2017年底，當時我還在香港科技大學就讀大學四年級。</p><p>登愣！</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*Tp5NVfQJsIjyJWAK" /><figcaption><a href="https://unsplash.com/photos/3Mhgvrk4tjM">https://unsplash.com/photos/3Mhgvrk4tjM</a></figcaption></figure><p>Gmail收到學校一封郵件，是關於一個數據科學的競賽International Data Analysis Olympiad (IDAO)，比賽內容是利用電商資料設計推薦系統模型，初賽優勝者可以去俄羅斯參加決賽，決賽優勝者可以直接保送俄羅斯的研究所。雖然對俄羅斯的研究所不是很感興趣，但沒有去俄羅斯玩過，所以就報名了比賽。</p><h3>摸索</h3><p>當時的我完全沒有任何推薦系統的背景，只有在Georgia Tech（喬治亞理工大學）交換期間上過人工智能、機器學習的課程，以及在YouTube上看過Stanford （史丹佛大學）的CS231n。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*U9pPPOAhtO6yjeca.png" /><figcaption>喬治亞州理工大學 （美國第三理工大學僅次MIT, CalTech）</figcaption></figure><p>於是我開始在網路上搜尋相關推薦系統的課程，很不幸的，不僅Coursera上沒有課，連Youtube相關並且品質比較好的影片都沒有，於是只能求助Google大神，利用各種關鍵字的排列組合：”Recommender systems”, “Netflix”, “Amazon” 等等，找到一些有用的文章（現在似乎已被刪除找不到），但概念都相當零散，每篇文章講的可能只是單一個別的演算法，某些演算法可能又是以其他演算法為基礎，完全沒有一套完整的學習系統，為了方便自己理解，也順便讓接觸這個領域的初學者輕鬆一點，於是我稍微整理了一些基本的演算法，並統整成一篇文章發到<a href="https://hackernoon.com/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75">Hackernoon</a> （Hackernoon為全球第三大Publication）。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KQgBOgTr5Ip9Hn1Z.jpeg" /></figure><p>出乎意外地，我的文章在Reddit (美國人用的PTT)得到不少好評，由此可知，這方面知識相當缺乏有系統的統整，而在那之後，我還發了另一篇比較進階的推薦系統<a href="https://towardsdatascience.com/introduction-to-recommender-system-part-2-adoption-of-neural-network-831972c4cbf7">文章</a>到TowardsDataScience，其中包括了我在IDAO所使用的演算法，這也讓我走上了我推薦系統這條不歸路。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/0*nEdyWtA9XgxzLxGx" /></figure><h3>創業</h3><p>畢業後，我收邀擔任<a href="https://www.rosetta.ai/">Rosetta.ai</a> 的CTO，Rosetta.ai主要業務是為電商提供個人化的推薦系統服務。聽著CEO Daniel描述著電商市場的大餅，裡面藏有多少商機，我毅然決然加入了這家公司。至於為什麼會請一個剛畢業乳臭未乾的毛頭小子擔任自家公司的CTO呢？Daniel其實是看到了我之前所發的兩篇文章，因為統整的還不錯所以決定拉我入伙的😂</p><p>在Rosetta.ai中，我將原本協同過濾的推薦系統，升級成深度學習模型為基礎的，和其他家推薦系統相比，我們平均幫用戶<strong>提升</strong>了<strong>248%的轉換率</strong>以及<strong>297%的點擊率</strong>，在業務部門強力推動下，目前全球我們已經有將近100家客戶！</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5oNxC0tMLfrC2ain0EsI6A.png" /><figcaption>Rosetta.ai 於日本東京參加亞洲最大AI加速器 Zero.ai的Demo Day</figcaption></figure><p>除了業界的表現外，為了增強我們的學界的技術聲望，我們也積極與台大林守德教授合作，參與<strong>全球最大推薦系統競賽2019 ACM RecSys Challenge</strong>，在將近1600個隊伍中，我們斬獲了<strong>第四名</strong>，並且受邀到丹麥哥本哈根發表我們的模型以及解決方案。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*sav5TPlmNAQh2KFZ.jpg" /><figcaption>2019 ACM RecSys Challenge哥本哈根論壇展示演算法</figcaption></figure><h3><strong>結語</strong></h3><p>在推薦系統的這條道路上，因為習得的技術，不僅接觸到電商的非技術層面，也得到出國演講的機會。回想起之前剛開始學習的困難，若當時有一套完整的推薦系統學習教材，我會更快步上軌道。因此，我決定和TibaMe課程平台共同推出了這門課:</p><h4><a href="https://www.tibame.com/course/609">打造智能推薦系統：用AI搞懂客戶精準行銷</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KOVXzxv8vYr0UEHSCZcoPg.png" /><figcaption><a href="https://www.tibame.com/course/609">https://www.tibame.com/course/609</a></figcaption></figure><p>這個課程我幫大家統整了推薦系統各項重點，同時這門課也是史上第一部中文的推薦系統線上教材，希望能幫助電商領域的資料科學家以及軟體工程師學習無痛學習推薦系統。</p><h4>課程三大特點：</h4><ol><li>解說各類傳統演算法以及進階深度學習的推薦系統，深入探討推薦系統的原理</li><li>附帶五部實作教程，徹底了解如何建立並評估推薦系統</li><li>介紹科技大廠如YouTube是如何打造智能推薦系統，讓你走在世界級技術尖端</li></ol><p><strong>1月28</strong>前購買課程享有<strong>早鳥優惠8折</strong>～</p><p>課程連結：<a href="https://www.tibame.com/course/609">https://www.tibame.com/course/609</a></p><p>想要走什麼路自己決定，推薦系統帶我看到了不一樣的風景，</p><p>你，要和我一同踏上旅程嗎？</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46ace329af38" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 1)]]></title>
            <link>https://medium.com/rosetta-ai/when-martech-meets-artificial-intelligence-the-good-the-bad-the-ugly-part-1-fe21fe428485?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe21fe428485</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[martech]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Wed, 01 Jan 2020 07:20:40 GMT</pubDate>
            <atom:updated>2020-01-05T02:25:51.519Z</atom:updated>
            <content:encoded><![CDATA[<h3>When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 1)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*12LxCRbsEfxi_y_j" /><figcaption>Image from Unsplash (<a href="https://unsplash.com/photos/hpjSkU2UYSU">https://unsplash.com/photos/hpjSkU2UYSU</a>)</figcaption></figure><p>The development of MarTech has been incredibly rapid. The number of <strong>MarTech companies</strong> rocketed from less than 150 in 2011 to more than 7000 in 2019, <strong>increased by over 46 times</strong>. It seems that MarTech has great potential in generating more revenue or decreasing costs, but the unknown risk lurking ahead also appears to be intimidating. In this series, I will discuss the good, the bad, and the ugly of MarTech.</p><h3>What is MarTech?</h3><p>MarTech is short for “Marketing Technology”. It refers to technologies that help optimize the marketing procedures. This term was coined by Scott Brinker, who is also known as “The Father of MarTech”. He categorizes MarTech into the following 6 domains:</p><ol><li>Advertising &amp; Promotion: Programmatically generating or posting advertisements.</li><li>Content &amp; Experience: Related to the production of content and the optimization of UX/ UI, such as personalization, SEO optimization, and content marketing.</li><li>Social &amp; Relationships: Social media or direct engagement with customers to improve conversion rate, such as chatbot and customer relation management.</li><li>Commerce &amp; Sales: Directly related to commerce activities and sales conversion, such as sales automation and e-commerce platform.</li><li>Data: Everything related to data, such as business intelligence, Data visualization, and customer data platform.</li><li>Management: Tools that help manage staff, products, finance, and projects.</li></ol><p>Of course, these domains are not mutually exclusive. Some solutions may span across multiple areas. For example, recommender systems or recommendation engines utilizes users’ behavioral data and contextual information to generate personalized product recommendation for optimizing the conversion rate. Therefore, we can say that they cover the 2nd, 4th, and 5th domains. In the next section, we will see how Artificial Intelligence can be applied to these different domains.</p><h3>When MarTech Meets Artificial Intelligence</h3><p><strong>Artificial Intelligence (AI)</strong> refers to intelligence demonstrated by machines that <em>behaves</em> like human or <em>thinks</em> like human. People often possess the misconception about AI as machines or algorithms that can learn by itself. In fact, that misconception is the definition for Machine Learning (ML), a subset of AI. ML explicitly refers to algorithms that learns the rules based on pairs of datas and expected outputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/0*fpgQHJX4oq2JqrbS.png" /><figcaption>Comparison between ML and other algorithms (<a href="https://www.pinterest.com/pin/66709638208891287/">https://i.pinimg.com/originals/3a/42/97/3a4297c9a348c65f6a37b58ab638ed3d.png</a>)</figcaption></figure><h4>AI v.s. ML</h4><p>Okay… so now you are telling me that ML ≠ AI and that the “machines learns by itself” and the “singularity” things are specifically referring to ML. Are you suggesting that some of the “intelligence” of the AI products could be pre-defined or pre-instilled by humans? Bingo! As a matter of fact, the majority of chatbot services nowadays, still rely on human-defined query-response rules, making the system literally “artificial” intelligence. I am not saying they suck. Actually, they are doing a pretty good job by saving a lot of time for managing customer relation. Such approaches have been dominating the markets because the ML-based counterparts are not mature enough yet. Yet, it is hard to foresee when the latter would surpass the former one.</p><h4>Applications of AI on MarTech</h4><p>In addition to Chatbot, let’s discuss some applications of AI on MarTech. We will especially focus on Machine Learning.</p><ul><li><strong>Recommender System (RS)</strong></li></ul><p>Often known as recommendation engines. Traditional RS approaches, such as Collaborative Filtering, consider user behavioral data and form a user-item matrix as demonstrated below. The goal is to predict the unknown ratings/preferences (empty entries) based on the known ratings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RarmBjx02bVjVeaHhWp2ZQ.png" /><figcaption>user-item matrix illustration. A-F are users (<a href="https://medium.com/hackernoon/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75">https://medium.com/hackernoon/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75</a>)</figcaption></figure><p>At <em>Rosetta.ai</em>, we leverage a Deep-Learning-based context-aware recommender system with specialization for different vertical domains to extract different features and optimize for various needs. Therefore, our recommender system is capable of successfully modeling the temporal dynamics of user interests. The careful A/B testings between our system and others show that our customers have <strong>improved their conversion rate by 248% and click-through-rate by 297%</strong> on average.</p><ul><li><strong>Personalized Content Generator</strong></li></ul><p>With Generative Adversarial Neural Network, meaningful automatic text generation is no longer impossible. Given the behavioral information and demographic data of the target user, a delicately implemented and tuned model is capable of generating personalized content which can be sent via email or SMS to maximize the retention rate. This could significantly alleviate the burden and workload of the marketing team.</p><ul><li><strong>Sales Forecasting</strong></li></ul><p>Sales forecast would enable merchants to determine the number of products/ ingredients to be ordered from their suppliers. By performing time-series analysis on the historical transaction data, we can obtain satisfactory prediction results on the expected amount of product sold next month or next quarter. <em>Rosetta.ai</em> takes a further step. In addition to the in-site transaction data, we take into account other factors, such as weather and regional KOL’s opinions in order for more accurate sales forecasting. The actionable insight provided by this service <strong>saves the tedious effort</strong> for our customers to manually determine the quantity of products to be ordered in the future. More importantly, it helps <strong>minimize the warehousing costs</strong> resulted from overstocking.</p><ul><li><strong>Sentiment Analysis</strong></li></ul><p>Sentiment analysis on product reviews could help merchants understand the market’s opinion in an autonomous manner. After collecting all the reviews, sentiment analysis can be automatically performed . Coupled with proper data visualization, not only will you know what your users think, but you will also know what they like or hate about your products. All you need to do is open the console and read the neat results processed by the sentiment analysis system. Dealing with user complaints have never been so easy and efficient.</p><h3>Conclusion</h3><p>The above are some of the applications of MarTech. In the next chapter, I will illustrate some of the pitfalls or limitation of current MarTech and AI. Till next time!</p><h3>Reference</h3><ol><li><a href="https://www.bnext.com.tw/article/54580/new-wave-of-marketing-martech">https://www.bnext.com.tw/article/54580/new-wave-of-marketing-martech</a></li><li><a href="https://medium.com/hackernoon/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75">https://medium.com/hackernoon/introduction-to-recommender-system-part-1-collaborative-filtering-singular-value-decomposition-44c9659c5e75</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe21fe428485" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/when-martech-meets-artificial-intelligence-the-good-the-bad-the-ugly-part-1-fe21fe428485">When MarTech Meets Artificial Intelligence — the Good, the Bad, and the Ugly (Part 1)</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 4th Place Approach to the 2019 ACM Recsys Challenge by Team RosettaAI]]></title>
            <link>https://medium.com/rosetta-ai/the-5th-place-approach-to-the-2019-acm-recsys-challenge-by-team-rosettaai-eb3c4e6178c4?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb3c4e6178c4</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Wed, 10 Jul 2019 05:17:44 GMT</pubDate>
            <atom:updated>2020-01-10T04:12:11.866Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*UtXAAm2r-iTNaHlL.png" /><figcaption><a href="https://tech.trivago.com/2019/03/11/recsys-challenge-2019/">https://tech.trivago.com/2019/03/11/recsys-challenge-2019/</a></figcaption></figure><p>The three-month recommender system competition, <strong>2019 ACM RecSys Challenge</strong>, has finally come to an end. We, <strong>RosettaAI</strong>, eventually <strong>won the 4th place among 1564 teams</strong> on the private leaderboard, clinching the final spot in the prize zone! Big thanks to <a href="https://www.csie.ntu.edu.tw/~sdlin/">Professor Lin</a> and our awesome teammates from National Taiwan University and the University of Texas at Dallas. Without the great teamwork, it’s impossible to accomplish such an achievement.</p><h3>Abstract</h3><p>The ACM Recsys Challenge this year, hosted by trivago.com, challenged data scientists to build a session-based and context-aware recommender system to solve a ranking problem. Given a sequence of user interactions, the goal is to rank a list of hotels such that the topper the hotel in the list, the more likely that it will be clicked-out, as shown in the following figures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*kxDeziH_NcSfLZI2.png" /><figcaption><a href="https://tech.trivago.com/2019/03/11/recsys-challenge-2019/">https://tech.trivago.com/2019/03/11/recsys-challenge-2019/</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*H2pZ1AlSG5qwBOhV.png" /><figcaption><a href="https://tech.trivago.com/2019/03/11/recsys-challenge-2019/">https://tech.trivago.com/2019/03/11/recsys-challenge-2019/</a></figcaption></figure><p>Our approach involves three different models, Neural Networks, LightGBM, and XGBoost. I was in charge of the whole data processing procedure, the implementation and design of the Neural Networks, as well as the majority of feature engineering. This blog post will summarize our approach. I’ll briefly illustrate the datasets, the loss function, the Neural Networks architecture, and feature engineering.</p><h3>Datasets</h3><p>Trivago provided two sets of data, the user-item interaction and the item metadata. In this blog post, the second one would not be discussed as it contains relatively less information. Below shows an example of a sequence of interactions in a session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-TytJONOfIK3kDkm.png" /><figcaption><a href="https://recsys.trivago.cloud/challenge/dataset/">https://recsys.trivago.cloud/challenge/dataset/</a></figcaption></figure><p>Each row represents an action of the user, the type of which is recorded in the <em>action_type</em> column. Its corresponding column, <em>reference</em>, tells us which object that action was operating on. For instance, in the first row, <em>search for destination</em> is matched with <em>Barcelona, Spain</em>. It means that the user enters <em>Barcelona, Spain</em> in the destination box. The integer value in the reference column represents the ID of hotels. The most important action is <em>clickout item</em>, implying a user click on a link that leads to an external site. Only in the rows associated with this action, <em>impressions</em> and <em>prices</em> are not null. Both of these two columns store a list of integers, indicating the list of hotels displayed to the user and the price of each hotel, respectively. In these <em>clickout item</em> rows, the <em>reference</em> column can be regarded as the target. The goal is to sort the impression list such the target is positioned in the front of the list.</p><p>The evaluation metric for this challenge is <strong>Mean Reciprocal Rank</strong> (MRR):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cGwcxjtWZxotLZzI.png" /><figcaption><a href="https://www.oreilly.com/library/view/c-machine-learning/9781788996402/assets/b7a61c4a-53bb-43ae-8140-4f81106476ac.png">https://www.oreilly.com/library/view/c-machine-learning/9781788996402/assets/b7a61c4a-53bb-43ae-8140-4f81106476ac.png</a></figcaption></figure><p>Basically, this metric awards algorithms that rank the target higher than the other candidates. Trivago has provided an example to illustrate the metric in case you don’t understand:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*mTT6wyYJGfdvD_kANg0YLA.png" /><figcaption><a href="https://recsys.trivago.cloud/challenge/dataset/">https://recsys.trivago.cloud/challenge/dataset/</a></figcaption></figure><h3>Loss Function</h3><p>At the first glance of this challenge, I thought about using a ranking loss, which I believe that the majority of teams would opt for, such as <a href="https://arxiv.org/pdf/1205.2618"><strong>Bayesian Personalized Ranking</strong></a> (BPR) or <a href="https://www.microsoft.com/en-us/research/publication/learning-to-rank-with-non-smooth-cost-functions/?from=http%3A%2F%2Fresearch.microsoft.com%2Fpubs%2F68133%2Flambdarank.pdf"><strong>LambdaRank</strong></a>. However, eventually, I decided to treat it as a binary classification problem, breaking down each item in the impression list into an individual sample. The loss is defined as <strong>Binary Cross-Entropy </strong>(BCE). I did so for two reasons:</p><ol><li>I was inspired by <a href="https://research.google.com/pubs/archive/45530.pdf">this paper</a> published by Youtube back in 2017. It appeared to me that use case of their second stage predictions resembles the context of this challenge.</li><li>The data reminded me of a previous competition which I undertook, <a href="https://www.kaggle.com/c/avito-demand-prediction">Avito Demand Prediction</a>, held on Kaggle by Avito (we were 5 places away from the gold medal zone). I remembered lots of top scorers using BCE loss, even though it was not a binary classification problem. It seems that if the output conforms to such probabilistic distribution, BCE is a robust loss function for the model to learn properly.</li></ol><h4>Neural Network Architecture</h4><p>My Neural Network is based on the previously mentioned <a href="https://research.google.com/pubs/archive/45530.pdf">Youtube paper</a> and <a href="https://arxiv.org/abs/1703.04247">DeepFM</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mGzZu8pqZf98nqWytGT63Q.png" /><figcaption>The Proposed Network Architecture</figcaption></figure><p>The features can be divided into three parts: continuous features, categorical features, and sequential categorical features.</p><h4>Continuous Feature</h4><p>Processing continuous features is simple. The only thing that has to be aware of is that continuous features should be properly scaled in order for the Neural Nets to learn better. In our implementation, MinMaxScaler from Sklearn and the <a href="https://www.kaggle.com/mathormad/knowledge-distillation-with-nn-rankgauss/data">RankGaussScaler</a> from this amazing Kaggle kernel work the best.</p><h4>Categorical Feature</h4><p>Categorical features were encoded with my self-implemented encoder rather than Sklearn’s LabelEncoder. The latter one performs sorting while fitting the data; thus, it takes a huge amount of time as the data is very large and some of the features are of string type. In addition, my encoder also allows me to directly access the forward and backward encoding-decoding mapping for every feature. After being encoded, these categorical features are then passed through embedding layers to obtain higher dimensional vectors that contain more information than the original value.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/64e3c522fd9713559f8885754008e7ed/href">https://medium.com/media/64e3c522fd9713559f8885754008e7ed/href</a></iframe><h4>Sequential Categorical Feature</h4><p>Similarly, sequential categorical features are also encoded and embedded with the same method. One thing to note is that same features share the same embedding weights. For instance, in the above figure, the blue, the red, and the green bars below are derived from the same item embedding matrix. Two of these features are applied bi-directional GRU to model the temporal dynamics in each of the sequences. Then, they are aggregated with Max Pooling to become vectors of the same dimension as the other categorical features.</p><h4>Feature Interaction</h4><p>As pointed out in the <a href="https://arxiv.org/pdf/1703.04247.pdf">DeepFM paper</a>, feature interaction allows the model to discover the implicit meaning behind user behaviors. Consider an app store recommender system, when interacting the app category and the timestamp feature, it might have a better understanding of the users’ intention. For instance, during meal-time, it may be a good idea to recommend food delivery app. Hence, the importance of effective feature interaction mechanism cannot be overestimated.</p><p>Typically, feature interaction refers to the dot products of the latent vector for each feature. Based on this terrific <a href="https://github.com/ChenglongChen/tensorflow-XNN">XNN repository</a> by <a href="https://github.com/ChenglongChen">ChenglongChen</a>, it turns out that we only need five lines of code to compute the aggregation of these dot products.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1ebcff067a27857976b73fb7cbf9c57/href">https://medium.com/media/a1ebcff067a27857976b73fb7cbf9c57/href</a></iframe><p>The idea is based on the following equations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*6i8xXLk4F-ivlbQOjz0KGQ.png" /></figure><p>Essentially, to compute the sum of dot products of the embeddings, we simply minus the square of sum by the sum of square of these vectors and divide it by 2. In this way, the second-order feature interaction can be efficiently computed.</p><h3>Feature Engineering</h3><p>In the latter part of this competition, we spend the majority of time in feature engineering for LightGBM and XGBoost. In the end, we came up with more than 60 features. Some of the interesting features are illustrated below:</p><h4><strong>Predicted Next Interaction Position</strong></h4><p>This is based on the position of the last step’s item and second last step’s item in the current impression list. It is computed as the following:</p><pre>predicted_pos = last_pos + (last_pos- second_last_pos)</pre><p>On the right-hand side, the right part tells us the distance in the impression between the position of the second last step to that of the last step. Then, we add that distance onto the last position. We can think of it as extending the user’s last motion, and thus it can be viewed as the predicted position of the next interaction.</p><h4>Time Difference</h4><p>We took the first, second and third derivative of timestamp between each step. Specifically, the first derivative is obtained by computing the difference between the timestamp of two consecutive rows, while the second one is defined as the difference between the first derivative. The third derivative is computed in a similar manner.</p><h4>Time Elapse</h4><p>By aggregating the time difference between each step, we compute the (maximum/ average/ sum) of the amount of time the user spends on each item in the impressions.</p><h3>Conclusion</h3><p>Overall, we had a lot of fun in this challenge. Although the competition was ridiculously fierce, especially in the last few weeks, the results turned out to be pretty well. Sincere appreciation again for all our great teammates and everyone who has given us advice along the way. Thank you all for making this possible. The GitHub repository for our solution can be found <a href="https://github.com/rosetta-ai/rosetta_recsys2019">here</a>. Should you have any question or comment, please leave it below. If you are looking for product trials or collaboration opportunities, please contact me directly via <strong>steeve@rosetta.ai</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb3c4e6178c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rosetta-ai/the-5th-place-approach-to-the-2019-acm-recsys-challenge-by-team-rosettaai-eb3c4e6178c4">The 4th Place Approach to the 2019 ACM Recsys Challenge by Team RosettaAI</a> was originally published in <a href="https://medium.com/rosetta-ai">Rosetta.ai Taiwan Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hands on Graph Neural Networks with PyTorch & PyTorch Geometric]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/hands-on-graph-neural-networks-with-pytorch-pytorch-geometric-359487e221a8?source=rss-2fc7b9c3f02a------2"><img src="https://cdn-images-1.medium.com/max/1116/1*AMTs2MPLKCwOZy4WaKbYTw.png" width="1116"></a></p><p class="medium-feed-snippet">In my last article, I introduced the concept of Graph Neural Network (GNN)and some recent advancements of it. Since this topic is getting&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/data-science/hands-on-graph-neural-networks-with-pytorch-pytorch-geometric-359487e221a8?source=rss-2fc7b9c3f02a------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/hands-on-graph-neural-networks-with-pytorch-pytorch-geometric-359487e221a8?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/359487e221a8</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Thu, 30 May 2019 16:21:54 GMT</pubDate>
            <atom:updated>2019-06-13T14:51:45.233Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[A Gentle Introduction to Graph Neural Network (Basics, DeepWalk, and GraphSage)]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3?source=rss-2fc7b9c3f02a------2"><img src="https://cdn-images-1.medium.com/max/2600/1*2jyn3aKiWkXfXYe_01qJNQ.jpeg" width="5616"></a></p><p class="medium-feed-snippet">Recently, Graph Neural Network (GNN) has gained increasing popularity in various domains, including social network, knowledge graph&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/data-science/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3?source=rss-2fc7b9c3f02a------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3?source=rss-2fc7b9c3f02a------2</link>
            <guid isPermaLink="false">https://medium.com/p/db5d540d50b3</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kung-Hsiang, Huang (Steeve)]]></dc:creator>
            <pubDate>Sun, 10 Feb 2019 14:58:37 GMT</pubDate>
            <atom:updated>2020-01-31T03:10:59.131Z</atom:updated>
        </item>
    </channel>
</rss>