<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kedan  Li on Medium]]></title>
        <description><![CDATA[Stories by Kedan  Li on Medium]]></description>
        <link>https://medium.com/@kedanli?source=rss-23aa5dbf79b9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*wsmhldAl4zecsFIM.</url>
            <title>Stories by Kedan  Li on Medium</title>
            <link>https://medium.com/@kedanli?source=rss-23aa5dbf79b9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:00:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kedanli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Interactive Learning System for Personal Email — Inferring user’s preference through small amount…]]></title>
            <link>https://medium.com/@kedanli/under-the-interactive-learning-inbox-infer-users-preference-from-minimal-amount-of-labeled-data-49b7212781c6?source=rss-23aa5dbf79b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/49b7212781c6</guid>
            <dc:creator><![CDATA[Kedan  Li]]></dc:creator>
            <pubDate>Thu, 23 Nov 2017 01:41:54 GMT</pubDate>
            <atom:updated>2017-12-02T23:29:23.210Z</atom:updated>
            <content:encoded><![CDATA[<h3>Interactive Learning System for Personal Email — Inferring user’s preference through small amount of labeled data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c5g1_EXnmpaGkkvdXbdNHg.png" /><figcaption>PickedMail — The personal AI Inbox</figcaption></figure><p>Nowadays, with the significant increase in the volume of email people receives daily, email clients such as Outlook and Gmail developed automated processes to highlight important messages in users’ inbox. Most of the existing solutions give limit control over the intelligence and no simple way for users to teach the intelligence about their preference. To solve the problem, we created PickedMail, an email client that captures users’ current personal preference and promote important messages using interactive learning techniques. As you work on emails, the personal AI picks up your preference silently and classifies future emails in the way you prefer. The unusual part of this AI is that it’s not fully relying on a bag of historical data, but learns directly from user provided inputs. After you relabel a few email of a kind, you’ll notice that inbox already picked up the changes and adjusted it’s classification rules accordingly. With hundreds of beta users testing over a two month period, the system has demonstrated its ability to predict individual’s preference accurately over time and adapt rapidly to preference changes.</p><h3>The Problem</h3><p>During college, my inbox has always been a disaster, crammed with the overwhelming volume of non-spams emails. Major Email clients made significant efforts to help users identify emails that require attention. Outlook separates emails into <a href="https://support.office.com/en-us/article/Focused-Inbox-for-Outlook-f445ad7f-02f4-4294-a82e-71d8964e3978">Focused and Others</a> tabs while Gmail provides a build-in <a href="https://www.google.com/mail/help/priority-inbox.html">Priority Inbox</a>. However, unlike spams, people’s sense of importance for emails are highly personalized and changes over time in unpredictable ways. With the classification rules being implicit, today’s email clients did not provide an easy way for users to communicate with the intelligence under the hood. Some user interactions even lead to false indications of the importance. Even today, both Important box and Low Priority box are full of misclassified emails.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qpChZYekmORO96lHjrdZ3A.png" /><figcaption>Screenshots of My Google Inbox</figcaption></figure><p><strong>Restate the problem more academically: </strong><br>As the <a href="https://www.radicati.com/wp/wp-content/uploads/2017/01/Email-Statistics-Report-2017-2021-Executive-Summary.pdf">number of messages people receive increases</a>, and the average size of screen people use to browse email decreases, there comes the need for designing user-interface that enables users to manipulate data at scale with simple interactions. Recent progress in artificial intelligence makes the problem possibly solvable. The primary difficulty lies in designing a way for users to communicate their thoughts to the AI which will then do the majority of the work. The mean must leverage both the user interface and the learning algorithm. The interactions should be natural enough that users can operate at ease. The system should be able to infer users’ intentions correctly from the input and noticeably update its classification rules.</p><p>Labeling example emails as is the most natural interaction. Using examples emails as input is feasible if the learning algorithm can infer from a minimal amount of labeled data, as users usually don’t have the patience to labels massive emails. Other types of inputs require users to come up with specific rules or constraints, therefore are typically complicated for users and over-exhaustive for the learning algorithm. Afterall, human users know how to label a particular message but not the decision surface. After comparing several input methods, we decided to let users provide example emails. The problem is then transferred to designing an algorithm that could correctly infer user’s intent with small numbers of training samples. Any statistical based supervised learning algorithm expect a considerable amount of data to fill in the learning space, especially if the data has a high dimension.</p><h3>Interactive Learning System</h3><p>The concept of transfer learning helps, under the assumption that most users share a degree of similarity in their sense of usefulness for emails (e.g., over 90% of gmails users consider purchase receipts as helpful emails.) The system trains an importance model that represent average people’s preference for emails and apply the model to individual users when they first onboard. Once a user starts using the application, the interface enables the user to indicate which emails should be classified differently. According to the inputs, the agent retrains itself to adapt to the specific user’s sense of importance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uD-S4VUDrpccPkKtFSk_Bg.png" /><figcaption>Interactive Learning System for Personal Email</figcaption></figure><p>The goal is to make the learning fast so users can sense the agent improving after relabeling a small number of messages. When a user indicates that some emails should be classified differently, the system performs a density-based clustering over the training samples and adjusts the labels of samples within the same cluster as the user input. Such use of unsupervised learning magnifies the effect of human-provided examples on the dataset. During field testing, I also discovered that users’ sense of importance for emails shift overtime dramatically — meaning a previously insignificant message could become important in the future. To deal with the drifting domain, I developed a forgetting function that sets higher weights for more recent relabels during retraining, to help keep the model up-to-date with user’s latest preference. With all these interactive learning features, the system creating the experience as if there is a secretary actively learning how to sort users’ emails.</p><h4>General Importance Model (GIM)</h4><p>We collected an email dataset of 2.4 million messages from 119 different Gmail accounts and used Important tag assigned by Gmail to train the General Importance Model. Assuming that personal preference is taken into account when Google assign the Important labels, there is a margin of uncertainty in the training data. Since we only expect the GIM to reach an accuracy that’s comfortable for users to start manipulating, it’s acceptable that the training data have a certain degree of noisiness.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MtLrjjFYpP-01xzsfq7FrQ.png" /><figcaption>Performance of different classification algorithms</figcaption></figure><p>We applied various text classification algorithm and performed a K-fold validation of 6 folds. <a href="https://github.com/yoonkim/CNN_sentence">Convolutional Neural Network for Sentence Classification</a> introduced by Kim Yoon, using a combination of filter sizes of 1, 2, 3, 4, 5 achieved the highest recall and F1 score. It’s difficult to increase the accuracy further because part of the data might not be completely learnable (e.g., some emails could be marked as Important by some users and Unimportant by others). When a new user onboard, we apply the General Importance Model to the user’s emails if their email service provider doesn’t have built-in importance labels. The initial classification accuracy range from 65%~80% without further personalization.</p><h4>Personal Importance Model (PIM)</h4><p>Retraining the Personal Importance Model is the crucial process to update the intelligence with the latest user inputs. PIM uses Naive Bayes Algorithm instead of CNN for its training speed of O(N). The system preferably utilizes the emails from the account for training if the email account has sufficient data. The reason is that email accounts are likely to receive emails similar to the ones they received in the past since the users are interested in the subjects or are already on the mailing list of some of those senders. Training with messages an account received in the recent past generates model more efficient at predicting the importance of future emails the account receive.</p><p>Personal Importance Models are frequently retrained to sync with user’s latest preference. All accounts with new message(s) relabeled by human since the last retrain are enqueued for retraining. In the production system, a ranking algorithm, which favors the accounts with most human modification, determines the training order of the PIMs in the queue. To ensure a reasonable training time and sufficient training data (some email accounts may have very little emails), we set a hard limit on the minimum and maximum size of training data for a retrain. If the number of messages in an account is under the minimum, the system randomly injects emails from the GIM training datasets until the minimum. If the number of emails exceeds the maximum, the system only uses the latest received messages up to the maximum.</p><p>While training Personal Importance Models, data points with human-assigned labels have significantly higher value than others, because human-assigned labels indicate how users want to manipulate their email AIs to behave. When users attempt to teach the intelligence the significance of a particular kind of email, they’re likely only to modify one or a few examples, hoping that the intelligent agent can understand their intentions through the samples. For example, when a user marks the latest Facebook notification emails as low priority, an ideal system should adjust the classification for future FB notifications, instead of expecting the user to relabel a majority of FB notification emails.</p><h3>Manipulate Training Labels Based on Human Inputs</h3><p>Before training the Personal Importance Models, the system must somehow magnify the impact of human-assigned labels, to make the learning fast and noticeable to the users. The first experiment I did was to increase the weight of human-labeled samples. However, it leads to poor results. Since human-assigned labels are arbitrary, it results in similar emails having different training labels, making the training data noisy. When using the raw labels for training, the training algorithm performed poorly.</p><p>A better solution is to normalize the training labels of the data points that are similar to the ones users relabeled, and use the adjusted labels to retrain the PIM. The manipulation ensures that the retrained Personal Importance Models will converge, regardless of how users manipulate the labels. The system adjusts the training labels using the following formula:</p><pre>1. Perform DBSCAN over user&#39;s email dataset with a min cluster size of 2. From the resulting clusters., find every cluster that contains human labeled data point(s) and applies step 2. to it.</pre><pre>2. In the cluster, if all human-labeled data point(s) has the same label, all the data points in the cluster adopt the label. Otherwise, apply step 3. to the cluster.</pre><pre>3. Perform KMeans Algorithm on with n centroids distributions, where <em>n</em> is the number of type of human-assigned labels (<em>n</em> = 2, with Important and Unimportant). Set each centroid at the mean of the human-labeled data points of each type. After E step of each iteration, check whether the current partition perfectly separates the human-assigned labels. If there is a separation, keep iterate until the algorithm converge, or there isn&#39;t a perfect separation anymore, and exit the algorithm, preserving the last partition that is a perfect separation.</pre><pre>4. If there is a perfect separation in step 3., split the cluster into <em>n</em> subclusters based on the partition. Otherwise, all data points with human-assigned label vote for a uniform training label for the cluster.</pre><p>Since the formula is long and seems arbitrary, I’ll walk through the process with an example and explain the reasoning behind each design decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6X3TTulNhQ3qr-0ExvdJTw.png" /><figcaption>An user’s email data with labels</figcaption></figure><p>Assume above is a user’s email data with labels. The system first performs DBSCAN on the dataset. Since DBSCAN forms clusters of points closely packed together based on distance, it’s an ideal approach to identify groups of emails that are highly similar when the system is unaware of how many groups there are. The goal of the remaining steps in the pipeline is to select a uniform label for each cluster.</p><p>We’ve examined DBSCAN clustering result on several email accounts and identified that over 70% of the clusters shows meaningful categories, eg. Amazon purchase receipts, bank statements, Twitter feeds, etc. Another advantage of performing DBSCAN is that it’s average runtime complexity is O(n log n), making the computation efficient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9vkuArmDtsddhWjsdBWzIw.png" /><figcaption>Perform DBSCAN Clustering on the Dataset</figcaption></figure><p>In clusters without human-labeled data point, no change is made. The system assumes that the labels currently assigned are aligned with user’s preference. For clusters that contain human-labeled data, we use human-assigned labels as the only deciding factor when selecting a uniform label for the cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pduA8cSwoSpZidT9u-VArg.png" /><figcaption>Identify the clusters that need further adjustment.</figcaption></figure><p>For clusters which human-labeled data point(s) are of the same label, we adopt the label for all data points in the cluster. An example would be: the General Importance Model assigned most Twitter newsletters as significant, but, a user decides that Twitter newsletters are trivial to him and relabeled the latest Twitter newsletters as unimportant. He expects the intelligent agent to classify all future Twitter newsletters as unimportant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P0bqmgiwoosEp4QSlp59xQ.png" /><figcaption>Identify Important, Unimportant and Ambiguous Cluster</figcaption></figure><p>The label becomes ambiguous if a cluster contains human-labeled data points of different classes. Many possible scenarios could lead to such outcome. For example, a user might misclassify some messages by accident; a cluster might be subdividable (we try to avoid this scenario by using a small eps in DBSCAN of 1.); a user might change his mind about the importance of some emails; a user might be illogical as well.</p><p>For an ambiguous cluster, we first perform KMeans algorithm over the cluster with 2 target centroids. The goal is to determine if there exists an optimal way to subdivide the cluster to coincide with the human intention. KMeans algorithm attempt to address the fact that DBSCAN only considers the distance but not the distribution of the data. Some cluster might contain data of both labels that are similar enough. For example, DBSCAN might produce a cluster with both Amazon Receipts and Amazon Returns.</p><p>When performing KMeans algorithm, instead of randomly selecting the 2 initial centroids, we set the centroids at the mean of the human-labeled data points of each class (important or unimportant). This helps KMeans algorithm converge toward a distribution that matches the distribution of the human labels. Once the algorithm found a match, we keep iterate to find more optimal assignment as long as there still exists a perfect match.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*a1h51nWb9X0jAd0A5a3h0g.png" /><figcaption>Results of KMeans Algorithm</figcaption></figure><p>The two subclusters in Cluster 2 successfully separated the human-labeled samples of different label. In this case, we split the cluster according to the result of KMeans algorithm. The two subclusters in Cluster 1 failed to separate the human-labeled data points of different classes. As a result, data points in Cluster 1 has to vote to decide a uniform training label.</p><p>Voting happens when KMeans algorithm fails to produce a way to 2 subclusters that separate the human-labeled data points of different classes. Voting inevitably overrides some of the human-assigned labels. Despite the cons, voting ensures that the training data produce a robust Personal Importance Model by cleaning the potentially noisy training data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EVpi6CoxnvsmPTFk2QlPDQ.png" /><figcaption>Outcome of Manipulation</figcaption></figure><h3>The Voting Mechanism with Forgetfulness</h3><p>Initially, the voting rule is as simple as selecting the majority. During field testing, I also discovered that users’ sense of importance for emails shift overtime dramatically — meaning a previously insignificant message could become important in the future. As the number of user-assigned labels increases and the number of ambiguous clusters increases over time, the simple voting rule of selecting the majority cause problem when users’ preference changes. For example, a user wanted to receive recruiting emails in the past and hand-landed a couple of recruiting emails as useful. Once he found a job and want to block all the recruiting emails, he now has to relabel even more emails than he previously did to reverse the system’s behavior.</p><p>Based on the fact that users care more about their current preference versus the past ones, we introduced a forgetting function that favors recent actions. During voting, the system assigns a weight to each data point in the cluster using a time-based forgetting function, and them sum up the weighted votes for every class. The current system adopts the following function:</p><pre>1. Sort all the data points with human-assigned labels in the cluster based on the timestamp labels are added in descending order. <br>2. Assign a weight of 1 to the first message, a weight of 1 × <em>d</em> to the second message and a weigh of 1 × <em>d</em> ^ <em>n</em> to any remaining messages, where <em>n</em> is the rank.</pre><p><em>d</em> is the depreciation rate, controlling the speed the model forget previously learned knowledge in ambiguous clusters. The value <em>d</em> is chosen based on how many new data points of Label A is required to reverse a cluster with a given number data points of Label B, simulating how many new emails user need to relabel to alter the behavior of the model (as in Table 1). The current system sets <em>d</em> as 0.8.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Sd1tFqzq8-uNj4nkMYYGA.png" /></figure><h3>Interface &amp; Interaction Designs</h3><p>The interface layout and interaction flow take great reference from mainstream email clients’ design to reduce users’ learning curve. Email list view takes the majority area of the main interface while Menu, Compose, Edit, Account-switcher buttons are located at the four angles of the screen. The search bar appears at the top of the email list and leads to a searching view. Clicking the menu bar opens the menu list containing links to other folders and settings. Clicking a message cell opens the email content. Everything is as expected. The browsing view borrows part of the interaction design from Safari on iOS: when user scroll toward the bottom, the top and the bottom bars disappear, leaving more room for content browsing(as in Figure 3 center).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P6dvTW6-acQyRr9SEeI9vQ.png" /><figcaption>Important emails are under the Picked tab and non-important emails under the Others tab. A user modifies the label of a message with a simple swipe.</figcaption></figure><p>While existing email clients might interpret user’s unintended actions as implications of importance, PickedMail makes the presence of artificial intelligence explicit: messages under Picked are considered important and those under Others are considered unimportant by the system. The interface allows users to modify the classification of an email with a swipe. The design intends to provide users a sense of control over the machine learning classifier, so they are free from worrying about an unintended action, such as reading an unimportant email for a long period, could cause the system to learn the message as important.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rJcVtqdWb867BfaCdxA_BA.png" /><figcaption>Reply, Forward, Archive, Delete and Snooze are available on the toolbar on email browsing view (left). The toolbar disappears as user scroll down on the content (center). Fast read mode marks emails as read as user scrolls over the screen (right).</figcaption></figure><p>Fast Read helps user go over unimportant messages quickly (as in Figure 3 right). With the mode turned on, emails displayed on the screen are marked as read automatically after an adjustable period. Compare to the traditional email list interaction design which requires a finger gesture to mark each email as read and the batch archive of Google Inbox, Fast Read improve productivity while preventing users from omitting messages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0bHld-LNG7kzlhsrl5D2tA.png" /><figcaption>PickedMail implements most of the functionality modern email clients provides(left). In beta versions, we also present a graph for the users to see the learning statistic.</figcaption></figure><h3>The Evaluation</h3><p>Personal Importance Model is, by its nature, difficult to evaluate, because whether a classification is correct is subjectively determined by the user. We did not use pseudo-data for testing because a generated oracle is not a good simulation for a human user. To calculate the accuracy of a personal importance model, we compare the importance label assigned by the system when a message arrived in user’s inbox with the final label after user browsed the email, assuming the user would modify the label if it is assigned incorrectly. To increase the accountability of the result, we only evaluate the active users — users who use the app at least 4 days per week.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mvL9T1xyAASX4Da3sFvALw.png" /><figcaption>The moving average accuracy of 50 data points of 14 active users over a period of 60 days since they started using PickedMail.</figcaption></figure><p>The moving average of 50 data points shows that most of the PIMs can maintain an average accuracy of above 90% over time. Many users’ accuracy dropped below 0.8 near the beginning, but as they relabel some emails, PMIs’ accuracy rises. In the chart, the accuracy of users No.3,8,9,10 dropped dramatically during the middle of the evaluation. Changes in users preference might cause the drop, e.g., a person who neglect all housing emails before, began looking for a new apartment, therefore start treating housing recommendations as important messages. The accuracy of the model decrease when users first indicate the preference change and rise again after the model gets updated. The flat tops of accuracy 1 for user No.4,5,12 might cause by users not examining the emails through the application (e.g., users might have read those emails through other email applications). Thus, the 100% accuracy does not necessarily represent the model’s actual performance.</p><h3>Experiment on Custom Tags</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYrWMD7Kx97w%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYrWMD7Kx97w&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYrWMD7Kx97w%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/40d65f9653a0987325ddec4544d352ce/href">https://medium.com/media/40d65f9653a0987325ddec4544d352ce/href</a></iframe><p>With the system’s success in identifying emails that require high attention, I thought about a more challenging problem: Is it possible to let users create any classifier they want with interactive learning? Moderns email clients allow users to create custom folders or tags for their inbox, but users have to design the rules themselves or add every email by hand. However, with complete customization, the concept of transfer learning became inapplicable. The learning system has to learn a new notion defined by the user solely through user-assigned labels.</p><p>We ran the experimental system which allows user to create custom tags by selecting some emails as examples. The interactive learning system works in the same way, except it’s learning a custom concept. As expected, the feedback wasn’t great. To create a custom cluster, a user has to come up with examples himself and searching for samples emails isn’t easy on the mobile interface. When selecting samples, human users tend to give repetitive instances that belong to the same cluster, causing unnecessary work. We tried automatically adding an entire cluster once user picked one email from the cluster, but, with the small interface, users are unaware of all the added emails and searched anyway. The accuracy of the custom tag can go over 90%, but over 60% of the tags user created comprise less than 5 senders. Compare to the time user spend on setting up the tag, the value gain is relative small.</p><h3>Conclusion &amp; Future Work</h3><p>PickedMail ceased operation on September 10, 2017. Since the interactive learning system could only operate on the backend server, the cost of service is approximately $1.3 per user per month. As most email clients are well established for free, interactive learning itself doesn’t give users enough incentive to switch to a paid service. Despite positive feedback on the user experience, I think PickedMail is not yet mature for entrepreneurship.</p><p>Through 6 months of operation, PickedMail processed emails for 546 email accounts. We demonstrated that it’s engineeringly feasible to build robust interactive learning system to assist in classification tasks, and users are open to the idea of interacting with artificial intelligence directly. As long as the intelligent agent is responsive enough, a considerable portion of users are willing to spend time teaching the intelligent agent partially due to a natural tendency to correct mistakes and partly due to expecting a smarter agent will help save time in long-term. The success of my proposed solution relies on the common ground users share on what emails are significant and the natural way email data distribute. Therefore, the method may not directly apply to other interactive learning classification problems that do not share the same traits.</p><p>There are many possible directions for future work. With some modification, the solution might apply to other personalized text classification problems such as feeds filter for social network or news &amp; articles recommendation. The method may also refer to personalized recommendation systems for other mediums such as music, movies, and games, with different feature extraction techniques. A more challenging direction is to figure out ways to build interactive learning without the help of transfer learning. Instead of modifying a concept, learning system will require users to teach entirely new concepts. This problem is particularly hard because the learning system has to, somehow, populate the sample space with the limited information provided by the end users. Hopefully, renovations in the way people interact learning systems will lead to new possible ways of solving this problem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49b7212781c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>