<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by SeongKyung Kim (Irene Kim) on Medium]]></title>
        <description><![CDATA[Stories by SeongKyung Kim (Irene Kim) on Medium]]></description>
        <link>https://medium.com/@skirene?source=rss-91b48bffb803------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_aReF2gZrVwRj_eCp9ScQg.jpeg</url>
            <title>Stories by SeongKyung Kim (Irene Kim) on Medium</title>
            <link>https://medium.com/@skirene?source=rss-91b48bffb803------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:28:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@skirene/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Anomaly Detection with DeepSAD]]></title>
            <link>https://skirene.medium.com/anomaly-detection-with-deepsad-f45388a48208?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/f45388a48208</guid>
            <category><![CDATA[anomaly-detection]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Wed, 26 Mar 2025 07:21:35 GMT</pubDate>
            <atom:updated>2025-03-27T08:25:18.762Z</atom:updated>
            <content:encoded><![CDATA[<h4>for Noisy, Imbalanced, or Unreliable Datasets</h4><h3>What is DeepSAD?</h3><p><strong>DeepSAD</strong> stands for <strong>Deep Semi-supervised Anomaly Detection</strong>. It <strong>combines deep neural networks with a specialized loss function</strong>, enabling effective learning from partially labeled datasets — where the majority of the data points are normal and labeled, but anomalies (abnormal points) are scarce or even completely unlabeled.</p><p>It’s an extension and advancement of Deep SVDD (Support Vector Data Description), and addresses the challenges of accurately identifying rare or unseen anomalies.</p><h3>Why Use DeepSAD?</h3><p><strong>Anomalies</strong> are rare, unusual instances in datasets that deviate significantly from typical patterns. Traditionally, anomaly detection methods either rely purely on unsupervised techniques (no labels at all) or supervised methods (requiring extensive labeled data). However, in real-world applications, labeled anomaly examples are often scarce, expensive, or unavailable. DeepSAD bridges this gap by utilizing a <strong>semi-supervised learning approach</strong> that leverages primarily normal labeled data and few (or no) labeled anomalies.</p><p>These are some of the strengths of DeepSAD:</p><ul><li><strong>Effective in limited-label scenarios</strong>: Can learn robust anomaly detection even with minimal labeled anomalies.</li><li><strong>Improved performance</strong>: Demonstrates higher accuracy compared to purely unsupervised methods (like Autoencoders or Isolation Forest) and supervised approaches requiring extensive labeled data.</li><li><strong>End-to-end learning</strong>: Capable of extracting meaningful features from raw data (e.g., images, sensor data, text), avoiding reliance on handcrafted features.</li></ul><h3>How Does DeepSAD work?</h3><p>DeepSAD extends Deep SVDD.</p><blockquote><strong>Deep SVDD</strong></blockquote><p>It learns a neural network that maps normal data points to a compact hypersphere in a latent space, aiming to minimize the sphere’s radius. Points outside this sphere are considered anomalies.</p><p>Deep SVDD minimizes the following loss function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UcivfKeDLWFV13ICP4Bb2w.png" /></figure><p><strong>λ </strong><em>(Regularization parameter),</em> controls model complexity. Higher λ means simpler model (more regularized weights), which reduces overfitting but can underfit.</p><p><strong><em>c </em></strong>(center), the hypersphere center c is set as the mean of the network outputs obtained from an initial forward pass of the data after the initialization.</p><p>Once the network is trained, the anomaly score for a test point x is given by the distance from φ(x; W) to the center of the hypersphere: s(x) = ||φ(x; W) − c||</p><p>After training your model to embed normal labeled samples close to center<strong> <em>c</em></strong>, you evaluate a <strong>n</strong>ew sample x_new​​ as follows:</p><ol><li>Pass the new sample through the trained model:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zYhTJ7Hgn8KfXCMEI2_Wtw.png" /><figcaption>This gives its position in the latent (embedding) space.</figcaption></figure><p>2. Compute distance from the center:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2mHXYJboUZpX6QuutJY9fQ.png" /></figure><p>3. Compare this distance to a threshold <strong>τ</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nWf45V-UqBEdF8Ericwerw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gUT77C1ThJVQxFvNSFQKDg.png" /><figcaption>Ruff, Lukas, et al. “Deep one-class classification.” International conference on machine learning. 2018.</figcaption></figure><p>Figures 2 and 3 illustrate the most normal and most anomalous in-class samples identified by Deep SVDD and KDE, respectively. Interestingly, One-Class Deep SVDD slightly outperforms its soft-boundary variant on both datasets — likely because the assumption of a clean (anomaly-free) training set holds in this case. Figure 4 highlights the most anomalous samples detected by One-Class Deep SVDD, which include adversarial attacks and incorrectly cropped or oddly angled images [<a href="https://proceedings.mlr.press/v80/ruff18a.html">1</a>].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*Ra3g4tu1HJi6oFgwtfVBrQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*40-g9SIMhw7cc5PNl4A7gw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*d-TJvFRWHNFHMoFL1BpsnA.png" /><figcaption>Ruff, Lukas, et al. “Deep one-class classification.” International conference on machine learning. 2018.</figcaption></figure><blockquote>Deep SAD</blockquote><p>DeepSad introduces a semi-supervised loss function to explicitly leverage any available labeled anomalies.</p><p>The model learns a representation where: <br> → <strong>Normal samples</strong> cluster close to a predefined center in latent space.<br> → <strong>Anomalous samples</strong> are explicitly pushed further away from this center.</p><p>The resulting model learns a clear boundary that effectively separates normal from anomalous data, significantly improving detection accuracy.</p><p>DeepSAD minimizes the following loss function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lzJVpzAVyS4ofYOwoeBHzQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PbnkzliJdXpIsQlzhldLoQ.png" /></figure><p>This new loss term introduced in DeepSAD is weighted via the hyperparameter η &gt; 0 which controls the balance between the labeled and the unlabeled samples and how strongly the labeled samples influence the training.</p><p>Setting η &gt; 1 puts more emphasis on the labeled data whereas η &lt; 1 emphasizes the unlabeled data. One might consider η &gt; 1 to emphasize labeled normal over unlabeled samples.</p><ul><li>High η → labeled samples have more impact, strong anomaly detection performance if labeled anomalies are reliable.</li><li>Low η → labeled samples less impactful, relying more on unlabeled samples.</li></ul><p>In the paper, it suggests simply setting η = 1 yields a consistent and substantial performance improvement.</p><p>For labeled normal samples (𝑦̃ = +1), the model applies a <strong>quadratic loss</strong>(euclidean loss) to the distance between each sample’s embedding ϕ(xi; W) and a reference center point c in the latent space. This loss has the form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jBnxQJqJfkXM71DgGMtgCA.png" /></figure><p>This means the farther a normal sample is from the center, the higher the loss. By minimizing this loss during training, the model is encouraged to map all <strong>normal samples close to the center</strong> <em>c</em>, effectively creating a <strong>compact cluster</strong> of normal data in the latent space. This compactness helps the model later distinguish anomalies: if a new sample maps <strong>far from the center</strong>, it’s likely to be <strong>anomalous</strong>.</p><p>Diving deeper into the loss function, the loss comprises three different terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8yVTEsYtrA0U2iavaNIYxw.png" /></figure><p>First term is for unlabeled data term, which <strong>minimizes the distance</strong> of <strong>unlabeled samples to the center <em>c</em></strong>. It assumes most unlabeled data points are normal. The network tries to cluster these data points around <em>c</em>, making a compact normal class representation.</p><p>In practice, unlabeled datasets often include a few anomalies, which are typically rare and unidentified. However, the anomalies, due to their intrinsically different structure or features, tend to resist fitting into this tight cluster. They <strong>don’t minimize as easily</strong> as normal samples because their features are significantly different, causing them to remain somewhat distant from the center.</p><p>While the model tries to reduce distances, anomalies generally remain further away, becoming less tightly clustered. Anomalies often appear at the boundary regions or outskirts of the normal data cluster.</p><p>Once the training completes, latent space structure naturally reflects this dynamic:</p><ul><li>A tight, dense cluster around <strong><em>c </em>(</strong>mostly normal points).</li><li><strong>Unlabeled anomalies</strong> generally remain <strong>further away</strong>, in peripheral regions, because they resisted full integration into the normal cluster.</li></ul><p>Therefore, even if anomalies were unlabeled during training, the <strong>trained model can still detect them</strong> later based on distances in latent space.</p><p>The factor 1/(n+m)​​ is a normalization term that computes the average loss per data point, considering all samples together (both labeled and unlabeled). Total number of samples = n+m. The sum terms are divided by n+m, ensuring the <strong>loss is scaled to an average.</strong></p><p>By dividing by total samples, both labeled and unlabeled data contribute proportionally to the loss, creating balanced influence. Without this normalization, larger groups (e.g., unlabeled data) might dominate training disproportionately.</p><p>Second term is a labeled data term, which introduces <strong>explicit supervision from labeled samples</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S3nczd071lvgH1n_7qgXgw.png" /></figure><p>The multiplication by <strong>𝑦̃ⱼ </strong>ensures labeled anomalies explicitly push away from the center <em>c</em>.</p><ul><li>If <strong>𝑦̃ⱼ = </strong>=+1 (normal sample), minimizes distance (pulls closer to <em>c</em>). Loss increases if distance is large and optimizer reduces distance.</li><li>If <strong>𝑦̃ⱼ = </strong>= -1 (anomalous sample), maximizes distance (pushes away from <em>c</em>). Minimizing below expression is equivalent to maximizing the denominator ∥ϕ(x<strong>ⱼ</strong>;W)−c∥². Thus, the optimization pushes anomalous points away from center c.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*D-MLNIY4k3_wvVDWku08_g.png" /></figure><p>For <strong>anomalies</strong> (<strong>𝑦̃ⱼ = </strong>= -1 ), the term is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zNF7fY6L0-wQE6UVpS0pfw.png" /></figure><p>To minimize the loss, the neural network has two choices:</p><ol><li><strong>Decrease numerator:</strong> Impossible, since numerator = 1 (fixed).</li><li><strong>Increase denominator:</strong> The network increases the distance ∥ϕ(x<strong>ⱼ</strong>​;W)−c∥², pushing anomalies away.</li></ol><p>Thus, anomalies are <strong>explicitly pushed outward</strong>.</p><ul><li>If <strong>𝑦̃ⱼ = </strong>= 0 (unlabeled sample), minimizes distance cautiously, gradually moves closer to the center. The training is careful not to overly assume all unlabeled points as normal, as there might be anomalies present.</li></ul><p>Third term, is a standard weight decay regularization to <strong>prevent overfitting</strong> and ensure a smoother optimization landscape. It explicitly adds a cost to larger neural network weights.</p><p><strong>Overfitting</strong> happens when a neural network learns training data too closely, including noise or irrelevant details, losing generalization to new, unseen data. B. Minimizing the third term pushes weights closer to zero, effectively shrinking their magnitude.</p><p>When we say we <strong>minimize</strong> the regularization term, we directly minimize the sum of squared weights:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*hjqrv8zygW4UGM5qgV6jrg.png" /></figure><p>Each individual weight <em>Wℓ</em> ​ contributes positively (square) to this term. To minimize the sum of squares, the <strong>optimal solution</strong> is to make each term as close to zero as possible, because:</p><ul><li><strong>If the weight is large</strong>, say <em>W </em>= 10, the contribution to loss is 1⁰² = 100.</li><li><strong>If the weight is small</strong>, say <em>W </em>= 0.1, the contribution is 0.¹²= 0.01, which is much smaller.</li></ul><p>The minimum possible contribution from each weight is zero (⁰²=0), thus weights will naturally shrink toward zero.</p><p>Without regularization (λ=0) the weights can grow large and model tries excessively to minimize training error and the latent representations become overly specific to training set features including noise.</p><p>During neural network training, <strong>gradient descent</strong> updates the weights to minimize the total loss (including regularization):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*6oz9-5tvvouwh2qMXPhbyw.png" /></figure><p>The derivative of a single weight is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*zoTIExReVUjIWloQUiqozA.png" /></figure><p>Thus, the gradient update for regularization alone is proportional to the weight itself:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*p5aZ6fDehNFbsNvWFtfUIA.png" /></figure><p>Here, α is the learning rate, and λ scales how strongly regularization affects the update. Each weight is directly subtracted by a small fraction proportional to itself. Large weights have larger corrections, shrinking them significantly. Small weights have tiny corrections, gently moving them closer to zero.</p><h3>Effective on Unreliable Dataset</h3><p>In real-world scenarios, datasets are often noisy, and labeling is both expensive and prone to human error. Even with a small amount of reliably labeled data, it’s possible to train models effectively for anomaly detection. The table below demonstrates how Deep SAD outperforms other methods as the proportion of labeled anomalies increases, highlighting its effectiveness in anomaly detection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6pFyEOwmS0JhvM7awHx6mw.png" /><figcaption><a href="https://proceedings.mlr.press/v80/ruff18a.html">Ruff, Lukas, et al. “Deep one-class classification.” <em>International conference on machine learning</em>. 2018.</a></figcaption></figure><p>This approach can also be reversed with little bit of tweak: if we’re more confident about what constitutes an anomaly — but uncertain about what “normal” looks like — we can train the model using only labeled anomalies and treat the rest as unlabeled. During training, the model learns to push anomalies away from the center of the latent space. Samples that differ significantly from these anomalies can then be interpreted as normal, while unlabeled samples that fall close to the anomaly cluster are likely to be anomalous as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ndDyo2zU1b9440MZubUAZA.png" /></figure><p>With this tweak in place, a PyTorch implementation will be covered in the next article!</p><h4>Reference</h4><p>[1]<a href="https://proceedings.mlr.press/v80/ruff18a.html"> Ruff, Lukas, et al. “Deep one-class classification.” <em>International conference on machine learning</em>. 2018.</a></p><p>[2] <a href="https://arxiv.org/pdf/1906.02694.pdf">Ruff, Lukas, et al. “Deep semi-supervised anomaly detection.” <em>arXiv preprint arXiv:1906.02694</em> (2019).</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f45388a48208" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformer Variant for Anomaly Detection in Sequential Data]]></title>
            <link>https://skirene.medium.com/transformer-variant-for-anomaly-detection-in-sequential-data-00c22c70b222?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/00c22c70b222</guid>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Thu, 30 Nov 2023 08:49:29 GMT</pubDate>
            <atom:updated>2023-11-30T08:49:29.724Z</atom:updated>
            <content:encoded><![CDATA[<p>Detecting behavioral patterns in sequential or time-series datasets is crucial for identifying anomalies, such as cheating in my case study. Many abusive behaviors occur in video games, and it’s impractical for humans to monitor every match. Therefore, I develop deep learning models to automatically detect these abusive behaviors. One of the most effective models for handling sequential data is the transformer, which forms the foundation of the Generative Pre-trained Transformer (GPT) used in ChatGPT.</p><p>Recently, I adapted a transformer variant for detecting abusive behaviors in my project. The transformer architecture consists of an encoder layer and a decoder layer. However, since my task involves classification, the decoder layer is not required. Therefore, I replaced the decoder layer with a bidirectional LSTM, enabling more effective capture of both global and local contexts. The following is the architecture of the transformer variant that I implemented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6zDpvyd3vVHDejsatwyECw.png" /><figcaption>Transformer variant that replaced the decoder with bidirectional LSTM</figcaption></figure><p>Instead of passing the output of the encoder layer to the decoder layer, in my approach, it goes through a bidirectional LSTM. Subsequently, for binary classification, a sigmoid function is used rather than a softmax function. This modification caters specifically to the requirements of binary classification tasks.</p><p>Another critical aspect to note is that my work involves sequential tabular data. Thus, there’s no necessity for an embedding layer, which is typically essential for transforming non-numerical data, like natural language, into a numerical format. Since my data is already in a numerical form, this transformation step becomes redundant.</p><pre>class PositionalEncoding(nn.Module):<br>    def __init__(self, d_model, dropout=0.1, max_len=256, device=&#39;cpu&#39;):<br>        super(PositionalEncoding, self).__init__()<br>        self.dropout = nn.Dropout(p=dropout)<br>        <br>        pe = torch.zeros(max_len, d_model)<br>        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)<br>        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))<br>        pe[:, 0::2] = torch.sin(position * div_term)<br>        pe[:, 1::2] = torch.cos(position * div_term)<br>        pe = pe.unsqueeze(0).transpose(0, 1)<br>        self.register_buffer(&#39;pe&#39;, pe) # register as non-learnable parameter<br>        self.d_model = d_model #number of features in the embeddings<br>        self.pe = self.pe.to(device)<br>        <br>    def forward(self, x):<br>        # print(&quot;Shape of x:&quot;, x.shape)<br>        # print(&quot;Shape of positional encoding:&quot;, self.pe.shape)<br>        x = x + self.pe[:x.size(0), :, :] # adding positional encoding along the sequence len<br>        return self.dropout(x)<br><br><br>class LSTM_Transformer_BinaryClassifier(nn.Module):<br>    def __init__(self, d_model, nhead, num_layers, dim_feedforward, max_len, dropout, lstm_hidden_size, lstm_n_layers, device=&#39;cpu&#39;):<br>        super(LSTM_Transformer_BinaryClassifier, self).__init__()<br>        # feature_dim (embedding dimension) is omited since data is already embedded<br>        self.d_model = d_model<br><br>        self.lstm = nn.LSTM(d_model, lstm_hidden_size, lstm_n_layers, batch_first=True, bidirectional=True)<br>        self.fc_lstm = nn.Linear(lstm_hidden_size * 2, d_model)  <br><br>        self.pos_encoder = PositionalEncoding(d_model, dropout, max_len, device=device)<br>        encoder_layers = nn.TransformerEncoderLayer(d_model, nhead, dim_feedforward, dropout)<br><br>        self.transformer_encoder = nn.TransformerEncoder(encoder_layers, num_layers)<br><br>        self.classifier = nn.Linear(d_model, 1)  # 1 output for binary classification<br>        <br>        self.init_weights()<br><br>    def init_weights(self):<br>        initrange = 0.1<br>        self.classifier.bias.data.zero_()<br>        self.classifier.weight.data.uniform_(-initrange, initrange)<br><br>    def forward(self, src, src_mask=None):<br>        # Positional encoding<br>        src = src.permute(1, 0, 2)  # data_collate padding result shape : [batch_size, sequence_length, feature_dim] =&gt; [sequence_length, batch_size, feature_dim] <br>        src = src * math.sqrt(self.d_model)<br>        src = self.pos_encoder(src)<br><br>        transformer_out = self.transformer_encoder(src, src_key_padding_mask=src_mask)<br><br>        # Pass through LSTM after Transformer<br>        transformer_out = transformer_out.permute(1, 0, 2)  # [batch_size, sequence_length, feature_dim]<br>        lstm_out, _ = self.lstm(transformer_out)<br>        output = self.fc_lstm(lstm_out[:, -1, :])  # Take the output of the last timestep from LSTM<br>        <br>        output = self.classifier(output)<br>        output = torch.sigmoid(output)<br>        output = output.squeeze(-1)  <br><br>        return output</pre><p>The placement of the LSTM layer depends on your specific task. It’s advisable to experiment with both placements to see which works better for your project. Below is the code for positioning the LSTM layer before the transformer.</p><pre>class LSTM_Transformer_BinaryClassifier(nn.Module):<br>    def __init__(self, d_model, nhead, num_layers, dim_feedforward, max_len, dropout, lstm_hidden_size, lstm_n_layers, device=&#39;cpu&#39;):<br>        super(LSTM_Transformer_BinaryClassifier, self).__init__()<br>        # feature_dim (embedding dimension) is omited since data is already embedded<br>        self.d_model = d_model<br><br>        self.lstm = nn.LSTM(d_model, lstm_hidden_size, lstm_n_layers, batch_first=True, bidirectional=True)<br>        self.fc_lstm = nn.Linear(lstm_hidden_size * 2, d_model)  <br><br>        self.pos_encoder = PositionalEncoding(d_model, dropout, max_len, device=device)<br>        encoder_layers = nn.TransformerEncoderLayer(d_model, nhead, dim_feedforward, dropout)<br><br>        self.transformer_encoder = nn.TransformerEncoder(encoder_layers, num_layers)<br><br>        self.classifier = nn.Linear(d_model, 1)  # 1 output for binary classification<br>        <br>        self.init_weights()<br><br>    def init_weights(self):<br>        initrange = 0.1<br>        self.classifier.bias.data.zero_()<br>        self.classifier.weight.data.uniform_(-initrange, initrange)<br><br>    def forward(self, src, src_mask=None):<br>        # Pass through LSTM before Transformer<br>        lstm_out, _ = self.lstm(src)<br>        src = self.fc_lstm(lstm_out)<br><br>        # Positional encoding<br>        src = src.permute(1, 0, 2)  # data_collate padding result shape : [batch_size, sequence_length, feature_dim] =&gt; [sequence_length, batch_size, feature_dim] <br>        src = src * math.sqrt(self.d_model)<br>        src = self.pos_encoder(src)<br>        output = self.transformer_encoder(src, src_key_padding_mask=src_mask)<br>       <br>        output = output[-1, :, :]<br><br>        output = self.classifier(output)<br>        output = torch.sigmoid(output)<br>        output = output.squeeze(-1)  <br><br>        return output</pre><p>Depending on your data preprocessing, you may need to adjust the shapes for ‘src’ and ‘output’. Although the default device for training the model is set to ‘cpu’, using a GPU is recommended for training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tRiSyYMbh5Ds2JXrkQsLaA.png" /><figcaption>Train and Validation Loss</figcaption></figure><p>The model trained stably and performed well for my task, outperforming the use of either the Transformer alone or the bidirectional LSTM alone. This model is now being used to automatically detect abusive users!</p><h4><a href="https://mlearning.substack.com/about">WRITER at MLearning.ai</a> / 28<a href="https://open.substack.com/pub/mlearning/p/curated-list-of-15k-gpts-agents-real-time-update?r=z7zu8&amp;utm_campaign=post&amp;utm_medium=web">K+ GPTs</a> / <a href="https://open.substack.com/pub/mlearning/p/it-is-time-to-accelerate-gpt-alternatives?r=z7zu8&amp;utm_campaign=post&amp;utm_medium=web">GPT alternatives</a> / <a href="https://open.substack.com/pub/evartology/p/custom-gpts-risk-hacking-store-openai-chatgpt?r=9hp4d&amp;utm_campaign=post&amp;utm_medium=web">GPTs Hacks</a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=00c22c70b222" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AutoEncoder for Abusing Detection]]></title>
            <link>https://skirene.medium.com/autoencoder-for-abusing-detection-5392018163f5?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/5392018163f5</guid>
            <category><![CDATA[deeplearing]]></category>
            <category><![CDATA[anomaly-detection]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Thu, 27 Apr 2023 12:57:32 GMT</pubDate>
            <atom:updated>2023-04-27T12:57:32.741Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using Deep Learning Models to Detect Abusive Behavior in Video Games</h4><p>Abusive behaviors are a pervasive issue in numerous business settings, including credit card companies, video games, and promotional events. One example is the creation of multiple accounts and their use in conjunction with macros to achieve a certain level in the game. This is not only unfair for other players but also detrimental to the reputation and long-term profitability of the companies involved. To address such abusive behaviors, there exist various deep learning models that can effectively mitigate this problem.</p><p>If you have a labeled dataset, we can explore multiple models in various forms. One example that I would like to share is the Autoencoder model, which can be used to detect abusive users with sequential datasets. The dataset consists of sequential data that includes information about each match and its corresponding gameplay details. As the dataset is confidential, I am unable to share it, but I can provide the implementation codes.</p><blockquote><em>Autoencoder is a type of unsupervised learning in which the model is trained to reconstruct its input data as output while minimizing the reconstruction error. The model consists of two parts: an encoder that maps the input data to a lower-dimensional feature representation, and a decoder that maps the lower-dimensional representation back to the original data space. </em><strong>The goal of the model is to generate an output that is as similar as possible to the input, given the lower-dimensional feature representation</strong><em>.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I5MVGIrROrAnD3U_2Jm1Ng.png" /><figcaption>Image source: Weng, L. (2018). “Tutorial — What is a Variational Autoencoder?.” Retrieved from <a href="https://lilianweng.github.io/posts/2018-08-12-vae/">https://lilianweng.github.io/posts/2018-08-12-vae/</a>.</figcaption></figure><p>Since abusive behavior is often highly imbalanced, it can be challenging to rely solely on deep learning models like RNN-based models without implementing additional techniques. However, by using an Autoencoder model to detect abusive users with sequential datasets, we can avoid the need for balancing the dataset. By training the Autoencoder to minimize the loss between the input dataset and final output, <strong><em>we can expect that the lower the loss, the more likely it is that the user is engaging in abusive behavior</em></strong>.</p><p>Implementing an Autoencoder with Pytorch is relatively easy. The code below includes the encoder and decoder blocks for the Autoencoder.</p><p>You need to specify:<br>1. number of features (n_features)<br>2. hidden node size (n_hidden)<br>3. number of layers (n_layers), <br>4. dropout<br>5. set bidirectional to true (bidirectional=True) if you want to use bidrectional LSTM model.</p><pre>class Autoencoder(AbsuingModels):<br>    def __init__(self, n_features, n_hidden, n_layers, dropout, bidirectional):<br>        super(Autoencoder, self).__init__()<br>        <br>        self.encoder = nn.LSTM(input_size=n_features, hidden_size=n_hidden, num_layers=n_layers, dropout=dropout, bidirectional=bidirectional, batch_first = True)<br>        self.encoder_2 = nn.LSTM(input_size=n_hidden, hidden_size=int(n_hidden*0.5), num_layers=n_layers, dropout=dropout, bidirectional=bidirectional, batch_first=True)<br>        self.decoder_2 = nn.LSTM(input_size=int(n_hidden*0.5), hidden_size=n_hidden, num_layers=n_layers, dropout=dropout, bidirectional=bidirectional, batch_first=True)<br>        self.decoder =  nn.Linear(in_features=n_hidden, out_features=n_features)<br>        <br>    def forward(self, x, mask):<br>        batch_size, sequence_length, n_features = x.size()<br>        <br>        # Apply mask to ignore padded values during training<br>        if mask is not None:<br>            x = x * mask.unsqueeze(-1)<br>            <br>        # Encoder<br>        encoded_seq, _ = self.encoder(x)<br>        encoded_seq, _ = self.encoder_2(encoded_seq)<br>        <br>        # Decoder<br>        decoded_seq, _ = self.decoder_2(encoded_seq)<br>        decoded_seq = decoded_seq.contiguous().view(-1, decoded_seq.shape[-1])  # flatten sequence length and batch size dimensions<br>        decoded_seq = self.decoder(decoded_seq)<br>        decoded_seq = decoded_seq.view(batch_size, sequence_length, -1)  # restore original dimensions<br>        <br>        # Apply mask to ignore padded values during loss calculation<br>        if mask is not None:<br>            decoded_seq = decoded_seq * mask.unsqueeze(-1)<br>            <br>        return decoded_seq</pre><p>The first code block contains helper functions that can be called from other files, located in acm.util.utils.</p><pre># These are helper functions inside the acm.util.utils. <br><br>class SmallTorchLogger:<br>    def __init__(self):<br>        self.best_loss = float(&#39;inf&#39;)<br>        self.best_loss_epoch = None<br>        self.best_loss_model = None<br>    <br>    def update_log(self, epoch, loss, model):<br>        if loss &lt; self.best_loss:<br>            self.best_loss_epoch = epoch<br>            self.best_loss = loss<br>            self.best_loss_model = deepcopy(model)<br><br>    def reset_log(self):<br>        self.best_loss = float(&#39;inf&#39;)<br>        self.best_loss_epoch = None<br>        self.best_loss_model = None<br><br>def log_loss_plot(losses, bins=50, kde=True):<br>    loss_plot = sns.distplot(losses, bins, kde=True)<br>    plt.savefig(&quot;ReconstructionError.png&quot;)<br>    plt.close()<br>    mlflow.log_artifact(&quot;ReconstructionError.png&quot;)<br>    os.remove(&quot;ReconstructionError.png&quot;)<br><br>def epoch_time(start_time, end_time):<br>  elapsed_time = end_time - start_time<br>  elapsed_mins = int(elapsed_time / 60)<br>  elapsed_secs = int(elapsed_time - (elapsed_mins * 60))<br>  return elapsed_mins, elapsed_secs<br><br>def log_autoencoder_result_file(threshold, ac_list, mc_list, cc_list, pred_losses, y_list):<br>    zipped = list(zip(ac_list, mc_list, cc_list, pred_losses, y_list))<br>    output_df = pd.DataFrame(zipped, columns=[&#39;accountid&#39;, &#39;match_count&#39;, &#39;isCC&#39;, &#39;pred_losses&#39;, &#39;label&#39;])<br>    output_df.to_parquet(&#39;result.parquet&#39;, index=False)<br>    count_anomaly = len(output_df[output_df[&#39;pred_losses&#39;] &gt; threshold])<br>    mlflow.log_metric(&#39;count_anomaly_predictions&#39;, count_anomaly)<br>    mlflow.log_artifact(&#39;result.parquet&#39;)<br>    os.remove(&quot;result.parquet&quot;)</pre><pre># Training Autoencoder <br><br>import torch<br>import mlflow<br>import time<br>import logging<br>import seaborn as sns <br>from acm.util.utils import SmallTorchLogger, epoch_time, log_loss_plot<br><br>def train_autoencoder( <br>          train_dataloaders, <br>          valid_dataloaders,<br>          model,<br>          device,<br>          epochs,<br>          loss_fn_type, <br>          optimizer_type,<br>          optimizer_params, <br>          scheduler_type,<br>          scheduler_params,<br>          ):<br>    <br>    optimizer = getattr(torch.optim, optimizer_type)(model.parameters(), **optimizer_params)<br>    <br>    scheduler = getattr(torch.optim.lr_scheduler, scheduler_type)(optimizer, **scheduler_params)<br><br>    loss_fn = getattr(torch.nn, loss_fn_type)()<br>    <br>    <br>    model = model.to(device)<br>    torch_logger = SmallTorchLogger()<br>    <br>    for epoch in range(epochs):<br>        running_loss = 0.0<br>        metrics = {<br>            &#39;train_loss&#39;: 0,<br>            &#39;valid_loss&#39;: 0,<br>            &#39;lr&#39;: 0,<br>        }<br>        <br>        start_time = time.time()<br>        <br>        model.train()<br>        for idx, batch in enumerate(train_dataloaders):<br>            x = model.preprocess(batch[0]).to(device)<br>            mask = batch[5].to(device)      <br>            <br>            optimizer.zero_grad()<br>            <br>            decoded_seq = model(x, mask)<br>            <br>            model_params = list(model.parameters())<br>            for param in model_params:<br>                param.to(device)<br>                <br>            loss = loss_fn(decoded_seq, x)<br>            <br>            loss.backward()<br>            optimizer.step()<br>            <br>            running_loss += loss.item()<br>            metrics[&#39;train_loss&#39;] += loss.item()<br>            # scheduler.step()<br>        <br>        metrics[&#39;train_loss&#39;] = metrics[&#39;train_loss&#39;] / (idx + 1)<br>        <br>     # Validation loop<br>        val_loss = 0.0<br>        with torch.no_grad():<br>            model.eval()<br>            for (idx, batch) in enumerate(valid_dataloaders):<br>                x_val = model.preprocess(batch[0]).to(device)<br>                mask_val = batch[5].to(device)<br>                <br>                decoded_seq_val = model(x_val, mask_val)<br>                loss_val = loss_fn(decoded_seq_val, x_val)<br>                val_loss += loss_val.item()<br>                metrics[&#39;valid_loss&#39;] += loss_val.item()<br>            <br>            end_time = time.time()    <br>            epoch_mins, epoch_secs = epoch_time(start_time, end_time)<br>            metrics[&#39;valid_loss&#39;] = metrics[&#39;valid_loss&#39;] / (idx + 1) #len(valid_dataloaders)<br>            metrics[&#39;lr&#39;] = scheduler.optimizer.param_groups[0][&#39;lr&#39;]<br>            torch_logger.update_log(<br>                epoch=epoch,<br>                loss=metrics[&#39;valid_loss&#39;],<br>                model=model,<br>            )    <br>        scheduler.step(metrics[&#39;valid_loss&#39;])<br>        logging.info(f&#39;Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s&#39;)<br>        logging.info(f&quot;\tTraining Loss: %.4f. Validation Loss: %.4f. &quot; % (metrics[&#39;train_loss&#39;], metrics[&#39;valid_loss&#39;]))<br>        <br>        mlflow.log_metrics(metrics, epoch)<br>              <br>    logging.info(&#39;-&#39;*20 + f&quot;TRAINING DONE&quot; + &#39;-&#39;*20)<br>    <br>    mlflow.pytorch.log_model(<br>        model.to(&#39;cpu&#39;),<br>        &quot;model_latest&quot;<br>    )<br>    <br>    mlflow.pytorch.log_model(<br>        torch_logger.best_loss_model.to(&#39;cpu&#39;),<br>        f&quot;model_best_loss_{torch_logger.best_loss_epoch}ep&quot;<br>    )<br>    <br>    torch.cuda.empty_cache()<br>    # final_model = torch.nn.DataParallel(torch_logger.best_loss_model.to(device))<br>    final_model = torch_logger.best_loss_model<br>    predict(final_model, train_dataloaders, loss_fn, device=&#39;cpu&#39;) #&#39;cpu&#39;<br>    <br>    <br>def predict(model, dataloader, loss_fn, device):<br>    model_device = next(model.parameters()).device<br>    predictions, losses = [], []<br>    with torch.no_grad():<br>        model.eval()<br>        for inputs in dataloader:<br>            x = model.preprocess(inputs[0]).to(model_device)<br>            mask= inputs[5].to(device)<br>            <br>            seq_pred = model(x, mask)<br>            seq_loss = loss_fn(seq_pred, x)<br>            <br>            predictions.append(seq_pred)<br>            losses.append(seq_loss.item())<br>            <br>    log_loss_plot(losses, bins=100, kde=True)</pre><pre># Inference code for AutoEncoder<br><br>import mlflow<br>import torch<br>import pandas as pd<br>from acm.util.utils import log_autoencoder_result_file, log_loss_plot<br><br>def inference_autoencoder(<br>    inference_dataloader,<br>    model,<br>    device,<br>    **kwargs,<br>):<br>    loss_fn_type = kwargs[&#39;loss_fn_type&#39;]<br>    loss_fn = getattr(torch.nn, loss_fn_type)()<br><br>    threshold = kwargs[&#39;threshold&#39;]<br>        <br>    y_list = []<br>    cc_list = []<br>    ac_list = []<br>    mc_list = []<br>    # predictions = []<br>    pred_losses = []<br>    <br>    model = model.to(device)<br>    with torch.no_grad():<br>        model.eval()<br>        for inputs in inference_dataloader:   <br>            x = model.preprocess(inputs[0]).to(device)<br>            y = inputs[1].to(device)<br>            cc = inputs[2].to(device)<br>            ac = inputs[3]<br>            mc = inputs[4].to(device)<br>            mask= inputs[5].to(device)<br>            <br>            seq_pred = model(x, mask)<br>            seq_loss = [loss_fn(seq_pred[i], x[i]).cpu().item() for i in range(seq_pred.size(0))]<br><br>            pred_losses.extend(seq_loss)<br>            y_list.extend(y.data.cpu().tolist())<br>            cc_list.extend(cc.data.cpu().tolist())<br>            ac_list.extend(ac)<br>            mc_list.extend(mc.data.cpu().tolist()) <br>     <br>    # log_loss_plot(pred_losses, bins=100, kde=True)<br>    log_autoencoder_result_file(threshold, ac_list, mc_list, cc_list, pred_losses, y_list)</pre><p>The histogram below shows the reconstruction loss results for the Autoencoder. It appears that the majority of abusive users have a loss of less than 0.0003.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*3OhvkmeiK3lSVsk4WRp4YA.png" /></figure><p>his value can be used to detect users or players with abnormal patterns. For example, normal users will typically have a higher loss compared to abusive users, since the Autoencoder model has learned to replicate abusive behaviors but not the reverse.</p><p>Therefore, when using the Autoencoder model for anomaly detection, we should look for very low loss values. Of course, it is important to carefully determine the threshold for your specific use case, but overall, the model works quite effectively!</p><p>For those who are curious, here is a chart I created to show different types of Autoencoders and their variants:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6fem1PmMV55-nWShbkML-w.png" /></figure><h4><a href="https://mlearning.substack.com/about"><strong>BECOME a WRITER at MLearning.ai</strong></a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5392018163f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handling Noisy Label Data with Deep Learning]]></title>
            <link>https://skirene.medium.com/handling-noisy-label-data-with-deep-learning-ff986deedc76?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff986deedc76</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Sun, 17 Jul 2022 04:50:29 GMT</pubDate>
            <atom:updated>2022-11-26T04:21:21.035Z</atom:updated>
            <content:encoded><![CDATA[<p>When your dataset is coarsely labelled, it isn&#39;t easy to actually perform any deep learning techniques on top of such data. The result is not trustworthy. However, if you are mostly sure that the majority of your data are labelled correctly but some are not, then you can just feed the data into the deep learning model and use the noise correction techniques. So here are the steps that you may want to refer to when handling a noisy label dataset:</p><h4><strong>1. Use the Deep Learning Model than Traditional ML Models</strong></h4><p>From its nature, deep learning models can handle 10–20% of noise in the training dataset. However, traditional machine learning models such as XGBoost or any tree-based model will quickly break when trained with a noisy training dataset. The tree-based model such as XGBoost and LGBM is known for their powerful performance in handling tabular data. Historically these models have won many Kaggle competitions. Nevertheless, they may not be quite suitable for handling real-world noisy datasets.</p><p>Here is a table below that can show how XGBoost drops its performance when trained on the noisy dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*1JjkefzYFk32LTjE8cI_1w.png" /><figcaption>Table 1</figcaption></figure><p>The noisy dataset was constituted by injecting 20% noise into the clean dataset. Samples from a positive class (1) were deliberately changed to a negative class (0). Thus, the dataset is now synthetically generated with one-sided noise. The two-sided noise dataset was created by inserting noise on both classes. Approximately 15% of samples from class 0 were converted to class 1 and vice versa. The result is 30% of the noise in the dataset. Quite interestingly, XGBoost performed slightly better on the two-sided noise dataset which contained more noise compared to the one-sided noise dataset. A hypothesis to explain such a phenomenon is that one-sided noise only moves the decision boundary of the tree-based model to one side whereas the two-sided noise dataset will shift the boundary on both sides, resulting in no changes in the decision boundary. The takeaway from the table above is that the noise doesn’t impact DNN’s performance that much. However, the performance of XGBoost quickly deteriorates as noise is introduced into the training dataset.</p><h4>2. Create Clean Dataset for Validation and Test set</h4><p>Deep learning models can handle noise in the training dataset quite well as long as there are enough datasets and the proportion of noise in the dataset isn’t overwhelming. However, we need to have a 100% clean dataset for validation and test set so that the model’s performance can be measured in a trustworthy manner. The size of the validation and test set doesn’t need to be large. It could be very small but requires very accurate labelling. In reality, creating such clean datasets may be difficult and expensive. What can we do in the case where we can do nothing about the dataset? Thankfully, there are methods to handle such cases.</p><h4>3. Play with the Loss Function</h4><p>When you can somehow manage to build accurate validation and test set, you have various options to choose from. For example, you can minimize the loss on a set of unbiased clean validation examples, which are consistent with the evaluation procedure. “<a href="https://arxiv.org/pdf/1803.09050.pdf">Learning to Reweight Examples for Robust Deep Learning</a>” is an awesome paper to implement the reweighting technique if you can build a validation dataset with no noise.</p><p>One way to approach the complicated case where all we have is a coarsely labelled dataset is to leverage a technique called <a href="http://proceedings.mlr.press/v97/shen19e/shen19e.pdf">Iterative Trimmed Loss Minimization</a>(<strong>ITLM</strong>). The beauty of ITLM is that it does not require any identified clean data. The idea of this paper is that the learning curve for the clean samples should be better than that of the bad samples in the early stages of training. The trimmed loss means jointly choosing a subset of <em>αn</em> samples θ such that it would achieve a minimum loss on the subset over all choices of subset and parameters. To minimize the loss, you alternate between<strong> (a) selecting samples with the lowest current loss</strong> and <strong>(b) retraining a model on these samples with the lowest loss at that moment</strong>. By iteratively performing the trimmed loss, you can filter out bad samples in the early stages. You may use early stopping to better select out the bad samples as in the later rounds everything will start to overfit.</p><p>Here is the TensorFlow implementation of ITLM:</p><pre>alpha = 0.95</pre><pre>class CustomModel(keras.Model):<br>    def train_step(self, data):<br>        # Unpack the data. Its structure depends on your model and<br>        # on what you pass to `fit()`.<br>        x, y = data<br>        _len = y.shape[0]<br>        _alpha = alpha<br>        _idx = math.floor(_len*_alpha)</pre><pre># forward for get loss from every samples<br>        y_pred = self(x, training=False)  # Forward pass<br>        loss_ITLM = keras.losses.binary_crossentropy(y, y_pred)<br>        loss_ITLM_np = loss_ITLM.numpy()<br>        argsort = np.argsort(loss_ITLM_np)</pre><pre>        # get new training data<br>        x, y = tf.gather(x, argsort[:_idx]), tf.gather(y, argsort[:_idx])</pre><pre>        with tf.GradientTape() as tape:<br>           y_pred = self(x, training=True)  # Forward pass<br>           # Compute the loss value<br>           # (the loss function is configured in `compile()`)<br>           loss = self.compiled_loss(y, y_pred, regularization_losses=self.losses)</pre><pre>        # Compute gradients<br>        trainable_vars = self.trainable_variables<br>        gradients = tape.gradient(loss, trainable_vars)<br>        # Update weights<br>        self.optimizer.apply_gradients(zip(gradients, trainable_vars))<br>        # Update metrics (includes the metric that tracks the loss)<br>        self.compiled_metrics.update_state(y, y_pred)<br>        # Return a dict mapping metric names to current value<br>        return {m.name: m.result() for m in self.metrics}</pre><p>Note that the alpha is the hyperparameter and the paper used 0.95.</p><p>You can build your own deep-learning model. I built four layered DNN model with batch normalization in between each fully connected layer.</p><pre>input_layer = Input(shape=(9,), name=&#39;input_layer&#39;)</pre><pre>Dense_1 = Dense(8, activation = &#39;relu&#39;, kernel_initializer = HeNormal(seed=1))(input_layer)</pre><pre>batch_norm = BatchNormalization(momentum = 0.9)(Dense_1)</pre><pre>Dense_2 = Dense(5, activation = &#39;relu&#39;, kernel_initializer = HeNormal(seed=1))(batch_norm)</pre><pre>batch_norm_2 = BatchNormalization(momentum = 0.9)(Dense_2)</pre><pre>Dense_3 = Dense(4, activation = &#39;relu&#39;, kernel_initializer = GlorotUniform(seed=1))(batch_norm_2)</pre><pre>batch_norm_3 = BatchNormalization(momentum = 0.99)(Dense_3)</pre><pre>Dense_4 = Dense(2, activation = &#39;relu&#39;, kernel_initializer = HeNormal(seed=1))(batch_norm_3)</pre><pre>batch_norm_4 = BatchNormalization(momentum = 0.9)(Dense_4)</pre><pre>output_layer = Dense(1, activation=&#39;sigmoid&#39;, kernel_initializer = GlorotUniform(seed=1))(batch_norm_4)</pre><p>Finally, compile your customized model equipped with ITLM. In TensorFlow, you can debug your model by setting ‘<em>run_eagerly</em>’ to true.</p><pre>model = CustomModel(input_layer, output_layer)</pre><pre>model.compile(loss=&#39;binary_crossentropy&#39;, optimizer=Adam(learning_rate=0.001),  metrics=[&#39;acc&#39;, &#39;Recall&#39;, &#39;AUC&#39;], run_eagerly=True)</pre><p>The table below summarizes the result trained and tested on the one-sided noisy dataset (note that it is a different dataset from Table 1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*ZJ7Zl74o7rRMRj65suUqRw.png" /></figure><p>The result indicates that utilizing the ITLM improves the overall accuracy and F1 score for the DNN model! When facing a coarsely labelled dataset issue, techniques such as ITLM can greatly help your model to learn from noisy dataset settings.</p><p><em>Reference<br></em>Ren, M., Zeng, W., Yang, B., &amp; Urtasun, R. (2018). Learning to Reweight Examples for Robust Deep Learning. <em>ICML</em>.</p><p>Shen, Y., &amp; Sanghavi, S. (2019). Learning with Bad Training Data via Iterative Trimmed Loss Minimization. <em>ICML</em>.</p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff986deedc76" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Neural Nets with The Shapley Value]]></title>
            <link>https://skirene.medium.com/demystifying-neural-nets-with-shapley-values-cca29c836089?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/cca29c836089</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[shap]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Sun, 05 Dec 2021 05:48:52 GMT</pubDate>
            <atom:updated>2021-12-05T06:01:35.957Z</atom:updated>
            <content:encoded><![CDATA[<h4>Unboxing The Black Box with The Shapley Value and Game Theory</h4><p>Explainability of deep learning is quickly getting its momentum despite its’ short history. Explainable AI derived from the rising demand for fairness and justice of neural networks’ decisions and to avoid coded bias. So-called black box AI could make assumptions and predictions of one entity based on a bias that resonates with the real world. Many techniques sprouted to mitigate such deleterious impact on people, especially on social minorities. The core function of these techniques is to explain the neural networks decision process and its behaviour. One of the most widely utilized tools to explain the model’s behaviour is called <a href="https://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html"><strong>feature importance</strong></a>. However, this turns out not to be the most trustworthy and robust method to reflect the model’s true behaviour. Here is why:</p><h3>Feature Importance</h3><ul><li>First and foremost let’s understand how feature importance operates. <strong>Feature importance measures increase in the model prediction error after permuting feature’s values</strong>. What this would mean is that the “<em>importance</em>” of a feature is <strong>defined by the model’s reliance on that feature</strong>. If the model error increases after shuffling the values of the feature, it implies that the feature played a big role in the prediction. Reversely, if the model error remains stagnant after shuffling, that feature would probably didn’t do much on the model’s decision.</li><li>However, feature importance only operates correctly on linear models. This is due to the nature of feature importance that it adds up both the main feature effect and the interaction effects with other features. Thus, any feature with a negative result is added up with a positive result. It’s like squashing the result into one big piece. The result is it that it doesn’t really reflect the individual negative and positive effects of the feature.</li><li>Additionally, adding correlated features makes the interpretation of the output more complex. Let’s say there was a single important feature ‘<em>treadmill’</em> for weight loss. Now I added ‘<em>stairmill</em>’, which is also effective for weight loss. Let’s also assume that the stairmill and treadmill are highly correlated. Now all of those two features are pulled down to the mediocre level in the feature importance plot instead of remaining in the top position. Such an outcome requires extra time and effort to interpret the result.</li></ul><p>The Shaley value is great alternative to overcome such drawbacks. So what exactly is the Shapley value?</p><h3>Shapley Value</h3><p>The Shapley value stems from game theory. Game theory is a study of interactive decision-making among rational agents. In the instance of the game, logical players <em>the decision-makers</em> would make strategic decisions depending on the actions of other players to win the game or achieve their desired goals. The Shapley value becomes handy when assessing how much each participant contributed to the result of the game. <strong>The value is calculated by weight averaging the marginal contribution of all agents across all possible coalitions</strong>. In machine learning, the player or the agent corresponds to the feature and the importance of that feature is computed with the Shapley value. Below is the equation to get the Shapley value. Looking at descriptions written in human language underneath the formula would be helpful to interpret the formula.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mLZ3UBCs56pe01lWou_fww.png" /><figcaption>Image by Author</figcaption></figure><p>Enough with all the mathematical symbols and let’s understand the value with a real-world example. There are two players and four different games. In the first game, both players did not play the game. In the second game, only player 1 played. For the third game, only player 2 played. In the last game, all players participated. The prediction column represents the predicted value for each case. Bullet points below explain how you can calculate the Shapley value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6mZ4WI9kH4CNWd-0l90D-w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ELEovOIE_f6Qq0tYDrTcQ.png" /><figcaption>Image by Author</figcaption></figure><p>Unlike feature importance, which would add a negative effect with positive interaction effects and main feature effects all at once, Shapley values takes into account the decrease in performance. Furthermore, it is more robust in generating the result. The upsides of the Shapley value doesn’t stop here. It works not only with linear models but also with neural networks! You can interpret any machine learning model with this value.</p><p>You can easily implement this value using <strong>SHAP</strong>(Shapley Additive exPlanations) library in python. The downside of the SHAP is that it is computationally expensive and slow. Additionally, one needs to be aware that the Shapley value should never be interpreted as a causal relationship. Just because a certain feature was helpful for the prediction does not always imply causation.</p><p>You may face some errors due to version incompatibility issues with Tensorflow. I’ll explain how to fix those here and run a simple experiment. The contents of the experiment cover image classification of American style make-up vs. Korean style make-up and which features contributed to the model’s prediction.</p><p>This is not an extensive experiment but to quickly check how SHAP could be applied in neural networks. In this experiment, I used a CNN model trained on a small dataset. Thus, the result may significantly improve by enlarging the dataset size. You can access the detailed code and dataset via this<a href="https://github.com/Irene-kim/Demystify_NeuralNets/tree/main/Shaply"> <strong>link</strong></a>.</p><h4>Here are very useful tips to solve errors while implementing SHAP.</h4><blockquote>Resource exhausted: OOM when allocating tensor with shape[ , , ,]</blockquote><p>When you run into this problem try these two steps:</p><ol><li>Reduced batch size to 16 (could be larger or smaller but to the point that it wouldn’t show the corresponding error)</li><li>Reduce input dimension to 100 (could be larger or smaller but to the point that it wouldn’t show the corresponding error)</li></ol><blockquote>If you are using Tensorflow version &gt; 2.4.0</blockquote><p>Restart the runtime and include the following code at the very top of your code.</p><pre>import tensorflow as tf<br>import tensorflow.compat.v1.keras.backend as K<br>tf.compat.v1.disable_eager_execution()</pre><h4>Experiment</h4><p>I have two folders that consist of training sets and test sets. Each image in the folder is sorted into two classes: ‘american_makeup’ and ‘korean_makeup’.</p><pre>base_dir = &#39;/content/gdrive/MyDrive/research/images&#39;<br>train_dir = os.path.join(base_dir, &#39;train&#39;)<br>test_dir = os.path.join(base_dir, &#39;test&#39;)</pre><pre>class_name = os.listdir(train_dir)<br>class_name_test = os.listdir(test_dir)</pre><pre>print(class_name)<br>#[&#39;american_makeup&#39;, &#39;korean_makeup&#39;] </pre><pre>print(class_name_test)<br>#[&#39;american_makeup&#39;, &#39;korean_makeup&#39;]</pre><p>As of now, each class is written with a categorical variable. I need to encode them into integer variables. Here I used LabelEncoder to change class names to an integer. Once you are done with integer encoding, the next step is to perform one-hot encoding. The class name ‘american_makeup’ is now labeled as [1,0] and ‘korean_makeup’ is labeled as [0,1].</p><pre>integer_encoded = LabelEncoder().fit_transform(class_name)<br>integer_encoded = integer_encoded.reshape(len(integer_encoded), 1)</pre><pre>onehot_encoded = OneHotEncoder(sparse=False).fit_transform(integer_encoded)</pre><pre>print(onehot_encoded)<br>#[[1. 0.]  [0. 1.]]</pre><pre>integer_encoded_test = LabelEncoder().fit_transform(class_name_test)<br>integer_encoded_test = integer_encoded_test.reshape(len(integer_encoded_test), 1)</pre><pre>onehot_encoded_test = OneHotEncoder(sparse=False).fit_transform(integer_encoded_test)</pre><pre>print(onehot_encoded_test)<br>#[[1. 0.]  [0. 1.]]</pre><p>Training dataset and test dataset are resized and reshaped and saved into new lists. Each of the corresponding labels is stored in train and test label lists.</p><pre>train_image = []<br>train_label = []<br>test_image = []<br>test_label = []</pre><pre># for train dataset<br>for i in range(len(class_name)):<br>    path = os.path.join(train_dir, class_name[i])<br>    img_list = os.listdir(path)<br>    for j in img_list:<br>        img = os.path.join(path, j)<br>        img = cv2.imread(img, cv2.IMREAD_COLOR)<br>        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)<br>        img = cv2.resize(img, (100, 100), interpolation =<br>              cv2.INTER_CUBIC)<br>        img = img.reshape((100, 100, 3))<br>        train_image.append(img)<br>        train_label.append(onehot_encoded[i])</pre><pre># for test dataset<br>for i in range(len(class_name_test)):<br>    path = os.path.join(train_dir, class_name_test[i])<br>    img_list = os.listdir(path)<br>    for j in img_list:<br>        img = os.path.join(path, j)<br>        img = cv2.imread(img, cv2.IMREAD_COLOR)<br>        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)<br>        img = cv2.resize(img, (100, 100), interpolation =<br>              cv2.INTER_CUBIC)<br>        img = img.reshape((100, 100, 3))<br>        test_image.append(img)<br>        test_label.append(onehot_encoded[i])</pre><p>There are few more steps that needs to be dome before applying SHAP. First, we need to normalize the data. Another important task is to actually know how your data is stored. American style make-up images were ordered first followed by Korean style make-up images in my dataset. In order to avoid any undesirable information leakage or learning I shuffled the whole data again.</p><pre># Normalize image data</pre><pre>X_train = shuffle(train_image.reshape(283, 100, 100, 3).astype(&quot;float32&quot;) / 255, random_state = seed)</pre><pre>X_test = shuffle(test_image.reshape(20, 100, 100, 3).astype(&quot;float32&quot;) / 255, random_state = seed)<br></pre><pre># Shuffle data or use library to randomly split your data</pre><pre>train_label = shuffle(train_label, random_state = seed)<br>test_label = shuffle(test_label, random_state = seed)</pre><p>Now you can finally design your model! The architecture of your model completely depends on how you would like to build your model <strong>as SHAP is model-agnostic</strong>!</p><pre>def CNN():</pre><pre>input_layer = keras.Input(shape=(100,100,3))</pre><pre>x = keras.layers.Conv2D(150, (3,3), padding=&#39;same&#39;, activation = &#39;relu&#39;, kernel_initializer = keras.initializers.HeUniform(seed=seed))(input_layer)</pre><pre>x = keras.layers.BatchNormalization()(x)</pre><pre>x = keras.layers.MaxPooling2D((2, 2))(x)</pre><pre>x = keras.layers.Conv2D(150, (3,3), padding=&#39;same&#39;, activation = &#39;relu&#39;, kernel_initializer = keras.initializers.HeUniform(seed=seed))(x)</pre><pre>x = keras.layers.BatchNormalization()(x)</pre><pre>x = keras.layers.MaxPooling2D((2, 2))(x)</pre><pre>x = keras.layers.Conv2D(150, (3,3), padding=&#39;same&#39;, activation = &#39;relu&#39;, kernel_initializer = keras.initializers.HeUniform(seed=seed))(x)</pre><pre>x = keras.layers.BatchNormalization()(x)</pre><pre>x = keras.layers.MaxPooling2D((2, 2))(x)</pre><pre>x = keras.layers.Conv2D(150, (3,3), padding=&#39;same&#39;, activation = &#39;relu&#39;, kernel_initializer = keras.initializers.HeUniform(seed=seed))(x)</pre><pre>x = keras.layers.BatchNormalization()(x)</pre><pre>x = keras.layers.MaxPooling2D((2, 2))(x)</pre><pre>x = keras.layers.Flatten()(x)</pre><pre>x = keras.layers.Dense(128, activation=&#39;relu&#39;, kernel_initializer = keras.initializers.HeUniform(seed=seed))(x)</pre><pre>x = keras.layers.Dropout(0.5)(x)</pre><pre>output_layer = keras.layers.Dense(2, activation=&#39;sigmoid&#39;)(x)</pre><pre>model = keras.Model(inputs=input_layer, outputs=output_layer, name = &#39;CNN&#39;)</pre><pre>model.compile(loss=&#39;binary_crossentropy&#39;, optimizer= keras.optimizers.SGD(learning_rate=0.001),  metrics=[&#39;acc&#39;, &#39;AUC&#39;])</pre><pre>return model</pre><p>After training the model, it was tested on ten American celebrity images and ten Korean celebrity pictures. Let’s see one of the example from the result.</p><pre>idx = 6<br>input_val = X_test[idx:idx+1]<br>output_val = model.predict(input_val)<br>real = test_label[idx:idx+1]</pre><pre>print(&quot;Prediction : &quot;, np.argmax(output_val))<br>print(&quot;Ground Truth : &quot;, np.argmax(real))</pre><pre>plt.imshow(input_val.reshape(100, 100, 3),interpolation=&#39;nearest&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*Rs8SQ5jQ7y4hCggdxLQ92Q.png" /></figure><p>The model accurately classified the image! How about the rest? Out of twenty images, three were wrong.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*QtnpLIX3oDx7_YJ3vHvRvg.png" /><figcaption>The Complete Testset Result</figcaption></figure><p>I wonder what affected the model’s classification. Generally speaking, American make-up and Korean make-up have two very distinct styles. American make-up tends to highlight the shape of arched eyebrows, long fake eyelashes, and very smokey eye make-up. On the other hand, Korean make-up tends to desire very natural eyebrows, light shadows, clear make-up, and red-orangish lips. I am going to see if my guess is what really contributed to the model’s prediction.</p><p>These are images of the American make-up style. Negative SHAP values refer to the negative impact and positive SHAP value refers to a positive impact on models decisions. The picture on the left is the groud-truth image. The one in the middle is label 0, which is American make-up style and the one on the far right is label 1, which is Korean style make-up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F2C6D8mNwwXnHiHbxhJWxA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vFr4czyBLd4tUa_7_ttZZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KETOdZfCPpE5Rsfl3_uMxQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7elUivzif6ttICY2NBHLSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*WSX6SJEC6Hk0bmOMwxXmMg.png" /></figure><p>Surprisingly, red dots are concentrated around eyes and eyebrows for the middle image, which is label 0 (American make-up). Blue dots around eyes and eyebrows on Korean-style images suggest that those parts are telling that it is not Korean make-up images. Now let’s see the opposite result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-fwvQnB4oYej8Qxza-77cA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tT4FmQ5RRwC3vVv8CzLoXw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ao0MARtxTEF3qLAbKa9Jg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*dMYQaew9VfQm5oMc4r24aw.png" /></figure><p>Korean style make-up is a lot less distinct compared to American make-up images. Nevertheless, we can guess that when the model classified images as Korean make-up it was largely due to the overall skin or structure of the face as red dots are spread across all faces.</p><p>How about misclassified cases?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*EGk81jpfPQIWnSCtb6TZUg.png" /></figure><p>Here we can see that both labels 0 and 1 have red and blue points everywhere across the image. One interesting point is that all of the mislabeled cases are not white. Such results can provide very important clues where to fix and work on your model. In real-life settings, such biased outcomes can bring disastrous consequences and discriminate against certain groups of people.</p><p>How useful is the SHAP? What a time to be alive :-)</p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cca29c836089" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Batch Normalization vs Drop out]]></title>
            <link>https://skirene.medium.com/demystifying-batch-normalization-vs-drop-out-1c8310d9b516?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c8310d9b516</guid>
            <category><![CDATA[batch-normalization]]></category>
            <category><![CDATA[dropout]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Mon, 11 Oct 2021 09:37:53 GMT</pubDate>
            <atom:updated>2021-10-11T10:57:27.322Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is batch normalization really the rule of thumb? Comparing the result with dropout on the CIFAR10 dataset</h4><p>Batch normalization(BN) has been known to improve model performance, mitigate internal covariate shift, and apply a small regularization effect. Such functionalities of the BN and empirical studies proving the effectiveness of BN helped to solidify people’s preference of using BN over dropout. BN quickly replaced the dropout layer in many deep learning models. Why is this the case? BN normalizes values of the units for each batch with its own mean and standard deviation. Dropout, on the other hand, randomly drops a predefined ratio of units in a neural network to prevent overfitting. Therefore, using the dropout layer and batch normalization layer — placing them next to each other to be more specific — creates disharmony between those two. Although the BN has a slight regularization effect, it’s more of a side-effect of the normalization process. Dropout, on the contrary, is a simple but strong regularizer to address the overfitting problem.</p><p>Here comes the dilemma: Should you then choose batch normalization over dropout or vice versa? Answering this question to some problems is quite simple. When you are using sequence models such as RNN or LSTM, you cannot use BN. Instead, layer normalization or dropout could be used as an alternative. In sequence models, dropout is a more widely adopted method of regularization. For other deep neural networks, however, utilizing BN instead of dropout especially in convolutional neural networks(CNN) is often considered as a rule of thumb.</p><p>Sprouting from my deep doubt and curiosity for BN almost always winning dropout, I conducted an experiment. Before getting started, I want to point out that my motivation for this experiment is far from rejecting the role of BN or proving one is better than the other. The objective of this experiment is to see whether BN really outperforms dropout and whether we can safely choose BN over dropout in our use cases.</p><p>The experiment was done via the colab environment using the CIFAR10 dataset. I built four different models with different model architectures to compare the result. The baseline model is the CNN model built with VGG blocks of two and BN layers. The second model is a deeper model than the baseline model. The third model has the same structure as the second model but ADAM was implemented for an optimizer. All the other models used SGD for an optimizer. Finally, the last model uses dropout instead of BN. You can see how each model was designed in this <a href="https://github.com/Irene-kim/Demystify_NeuralNets">GitHub link</a> along with step-by-step code guidelines to run the experiment. For clarification, this experiment used four different models:</p><ol><li>VGG2 with BN + SGD (Baseline)</li><li>VGG2 with BN on each CNN layer + SGD</li><li>VGG2 with BN on each CNN layer + ADAM</li><li>VGG2 with Dropout + SGD</li></ol><p>The image below shows three examples each for ten different categories in the CIFAR 10 dataset after data preprocessing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*OGFzCT1rTcJTFLnq4EzbCg.png" /><figcaption>CIFAR 10 dataset</figcaption></figure><h4>The training and validation process of each model and the results are as follows (blue line — train set, red line — validation set):</h4><p>1. VGG2 with BN + SGD (Baseline):<br>acc: 0.6658, loss: 0.9705, auc: 0.9457</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O0JbcoIYteigLAwp5-dHOg.png" /><figcaption>Model Accuracy, Loss, and Learning Rate</figcaption></figure><p>2. VGG2 with BN on each CNN layer + SGD:<br> acc: 0.6786, loss: 0.9437, auc: 0.9489</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*70vnxbAd8dSibou9FhK2UA.png" /></figure><p>3. VGG2 with BN on each CNN layer + ADAM:<br> acc: 0.7512, loss: 0.7312, auc: 0.9680</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qbXGVEkpCDZtI7_BWVGyfQ.png" /></figure><p>4. VGG2 with Dropout + SGD:<br>acc: 0.7803, loss: 0.6453, auc: 0.9741</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EcWIfEYsrT9S7k0tffB-Bg.png" /></figure><p>BN models with SGD optimizers seem to reduce overfitting compared to other models. Interestingly, the dropout model far outperforms other models with BN in terms of accuracy and loss! Examining the training and validation history, model 4 with the dropout layer presented the best performance followed by model 3. Its test accuracy reached approximately 0.8 whereas other models’ accuracy were lower. Likewise, the loss for model 4 for both training and validation datasets were the lowest.</p><p>Overfitting seems to be the problem for all models. In order to alleviate this problem, I introduced an early stopping and get the best hyperparameters to build the best performing models.</p><h4>Let’s check how each model classifies the image of a ship with our own eyes! The label for the ship image is 8.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*5qg9RvtIPhiN4Y0x6W7Qtw.png" /><figcaption>Image of a ship</figcaption></figure><ol><li>VGG2 with BN + SGD (Baseline)<br> — Prediction : 8</li><li>VGG2 with BN on each CNN layer + SGD<br> — Prediction : 0</li><li>VGG2 with BN on each CNN layer + ADAM<br> — Prediction : 8</li><li>VGG2 with Dropout + SGD<br> — Prediction : 0</li></ol><p>Model 2 and 4 here made incorrect predictions. They thought it was a plane instead of a ship.</p><p>Let’s try another one! This time the image is plane this time, which is labeled as 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*uT7uhEBCJ3sOeQqLeyUf0w.png" /><figcaption>Image of a plane</figcaption></figure><ol><li>VGG2 with BN + SGD (Baseline)<br> — Prediction : 0</li><li>VGG2 with BN on each CNN layer + SGD<br> — Prediction : 4</li><li>VGG2 with BN on each CNN layer + ADAM<br> — Prediction : 0</li><li>VGG2 with Dropout + SGD<br> — Prediction : 0</li></ol><p>This time only model 2 made an inaccurate decision. Model 2 predicted that it was a deer. Maybe the pointy horn-looking plane tail has to do with the mistake.</p><p>Now let’s see how models perform with animal images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*enOX14VUfeuh9hTJJ4cvVA.png" /><figcaption>Image of a frog</figcaption></figure><ol><li>2 with BN + SGD (Baseline)<br> — Prediction : 6</li><li>VGG2 with BN on each CNN layer + SGD<br> — Prediction : 2</li><li>VGG2 with BN on each CNN layer + ADAM<br> — Prediction : 6</li><li>VGG2 with Dropout + SGD<br> — Prediction : 6</li></ol><p>Again, model 2 made a mistake. Model 2 predicted that the frog was a bird.</p><p>This time let’s see how models predict this strangely-looking deer photo. The ground truth label for deer is 4.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*Ypr9o9VnFJotYLPLCAchcg.png" /><figcaption>Image of a deer</figcaption></figure><ol><li>2 with BN + SGD (Baseline)<br> — Prediction : 0</li><li>VGG2 with BN on each CNN layer + SGD<br> — Prediction : 0</li><li>VGG2 with BN on each CNN layer + ADAM<br> — Prediction : 4</li><li>VGG2 with Dropout + SGD<br> — Prediction : 4</li></ol><p>Model 1 and 2 thought the picture of deer was a plane! The background and the horn might have confused models to the sky and pointy tail wings of the plane.</p><p>Overall, the model with the dropout layer seems to perform fairly well compared to BN models. The result is very interesting to me as many people in the data science field blindly trust BN to work better than the dropout layer and end up never trying to compare these two. What the result suggests is that the dropout layer can perform better than BN in some cases. Last but not least, the result is telling us to always test and see with our own eyes! Although BN is an advanced and complicated technique, a simple dropout layer can also achieve similar or even better results in some cases with image data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c8310d9b516" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bayesian Optimization]]></title>
            <link>https://skirene.medium.com/bayesian-optimization-c9dd1381cd1d?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9dd1381cd1d</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[bayesian-optimization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Wed, 15 Sep 2021 12:26:10 GMT</pubDate>
            <atom:updated>2021-09-23T02:50:41.554Z</atom:updated>
            <content:encoded><![CDATA[<h4>Is it really the best hyperparameter tuning method? Comparing it with a grid search, random search, and hyperband search</h4><p>Hyperparameter tuning is a time-intensive and computationally expensive task. What we want is a tuning mechanism that finds the optimal hyperparameters as quickly as possible. There are a few methods to implement hyperparameter tunings such as grid search, random search, and hyperband. Each of them has its own benefits and drawbacks. And there comes<strong><em> Bayesian optimization</em></strong>. Making other tuning techniques seem a bit old-fashioned but does this fancy method beat all the other tuning approaches? Is it a<strong><em> golden rule of thumb</em></strong>? My quick answer is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GPn7xqHMtFSeV_IH4YxSmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zXNiHIoa9HzUXyIdPYIstA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FHl4veYqc_jeucpfRxCqZA.png" /><figcaption>Yello ‘X’ shape is the number of trials (Image Source: Author)</figcaption></figure><p>If you have limited computational resource and wants to get presumably the best hyperparameters within few iterations then<strong> yes</strong>. However, is it drastically better than other methods? <strong>No</strong>.</p><p>Let’s dig a little more into the Bayesian optimization.</p><blockquote>What is<strong> Bayesian optimization</strong>?</blockquote><p>Previously, in <strong>grid search,</strong> you are trying every possible option digging holes even that are not promising at all. <strong>Although impractical you won’t accidentally miss the global optimum.</strong> However, you need to have potentially unlimited computational power to achieve this.</p><p><strong>Random search</strong>, on the other hand, attempts to<strong> randomly poke here and there to give a good estimate of where the optimal hyperparameters are</strong>. It is a much more preferred way to do hyperparameter tuning. But you still need many iterations to have a good glimpse of where the range of global optimum is.</p><p><strong>Hyperband search </strong>is a variation of the random search. Unlike other search methods that will continue running the iterations with clearly bad hyperparameter sets until the prescribed iteration numbers exhaust. Hyperband search, instead,<strong> will terminate those bad configurations after very few iterations and only run sets of parameters that are promising to output good results based on the previous history</strong>.</p><p><strong>Bayesian optimization search </strong>is a completely different approach compared to the aforementioned search techniques. Bayesian optimization is a <strong>probabilistically principled method based on the Bayes theorem to find the global optimum of the black-box function</strong>.</p><p>What does this mean? If you know the ground-truth objective function such as f(x) = 0, we also know where the min and max values are. However, when we are using deep learning models, we can never know what the underlying objective function is. Therefore, all we could do is do lots of trial-and-error until either you get exhausted or your resource gets exhausted. However, no one wants either of these options! We want an intelligent way to explore the parameter space. Looking at the figures below, you can see how Bayesian optimization search finds global minimum with fewer iterations compared to other search methods.</p><p>This approach becomes handy when the objective function is complex and expensive to evaluate. As we do not know the objective function in most neural networks and deep learning model is costly to evaluate, a Bayesian optimization search seems like a light at the end of the tunnel. This is how Bayesian optimization works:</p><ul><li><strong>Bayes Theorem</strong> (conditional probability of an event): <br> P(A|B) = P(B|A) * P(A) /P(B) <br> <em>proportional quantity: P(A|B) = P(B|A) * P(A)</em></li><li><strong>Prior</strong>: Samples are drawn and will be evaluated with a random function. The outcome will be used to define the prior. The prior represents the behavior of the function.</li><li><strong>Posterior</strong>: Posterior is the conditional probability (the reverse is called likelihood) assigned taking the priors into account. Thus, it is an approximation of the objective function and evaluates the cost of specific parameter configurations. The posterior becomes a prior as they are updated in the next iteration. Posterior is used in the acquisition function to determine the next points to sample in the search space that is most likely to return good results.</li><li><strong>Surrogate Function</strong> <br>The surrogate function is an approximation of the black-box objective function. A Gaussian process(Gaussian distribution and random process) operates as a surrogate function. The Gaussian process builds the probabilistic model of the objective function, assuming all random variables have a multivariate normal distribution. In short, utilizing the Gaussian process guides how to explore the parameter samples in the search space.</li><li><strong>Acquisition Function</strong>: It determines the next query points from the parameter space. It uses <strong><em>exploration</em></strong><em>(where are optimal values? Let’s explore unknown areas or most uncertain areas)</em> and <strong><em>exploitation</em></strong><em>(where is the optimal value? Let’s find points where it gives the highest value based on the previous result) </em>strategy to guide selecting the next sample. The acquisition function tries to yield a good balance between exploration and exploitation. There are multiple ways to do this. “<em>Expected Improvement (EI)</em>” and “<em>Maximum Probability of Improvement</em>” are popular methods. In short, the acquisition function decides what to sample next.</li></ul><p>The gif image below simulates the Bayesian optimization process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*VUq2_g375O2yOcK0oK2aaw.gif" /><figcaption>image resource: <a href="https://github.com/AnotherSamWilson/ParBayesianOptimization">https://github.com/AnotherSamWilson/ParBayesianOptimization</a></figcaption></figure><p>Bayesian optimization, however, may not be the best tuning mechanism. There has been multiple research that the <strong>Bayesian optimization approach only contributes to negligible additional improvement compare to random search</strong>. Moreover, as <strong>the Bayesian optimization is weak at performing in high dimensionality. </strong>Sometimes, using just simple and straightforward is easier to implement, explain, and efficient in many real-life use cases if it’s causing only negligible performance differences. Depending on your use case, simple is the best!</p><p>One change made in this article is that although many people think that as Bayesian optimization uses the Gaussian process as a surrogate function the optimization process is sequential. This also implies that it cannot leverage parallel resources. However, there are ways you can enable parellel using tools such as <em>SigOpt</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9dd1381cd1d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning Definition Cheatsheet: Sequence Model version]]></title>
            <link>https://skirene.medium.com/deep-learning-definition-cheatsheet-sequence-model-version-b292b1a90cbc?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/b292b1a90cbc</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Sat, 10 Jul 2021 07:50:06 GMT</pubDate>
            <atom:updated>2022-07-17T03:32:17.541Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>This article is for people who already know the basic mechanisms of deep learning and sequence models and want to quickly review the core concepts.</blockquote><p>Deep learning has been deeply entrenched in so many aspects of our daily lives. People nowadays can so easily get access to a high-quality deep learning education for free. After learning so much, your brain gets overwhelmed and all the information gets mixed and tangled inside your brain. This article summarized definitions and some of the main functions of the sequence models including RNNs, LSTM, GRU, BiLSTM.</p><h3>RNNs (Recurrent Neural Networks)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0BqpR8H2n1A0dFRTfzJyfg.png" /><figcaption>RNN Architecture</figcaption></figure><h4>Main functions</h4><ol><li>Overcomes the constraints of traditional neural networks, which could not share features learnt across different positions of the input data.</li><li>Uses back-propagation through time to compute gradients.</li><li>It has parameters W, input vector at some time step(Xt). The same function and the same set of parameters are used at every time step (keeping a single hidden vector at every time step).</li></ol><h4>Limitations</h4><ol><li>Suffers from vanishing gradients problem and thus losing useful information when processing long sequences, which is also referred to as long-term dependency problem.</li><li>Computationally expansive and slow. The model propagates both forward and backward through the entire sequence to compute the loss and gradients.</li></ol><h3><strong>LSTM (Long Short-Term Memory)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UqoxOh60C10QcVFJlLrKLw.png" /><figcaption>LSTM Structure</figcaption></figure><h4>Main functions</h4><ol><li>A variant of RNN architecture — Rather than keeping a single hidden vector at every time step, it keeps two different hidden vectors at each time step (use previous hidden state &amp; current state to compute updated hidden state).</li><li>Structure of LSTM: <em>input gate, forget gate, output gate, gate gate<br>- </em><strong><em>input gate</em></strong><em>: </em>Whether to write the cell (ranges: 0–1) <br>-<strong> <em>forget gate</em></strong>: Whether to erase the cell (ranges: 0–1) <br>- <strong><em>gate gate</em></strong>: How much to write to the cell (ranges: -1–1)<br>- <strong><em>output gate</em></strong>: How much to reveal the cell (ranges: 0–1)</li><li>Sigmoid function outputs numbers between 0-1. 0 means let nothing go through and 1 means let everything go through.</li><li>Difference between hidden states and cell states<br>- <strong><em>hidden state</em></strong>: Remembers information of immediately previous events and forgets/overwrites at every time step. <br>- <strong><em>cell state</em></strong>: Carries information both in distant and close time steps at the current state, creating an uninterrupted gradient highway. The cell state is the long-term memory, which stores information from previous cells by implementing point-wise multiplication with the forget gate and adding the result of the point-wise multiplication of the input gate and output gate.</li></ol><h4>Limitations</h4><ol><li>Computationally expensive and takes a long time to train.</li><li>Complex architecture, but cannot completely remove the vanishing gradient problem.</li><li>Cannot get information from the opposite side of the current time step.</li></ol><h3>GRU (Gated Recurrent Unit)</h3><h4>Main functions</h4><ol><li>Improved the design complexity of LSTM and faster to train.</li><li>Structure of GRU: <em>update gate, reset gate</em><br>- <strong><em>update gate</em></strong>: How much information to pass through from previous time steps (0–1). Combination of forget gate and input gate in LSTM. <br>-<strong><em> reset gate</em></strong>: How much information to forget from prior time steps (0–1).</li><li>It does not have a cell state that holds long-term memory. GRU has only hidden states that are able to keep useful information from both long-term and short-term dependencies.</li></ol><h4>Limitations</h4><ol><li>Slow convergence and low learning efficiency (see<em> Reference1</em>).</li></ol><h3>BiLSTM (Bidirectional Long Short-Term Memory)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4jvjxL7Ew6XoGa9ZmLFAFg.png" /><figcaption>Bidirectional LSTM Architecture</figcaption></figure><h4>Main functions</h4><ol><li>It is a powerful algorithm consisted of two LSTMs. It holds information from both preceding and future time steps.</li><li>Installs an LSTM at the beginning of the sequence and the other LSTM at the end of the sequence.</li><li>Improves the ability of the model to capture contextual information in the sequence.</li></ol><h4>Limitations</h4><ol><li>You need the entire sequence of data before making predictions. Therefore, it has many constraints in deploying in real-time applications</li><li>Computationally expensive and slow.</li><li>Stacking deep layers to a certain extent generates the vanishing gradient problem.</li></ol><p><em>References</em></p><ol><li>Journal of Physics| <a href="https://iopscience.iop.org/article/10.1088/1742-6596/1325/1/012089/pdf">Xin Wang et al 2019 J. Phys.: Conf. Ser. <strong>1325 </strong>012089</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b292b1a90cbc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Instagram Scraper in 3 minutes]]></title>
            <link>https://skirene.medium.com/building-a-instagram-scraper-in-3-minutes-a6aac0a2512f?source=rss-91b48bffb803------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6aac0a2512f</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[instagram]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[SeongKyung Kim (Irene Kim)]]></dc:creator>
            <pubDate>Tue, 22 Jun 2021 03:02:26 GMT</pubDate>
            <atom:updated>2021-06-23T01:19:36.496Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z5oFE74S1k7TcEdo-D5oaQ.png" /><figcaption>Image from Instagram</figcaption></figure><p>Twitter is the most widely used source to collect texts for sentiment analysis, text classification, and many more for machine learning projects. Twitter provides its own API for users to crawl their text data. However, Instagram, nowadays, is more actively used by teenagers and younger adults in their 20s and 30s. Unfortunately, you need to make your own scraper or use the open-source library to crawl their text data or comments. I found it extremely difficult to use an open-source library package for scraping Instagram comments. This article will guide you to build your own Instagram comment scrapers within<em> 3 minutes</em>!</p><h4>1. Install and Load Required Library Packages</h4><pre>import pandas as pd<br>import time<br>import re<br>import os<br>import csv<br>import json<br>from urllib.request import urlopen<br>from urllib.parse import quote_plus<br>from bs4 import BeautifulSoup<br>from selenium import webdriver</pre><p>You then need to download webdriver from this site: <a href="https://chromedriver.chromium.org/downloads">https://chromedriver.chromium.org/downloads</a></p><p>Put your downloaded webdriver (it would probably name as <strong><em>chromedriver</em></strong> by default) in the folder you’d like to work in.</p><h4>2. Save the Driver and Get the URL</h4><pre>driver = webdriver.Chrome(‘path to your driver’)</pre><pre># link to the post you want to scrap comments from (This is Kim Kardashian&#39;s Instagram URL)<br>url = &quot;<a href="https://www.instagram.com/p/CQWQDCOgSt9/">https://www.instagram.com/p/CQWQDCOgSt9/</a>&quot;</pre><pre># Driver will access to the URL<br>driver.get(url)</pre><p>You then save your driver and put it in ‘<em>path to your driver</em>’! So mine would look like this: <em>webdriver.Chrome(‘/Users/irenekim/Desktop/Personal/Crawling/chromedriver’)</em></p><p>Instagram allows collecting comments and images from public accounts. Also, it is not a violation of privacy as long as you are using a public figure’s open account. But always double-check not to violate anyone’s data privacy!</p><p>In case you want to know the current working directory you can find this out with a simple code and it will show which working directory you are in right now in your python:</p><pre>os.getcwd()</pre><h4>3. Operate your Driver to Crawl the Comments</h4><pre># Create an empty container to put all the comments inside<br>comments = []</pre><p>These codes will automatically click the load more button on Instagram for you!</p><pre>hasLoadMore = True<br>while hasLoadMore:<br>    time.sleep(1)<br>    try:<br>        if driver.find_element_by_css_selector(&#39;#react-root &gt; section &gt; main &gt; div &gt; div &gt; article &gt; div.eo2As &gt; div.EtaWk &gt; ul &gt; li &gt; div &gt; button &gt; span&#39;):<br>            driver.find_element_by_css_selector(&#39;#react-root &gt; section &gt; main &gt; div &gt; div &gt; article &gt; div.eo2As &gt; div.EtaWk &gt; ul &gt; li &gt; div &gt; button &gt; span&#39;).click()<br>    except:<br>        hasLoadMore = False</pre><p>‘<strong>_6lAjh</strong>’ is the class name for username in Instagram. The path written in ‘<strong>find_elements_by_css_selector</strong>’ is the path to the span button in Instagram to load more comments. You can play around with this path by right-clicking the page and accessing to ‘<em>Inspection</em>’ page.</p><pre>users_list = []<br>users = driver.find_elements_by_class_name(&#39;_6lAjh&#39;)</pre><pre>for user in users:<br>    users_list.append(user.text)<br>    <br>i = 0<br>texts_list = []</pre><pre>texts = driver.find_elements_by_css_selector(&#39;#react-root &gt; section &gt; main &gt; div &gt; div &gt; article &gt; div.eo2As &gt; div.EtaWk &gt; ul &gt; ul &gt; div &gt; li &gt; div &gt; div &gt; div.C4VMK &gt; span&#39;)</pre><pre>for txt in texts:<br>    texts_list.append(txt.text)<br>    i += 1<br>    comments_count = len(users_list)<br><br>for i in range(1, comments_count):<br>    user = users_list[i]<br>    text = texts_list[i]<br>    comments.append(users_list[i])<br>    comments.append(texts_list[i])<br>    print(&quot;User &quot;,user)<br>    print(&quot;Text &quot;,text)<br>    print()</pre><p>The result would look something like this! I deleted the usernames here for privacy matters.</p><pre>User  user1<br>Text  🙌 I love this 😍<br><br>User  user2<br>Text  Fatherhood ❤️</pre><p>Calling the comments would give you all the lists of usernames and their comments.</p><pre>[&#39;user1&#39;,<br> &#39;This post is everything ❤️&#39;,<br> &#39;user2&#39;,<br> &#39;Kids r adorable&#39;]</pre><h4>4. Making Nice and Pretty Looking CSV file for Model Training</h4><p>What I am interested in is separating all the username and their comments because what we need is usually the texts but not the username. So I am going to make a CSV file that has all the accounts in the “User” column and their corresponding comments in the “Text” column.</p><pre>rows = []<br>for user, text in zip(comments[::2], comments[1::2]):<br>    print(user, text)<br>    rows.append([user, text])</pre><pre>fields = [&quot;User&quot;, &quot;Text&quot;]<br>filename = &quot;cleanfile.csv&quot;<br>with open(filename, &#39;w&#39;,  encoding=&#39;utf-8&#39;, newline=&#39;&#39;) as csvfile: <br>    csvwriter = csv.writer(csvfile) <br>    csvwriter.writerow(fields) <br>    csvwriter.writerows(rows)<br></pre><pre>df = pd.read_csv(r&quot;path to your filename&quot;)<br>print(df)</pre><p>Just a reminder, don’t forget to put .csv at the end of your path to your filename!</p><p>The result is a clean dataframe:</p><pre>     User                Text<br>0  user100         I love you Kim<br>1  user200         Causal nothing, it&#39;s called wanting attention<br>2  user300         This women needs mental health this is just wr...</pre><p>As you can see, there are some mean comments… <br>I actually wrote my master’s dissertation on detecting cyberbullying comments that are specifically targeting women using deep learning! Anyways, this is it for creating your own Instagram comment scrapers.</p><p>Easy!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6aac0a2512f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>