<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Giang Tran on Medium]]></title>
        <description><![CDATA[Stories by Giang Tran on Medium]]></description>
        <link>https://medium.com/@giangtran240896?source=rss-659a6d3f5e8a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*CXsAiAcFne_x8p-w6yQtew.jpeg</url>
            <title>Stories by Giang Tran on Medium</title>
            <link>https://medium.com/@giangtran240896?source=rss-659a6d3f5e8a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 07:22:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@giangtran240896/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[DARTS — Differentiable Architecture Search summary.]]></title>
            <link>https://medium.com/@giangtran240896/darts-differentiable-architecture-search-summary-aa38adfa8cad?source=rss-659a6d3f5e8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa38adfa8cad</guid>
            <category><![CDATA[automl]]></category>
            <category><![CDATA[nas]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Giang Tran]]></dc:creator>
            <pubDate>Tue, 24 Mar 2020 02:11:12 GMT</pubDate>
            <atom:updated>2020-03-24T02:11:12.847Z</atom:updated>
            <content:encoded><![CDATA[<h3>DARTS — Differentiable Architecture Search summary.</h3><p>This post summarize the work Differentiable Architecture Search (Liu et al, 2018) — CNN Part.</p><p>As the name suggest, alongside with other AutoML/NAS(Neural Architecture Search) approaches using Reinforcement Learning, Bayes Optimization or Evolutionary algorithm, DARTS uses gradient-based method by relaxing the search space to continuous, then jointly optimize architecture parameters <strong>α</strong> and weight parameters <strong>w</strong> (bi-level optimization).</p><p>The authors observed that the typical Convolutional Neural Network is the repetition of some motifs/blocks/cells (like Resisual block in ResNet, Inception Module in InceptionNet). So, instead of find the best network, they only find the optimial cell.</p><p>In DARTS, a cell is a Directed Acyclic Graph (DAG), it includes <strong>k</strong> nodes, each node connects to preceding nodes. Each edge that connects a pair of nodes <strong>(i,j)</strong>, applies some predefined operations (convolution, separable convolution, pooling, and more).</p><p>After training finish, the authors derive the final architecture from architecture parameters <strong>α</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TgKVEd0SEECKgjUi8OHffQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa38adfa8cad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformer model (NLP) summary.]]></title>
            <link>https://medium.com/@giangtran240896/transformer-model-nlp-summary-ea65db4d5a65?source=rss-659a6d3f5e8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea65db4d5a65</guid>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Giang Tran]]></dc:creator>
            <pubDate>Sat, 21 Mar 2020 08:55:29 GMT</pubDate>
            <atom:updated>2020-03-21T08:55:29.376Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://arxiv.org/abs/1706.03762">Transformer</a> is a breakthrough NLP model, it completely depends on attention mechanism (eliminates convolutional and recurrent neural network). It is the core/backbone that build up many state-of-the-art models: BERT, XLNet,… It also utilizes the use of parallelism to speed up training.</p><p>The key features in Transformer:</p><ul><li>Self-Attention Layer.</li><li>Cross-Attention Layer (Encoder-Decoder Attention Layer).</li><li>Positional Embedding.</li><li>Layer Normalization.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xqXlcftn91elOBQ9hNED4Q.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea65db4d5a65" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transfer Learning: A closer look]]></title>
            <link>https://medium.com/@giangtran240896/transfer-learning-a-closer-look-1981f9716e92?source=rss-659a6d3f5e8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1981f9716e92</guid>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[knowledge-transfer]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Giang Tran]]></dc:creator>
            <pubDate>Mon, 14 Oct 2019 07:02:45 GMT</pubDate>
            <atom:updated>2019-10-17T03:26:34.017Z</atom:updated>
            <content:encoded><![CDATA[<p>Deep Learning has major successful achivements on cognitive tasks in recent years like: image classification, object detection, speech recognition, etc. It comprises many layers connect to each others, creating a so-called deep neural network. People nowaday usually approach a Deep Learning problem by using <strong>transfer learning</strong>.</p><p>A major benefit of <strong>transfer learning</strong> is that it can transfer knowledge of trained network from one task to another with minimal effort of training when we know that the base task is similar as the target task. The underlying knowledge contains in <strong>hidden layers (Fully-Connected Layer/Convolutional Layer and Pooling Layer)</strong>.</p><figure><img alt="Transfer Learning" src="https://cdn-images-1.medium.com/max/1024/1*D5S6ylZwUZAxj0lMyipZ2g.png" /><figcaption>Transfer Learning from Task A to Task B. (<a href="https://medium.com/@subodh.malgonde/transfer-learning-using-tensorflow-52a4f6bcde3e">source</a>)</figcaption></figure><p>In Fully-Connected Neural Network (FCNN), each neuron in the layer is fully-connected with all neurons of previous layer by the <strong>weights</strong>. Each neural holds a piece of information to carry on to compute the loss at the output, we can see it as a <strong>feature extractor</strong>.</p><p>In Convolutional Neural Network (CNN) generally includes 2 abstract layers: <strong>Convolutional Layer</strong> and <strong>Pooling Layer</strong>. For Convolutional Layer, instead of fully-connected, a neuron (one point in each feature map) in the current layer (feature maps) only connects a small part (small number of neurons) of its previous layer, it means a <strong>kernel/filter</strong> slides through spatial dimension of previous layer to output a feature map (below image). And Pooling layer reduces the spatial dimension to boost computation also get a better representation in subsequent layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*VgaLLUDLcQy-1La8vBl7Xw.gif" /><figcaption>Convolutional Operation.</figcaption></figure><p>CNN has 2 main advantages over fully-connected neural network:</p><ul><li>First, typical images, or spectral representations of spoken words, are large, often with several hundred variables. A fully- connected first layer with, say a few 100 hidden units, would already contain several 10000 weights. Overfitting problems may occur if training data is scarse.</li><li>Second, since fully-connected layer connects every single pixels of previous layer, if the object in image translates or distorts it could lead to totally different representation.</li></ul><p>For that in mind, we have CNN as a better <strong>feature extractor</strong> than FCNN in term of images.</p><p>In <strong>transfer learning</strong>, we first train a base network on a base dataset, and then we repurpose the learned features, or transfer them, to a second target network to be trained on a target dataset.</p><p>On the first layers it learns features similar to <a href="https://en.wikipedia.org/wiki/Gabor_filter">Gabob filters</a> and color blobs. Such first-layer features appear not to be <strong>specific</strong> to a paricular dataset or task, but <strong>general </strong>in that they are applicable for many other datasets and tasks.</p><p>When the target dataset is significantly smaller than the base dataset, transfer learning can be a powerful tool to enable training a large target network without overfitting.</p><p>One can choose to backpropagate the errors from the new task into the base (copied) features to <strong>fine-tune</strong> them to the new task, or the transferred<br>feature layers can be left <strong>frozen</strong>, meaning that they do not change during training on the new task.</p><p>The choice of whether or not to <strong>fine-tune</strong> the first <strong>n </strong>layers of the target network depends on the size of the target dataset and the number of parameters in the first <strong>n </strong>layers.</p><p>So, if target data is large and the similarity between base data and target data is not the same. We should expect to train from scratch from base network to target network, and <strong>unfreeze </strong>weights at all layers in training process.</p><p>For target data is small, we should to <strong>fine-tune </strong>to choose how many last-layers to <strong>unfreeze</strong>.</p><h3><strong>References</strong></h3><p>[<a href="http://yann.lecun.com/exdb/publis/pdf/lecun-bengio-95a.pdf">1</a>]: Convolutional Network for Images, Speech and Time-Series . Yann Lecun and Yoshua Bagio. 1995</p><p>[<a href="https://arxiv.org/pdf/1411.1792.pdf">2</a>]: How transferable are features in deep neural networks. Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson. 2014</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1981f9716e92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[ML from scratch] Logistic Regression — Gradient Descent]]></title>
            <link>https://medium.com/analytics-vidhya/ml-from-scratch-logistic-regression-gradient-descent-63b6beb1664c?source=rss-659a6d3f5e8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/63b6beb1664c</guid>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[sigmoid]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <dc:creator><![CDATA[Giang Tran]]></dc:creator>
            <pubDate>Sun, 29 Sep 2019 06:27:49 GMT</pubDate>
            <atom:updated>2020-05-06T04:28:44.398Z</atom:updated>
            <content:encoded><![CDATA[<h3>[ML from scratch] Logistic Regression — Gradient Descent</h3><p>In this article, we will delve into the math behind Logistic Regression, and how it differs with classical classifier Support Vector Machine.</p><p>Logistic Regression is a model using for classification problems. Although “regression” in its name but logistic regression uses mostly for classification problems, especially binary classification.</p><p>For binary classification problem, the targets is a vector of {0, 1} corresponding negative and positive class, respectively.</p><p>Logistic Regression uses sigmoid function as the output which is a popular activation function in neural network. It can understand as the conditional probability for true class given <a href="https://medium.com/@giangtran240896/ml-from-scrach-linear-regression-normal-equation-gradient-descent-1af26b542c28">linear function</a>. It has the form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/199/1*477fFnx-V9L6-dSeU07zeA.png" /></figure><p>And the graph of sigmoid function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HXCBO-Wx5XhuY_OwMl0Phw.png" /><figcaption>Graph of sigmoid function.</figcaption></figure><p>We can see that its upper bound is 1 and lower bound is 0, this property makes sure we output a probability.</p><p>Derivative of sigmoid:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*kc85As-QOMdf-SXJpPX-jw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*otMeRmhr4dsiRlmgA82IMQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/367/1*tiYupHLghvriXW2H_JjiKg.png" /></figure><p>Intuitively, given a dataset with <strong>X</strong> is a matrix of features and <strong>y</strong> is vector label either positive or negative class, we want to classify which data point <strong>Xi</strong> belongs to. That means, visually, we find a line/plane/hyperplane (decision boundary) that split our data into 2 regions. That intuition is using mostly for <a href="https://en.wikipedia.org/wiki/Support-vector_machine">SVM</a> algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*4xeOZVXqPwjEtW98APSM4g.png" /><figcaption>SVM: best line split data into 2 regions.</figcaption></figure><p>But, logistic regression intuition is different. It maps data points to a higher dimension, e.g: 2 dimension -&gt; 3 dimension, with the new added dimension corresponding probability of classes. By default, the threshold that data points have probability ≥ 0.5 is class 1, and class 0 otherwise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*YW_pEBtrVzPI0JKIUKwdfQ.png" /><figcaption>The shape of logistic regression mapping from 2D -&gt; 3D.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*XJB2lzeuaD60qsIMFiyPFg.png" /><figcaption>Look from top-down aspect.</figcaption></figure><p>Suppose we have a matrix of features and a vector of corresponding targets:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*n2bjbDTWEuXpd8b5sP3rCw.png" /></figure><p>where N is number of data points and D is number of dimension at each data point.</p><p>Linear transformation <strong>h</strong> mapping from <strong>X</strong> to <strong>y </strong>by parameter <strong>w</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*poI7SXHGn2TlkmhDNpumvQ.png" /></figure><p>Apply element-wise of sigmoid function <strong>z</strong> to <strong>h </strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/441/1*FFsd-rAAzlhA-G26AfP3uw.png" /></figure><p>Since sigmoid outputs probability, we use negative log likelihood to represent the error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*1KyqULUn5ZjdCUOPSZa7mg.png" /></figure><p>where N is number of data points, yi is true label, zi is predicted probability of sigmoid. We want to minize this loss with respect to parameters <strong>w</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*12c68mkwf6-5VHILtm6iPA.png" /></figure><p>Surprisingly, the derivative <strong>J</strong> with respect to <strong>w </strong>of<strong> </strong>logistic regression is identical with the derivative of linear regression. The only difference is that the output of linear regression is <strong>h </strong>which is linear function, and in logistic is <strong>z </strong>which is sigmoid function.</p><p>After found derivative we use gradient descent to update the parameters:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*s-UND5G3T5wwEQ-NE3VBJQ.png" /></figure><p>We’re sure that it will converge in finite steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Fgba4pEC3IQQmLX7-gAlTQ.gif" /><figcaption>Training logistic regression visualization.</figcaption></figure><p>For logistic regression implementation, checkout <a href="https://github.com/giangtranml/ml-from-scratch/tree/master/logistic_regression">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63b6beb1664c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/ml-from-scratch-logistic-regression-gradient-descent-63b6beb1664c">[ML from scratch] Logistic Regression — Gradient Descent</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[ML from scratch] Linear Regression]]></title>
            <link>https://medium.com/analytics-vidhya/ml-from-scrach-linear-regression-normal-equation-gradient-descent-1af26b542c28?source=rss-659a6d3f5e8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1af26b542c28</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[normal-equation]]></category>
            <dc:creator><![CDATA[Giang Tran]]></dc:creator>
            <pubDate>Sat, 28 Sep 2019 07:14:25 GMT</pubDate>
            <atom:updated>2020-05-06T02:14:48.968Z</atom:updated>
            <content:encoded><![CDATA[<h4>Normal Equation and Gradient Descent</h4><p>The very first Machine Learning model when we start learning is Linear Regression.</p><p>In this article, I will explain in depth 2 popular approaches to solve a linear regression problem: <strong>Normal Equation</strong> and <strong>Gradient Descent.</strong></p><p>To work best, linear regression must follow these assumptions:</p><ul><li><strong>Linear Relationship between the features and targets.</strong></li><li><strong>Little or no Multi-collinearity between the features.</strong></li><li><strong>Normal distribution of error terms.</strong></li><li><strong>Homoscedasticity Assumption (a.k.a constant variance assumption).</strong></li><li><strong>Normal distribution of error terms.</strong></li><li><strong>Little or No auto-correlation in the residuals.</strong></li></ul><p>I won’t cover the details of these assumptions here. For more, please check this <a href="https://towardsdatascience.com/assumptions-of-linear-regression-algorithm-ed9ea32224e1">post</a>.</p><p>The idea of Linear Regression is simple. We’re given a data with a set of features and output range is continuous. Our task is to find a best line (plane in 3D and hyperplane in multi-D) that has the error between the line and outputs is as close as possible.</p><p>Suppose we have a matrix of features and a vector of corresponding targets:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/377/1*srjHYs0GVipFVmWiKz0oag.png" /></figure><p>where N is number of data points and D is number of dimension at each data point.</p><p>At the simplest form, linear regression is linear transformation <strong>h</strong> mapping from <strong>X</strong> to <strong>y </strong>by parameter <strong>w</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/1*Q0ZHccrm3Ds2D9KjZyO1fQ.png" /></figure><h3><strong>1. Normal Equation:</strong></h3><p>Use a little bit of linear algebra knowledge, we can find <strong>w </strong>in closed-form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*918C8XrjQQWkTG9d4FCrEQ.png" /></figure><p>However, in practice <strong>X </strong>is rarely square matrix, so we usually cannot invert <strong>X, </strong>the generalize approach is transpose it to square matrix, then take the invert:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*1PrCkuXUCYFxqakHzTtkeQ.png" /></figure><p>This approach has 2 drawbacks:</p><ul><li>First, X transpose X is really expensive computation. Suppose X has shape = (60000, 20) then X transpose X has 2⁰² * 60000 computation. Or even worst if X has high dimension, the computer memory will blow up.</li><li>Second, not every X transpose X is invertible. And invertible computation is also expensive.</li></ul><p>This is where <strong>gradient descent</strong> algorithm comes in.</p><h3>2. Gradient Descent:</h3><p>The idea of algorithm is at any point we move small step as backward direction of its derivative. The algorithm always make sure it will approach global minimum in finite steps with appropriate <strong>learning rate </strong>(this is true for linear regression but not for other advanced models).</p><p>The most popular loss for regression is Mean Squared Error (MSE) which has the form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*vT0HWMDlL4neyZE5YTktrg.png" /></figure><p>Our task now is to find parameter <strong>w</strong> that minimizes the error of <strong>y</strong> and <strong>h(w)</strong>. So how do we find the best <strong>w</strong>? As we learnt from high school that at a point <strong>x</strong> that make the function <strong>f(x) </strong>has derivative <strong>f(x)’</strong> is zero is either minimum or maximum, since we know that the shape of MSE is like a bowl (convex function), then we can make sure that point is minimum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*C3zklPy47yFs3eD0xMKYRQ.png" /></figure><p>Derivative of <strong>J</strong> with respect to <strong>w </strong>is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/365/1*cpcJ2Sqc-So9HBpppn9BrA.png" /></figure><p>For each step, use gradient descent algorithm to optimize <strong>w:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*s9vgrPUqWFeqaLS5dVolRA.png" /></figure><p>If learning rate is too large, our update with derivative will blow up and it never converges.</p><p>If learning is too small, the convergence is so slow, or probably stuck at 1 point forever.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*0yL87-2dAMVTCB1lDzPK-Q.gif" /><figcaption>Gradient descent with linear regression.</figcaption></figure><p>Below is polynomial linear regression, although it’s a curve but our optimization is still respect to <strong>w </strong>which is linear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*bUoZZ1T6KRUNPHAJ3CZtqg.gif" /></figure><p>For linear regression implementation, checkout <a href="https://github.com/giangtranml/ml-from-scratch/tree/master/linear_regression">here</a> .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1af26b542c28" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/ml-from-scrach-linear-regression-normal-equation-gradient-descent-1af26b542c28">[ML from scratch] Linear Regression</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>