<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Siddhant Haldar on Medium]]></title>
        <description><![CDATA[Stories by Siddhant Haldar on Medium]]></description>
        <link>https://medium.com/@siddhanthaldar14?source=rss-78da74ae2256------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*I115OXRWX3VNyWXq</url>
            <title>Stories by Siddhant Haldar on Medium</title>
            <link>https://medium.com/@siddhanthaldar14?source=rss-78da74ae2256------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:59:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@siddhanthaldar14/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Causal Inference : An Introduction]]></title>
            <link>https://medium.com/analytics-vidhya/causal-inference-an-introduction-f424df7c76ef?source=rss-78da74ae2256------2</link>
            <guid isPermaLink="false">https://medium.com/p/f424df7c76ef</guid>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[counterfactual]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Siddhant Haldar]]></dc:creator>
            <pubDate>Mon, 18 May 2020 09:13:08 GMT</pubDate>
            <atom:updated>2020-05-20T14:16:47.473Z</atom:updated>
            <content:encoded><![CDATA[<h3>Causal Inference : An Introduction</h3><p>Lately, the concept of causality has been gaining popularity in the domain of machine learning and artificial intelligence due to its inherent relation to the working of the world. Causality refers to the relation between a cause and its effect, i.e., it is based on the assumption that every effect arises due to a specific cause or a set of causes. In this blog, we will get introduced to the basics of causal inference, familiarise ourselves with the terminology used in the domain and delve deeper into the science of counterfactuals and counterfactual inference, which is becoming increasingly popular in recent times.</p><p><em>Prerequisite : This blog expects the readers to be familiar with the basic concept of neural networks and how they are trained with the help of </em><a href="https://medium.com/data-science-group-iitr/loss-functions-and-optimization-algorithms-demystified-bb92daff331c"><em>loss functions and optimizers</em></a><em>.</em></p><h3>Causal Inference</h3><p>The domain of causal inference is based on the simple principle of cause and effect, i.e., our actions directly cause an immediate effect. With causal inference, we can directly find out how changes in policy (or actions) create changes in real world outcomes. Let us familiarise ourselves with terminology used in the domain.</p><p><strong>Unit : </strong>An individual sample in the data.</p><p><strong>Variable : </strong>A characteristic of the unit of analysis in the dataset. For example, countries, people, weather, etc.</p><p><strong>Population : </strong>Collected set of all units of analysis.</p><p><strong>Outcome Variable : </strong>The particular variable that we want to affect.</p><p><strong>Policy / Treatment Variable : </strong>The variable used to create changes (i.e. actions).</p><p><strong>Intervention : </strong>Changing the value of the treatment variable keeping other variable values constant in order to study the effect of changes in the treatment variable.</p><p><strong>Confounder : </strong>Variables that affect both the input and the outcome variables.</p><p><strong>Counterfactual Outcome : </strong>The outcome that would have occurred had the unit been exposed to a different treatment (cannot be observed).</p><p><strong>Unit Level Causal Effect : </strong>Causal effect on a single unit in the data due to a change in the value of a particular variable keeping all other variables constant.</p><p><strong>Interaction Effects : </strong>This refers to the fact that other variables may also influence the value of a causal effect.</p><p><strong>Heterogeneity : </strong>Refers to the concept that different units exhibit different causal effects on being subjected to the same treatment.</p><p>Given an distribution of input variables <strong>X </strong>and a distribution of outcome variables (effects) <strong>Y</strong>, causal calculus differentiates between two kinds of conditional distributions - the observational distribution <em>P(y|x) </em>and the interventional distribution <em>P(y|do(x))</em>. <em>P(y|x) </em>represents the distribution of <strong>Y</strong> given that we observe variable <strong><em>X</em></strong> takes value <em>x </em>whereas <em>P(y|do(x)) </em>describes the distribution of <strong>Y </strong>that would be observed by artificially forcing the variable <strong>X </strong>to take a value <em>x</em>, keeping the rest of the variables in accordance with the process that generated the data. Such interventional distributions can be obtained using <a href="https://en.wikipedia.org/wiki/Randomized_controlled_trial"><strong>randomized controlled trials (RCTs)</strong></a>. However, it might not always be possible to carry out such experiments due to ethical or practical concerns. More about causal inference and its relation with do-calculus can be found <a href="https://www.inference.vc/untitled/">here</a>.</p><p>The <strong>Fundamental Problem of causal inference</strong> is that in the real world, each unit can be subjected to just one of the multiple treatments and only the outcome corresponding to that treatment can be observed. The effect of other treatments on the same unit cannot be directly observed and that is what we want to predict using causal inference techniques.</p><p><strong>Note : </strong>Correlation does not imply causation. Correlation refers to how two variables are related whereas causation studies if one variable affects the others. Two variables might be varying in a correlated fashion but one of them might not be causing changes in the other.</p><h4>Frameworks for Causal Inference</h4><p><strong>Structural Equation Modelling (SEM).</strong> <a href="https://www.lexjansen.com/wuss/2006/tutorials/TUT-Suhr.pdf"><em>Structural equation modelling</em></a> is a methodology for representing, estimating, and testing a network of relationships between variables (measured variables and latent constructs). The purpose of the model, in the most common form of SEM, is to account for variation and covariation of the measured variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*3DO2Qi5ZngVJLFffGAfkTA.png" /><figcaption>Structural Equation Model — Relationship between academic and job constructs</figcaption></figure><p><strong>Potential Outcomes Framework. </strong>Also known as the <a href="https://en.wikipedia.org/wiki/Rubin_causal_model"><em>Rubin causal model (RCM)</em></a>, the <em>potential outcomes framework</em> is based on the idea of potential outcomes. For example, a person would have a particular income at age 40 if she had attended college, whereas she would have a different income at age 40 if she had not attended college. To measure the causal effect of going to college for this person, we need to compare the outcome for the same individual in both alternative futures. Since it is impossible to see both potential outcomes at once, one of the potential outcomes is always missing.</p><h3>Counterfactual Inference</h3><p>Counterfactual inference can, in a way, be defined as the prediction of an alternate reality. Given a pair of a cause and its effect, counterfactual inference focuses on answering the question — “What would have been the effect of a different treatment applied to the unit keeping all the other conditions constant?”. For example, say a patient suffering from a particular ailment is administered with a drug <strong>D1 </strong>and they show signs of improvement. A counterfactual enquiry in this scenario would be to ask whether the patient would still have shown a similar improvement had they been subjected to a different drug <strong>D2</strong>.</p><p>There are multiple ways of doing counterfactual inference but since this is an introductory blog, we will just be mentioning a few very basic modelling techniques.</p><p><strong>Bayesian Structural Modelling. </strong>In this method, a causal graph is designed and Bayesian techniques are used to estimates outcomes corresponding to a specific initialisation. <a href="https://arxiv.org/pdf/1506.00356.pdf">This</a> paper proposes a similar model for counterfactual inference on time series data wherein a variable value is intervened at a particular time step and the effect of this intervention is observed in the subsequent time steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/1*57uTFX6QTNPEwJ7mlW_4XA.png" /><figcaption>Graphical model proposed in the paper</figcaption></figure><p><strong>Deep Learning based Generative Modelling. </strong><a href="http://papers.nips.cc/paper/7223-causal-effect-inference-with-deep-latent-variable-models.pdf">This</a> paper introduces a method based on Variational Autoencoders (VAE) which follows the causal structure of inference with proxies. A specific causal graph is considered as shown in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*-1YVVo4WdtxeDv4prxKbPQ.png" /><figcaption>Overall architecture of the model and inference networks for the Causal Effect Variational Autoencoder (CEVAE). White nodes correspond to parametrized deterministic neural network transitions, gray nodes correspond to drawing samples from the respective distribution and white circles correspond to switching paths according to the treatment t.</figcaption></figure><p><strong>Deep Learning based Deterministic Approaches. </strong>In this technique, neural prediction networks are used such that given an input and the treatment the unit has been subjected to, a set of prediction values can be obtained. <a href="https://arxiv.org/pdf/1606.03976.pdf">This</a> paper proposes the TARNet which follows a similar approach wherein there are a set of shared layers where the information is shared between the treatments branches and then there is an individual branch for each treatment that the unit might be subjected to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/0*wmKX3HGs7eiEwvKy" /><figcaption>TARNet proposed in the paper.</figcaption></figure><p>These are very basic approaches targeted at performing counterfactual inference. Other complications such disjoint distributions of the source and target samples and non-availability of sufficient training data for certain treatments among others are a topic for another blog.</p><h4>Evaluation Metrics for Causal Inferences</h4><p>In order to train the above discussed models, several evaluation metrics are used.</p><p><strong>Individual Treatment Effect (ITE). </strong>Individual Treatment Effect estimation aims to examine whether a treatment <em>T</em> affects the outcome <em>Y(i)</em> of a specific unit <em>i</em>. If the treatment <em>T(i)</em> has not been applied to unit <em>i</em>, <em>Y</em>₀<em>(i) </em>is called the potential outcome of treatment <em>T(i) = 0</em> and <em>Y</em>₁<em>(i)</em> the potential outcome of treatment <em>T(i) = 1</em>. The individual treatment effect on unit <em>i</em> is defined as the difference between the potential treated and control outcomes :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/173/1*qV_DrRFwJL68vtAguRCN7A.png" /></figure><p>The challenge to estimate ITE lies in estimating the missing counterfactual outcome.</p><p><strong>Average Treatment Effect (ATE). </strong>ATE is the average of all values of unit level causal effects in a population. The average outcome when all units are affected by the policy is called average outcome under the policy and the average outcome when none of the units are affected by the policy is called average outcome without the policy. The average treatment effect is the difference between the average outcome under the policy and the average outcome without the policy.</p><p>The average treatment affect (ATE) <em>ψ</em> is given by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*hxgAX8H6AX6y_so-UIDzxg.png" /></figure><p><strong>Precision in Estimation of Heterogenous Effect (PEHE).</strong> In the binary setting, PEHE measures the ability of a predictive model to estimate the difference in effect between two treatments <em>t</em>₀ and <em>t</em>₁ for samples <strong><em>X</em></strong>. To compute the PEHE, we measure the mean squared error between the true difference in effect y₁(n) − y₀(n), drawn from the noiseless underlying outcome distributions µ₁ and µ₀, and the predicted difference in effect yˆ₁(n) − yˆ₀(n) indexed by n over N samples :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*TksOcqIT8_wTwnet09TVvw.png" /><figcaption><a href="https://arxiv.org/pdf/1810.00656.pdf">https://arxiv.org/pdf/1810.00656.pdf</a></figcaption></figure><p>When the underlying noiseless distributions <em>µ</em> are not known, the true difference in effect<em> y₁(n) − y₀(n)</em> can be estimated using the noisy ground truth outcomes <em>y.</em></p><p>These 3 metrics have primarily been used for training and evaluating models performing counterfactual inference. Several other metrics aimed at maintaining balanced distributions in the latent space and nearest neighbour matched counterfactuals exists but that is a topic for another blog.</p><h3>Conclusion</h3><p>This blog has briefly touched upon the basics of causal inference and counterfactual studies in the domain. As mentioned previously, there’s a lot more to know about in this domain. However, more advanced topics would be covered in future blogs. I hope this blog serves as a worthy introduction to the domain of causal inference.</p><p><em>Thank you for your time in reading this blog. Kindly </em><a href="https://www.linkedin.com/in/siddhant-haldar-2ba70a136/"><em>contact</em></a><em> me if you have any queries about the topic.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f424df7c76ef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/causal-inference-an-introduction-f424df7c76ef">Causal Inference : An Introduction</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradient-based Adversarial Attacks : An Introduction]]></title>
            <link>https://medium.com/swlh/gradient-based-adversarial-attacks-an-introduction-526238660dc9?source=rss-78da74ae2256------2</link>
            <guid isPermaLink="false">https://medium.com/p/526238660dc9</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[adversarial-attack]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[backpropagation]]></category>
            <dc:creator><![CDATA[Siddhant Haldar]]></dc:creator>
            <pubDate>Wed, 08 Apr 2020 23:25:33 GMT</pubDate>
            <atom:updated>2020-04-09T20:31:59.860Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gradient-based Adversarial Attacks : An Introduction</h3><p>Neural networks have lately been providing the state-of-the-art performance on most machine learning problems, even to the extent of producing superhuman performance on several tasks. However, though such superior performance might seem very alluring, it has been observed that these neural networks are not very robust in terms of dealing with slightly different distributions. Consider a neural network that has been trained on an input distribution <strong>X </strong>with the corresponding label set <strong>L</strong>. Given a input example <em>x </em>with a corresponding label <em>l</em>, it can be shown that an adversarial example <em>x’ </em>can be obtained from <em>x </em>by adding a very small perturbation to the original input such that <em>x’ </em>is classified differently as compared to <em>x</em>. The fact that such a small perturbation can cause the existing neural networks to falter prevents these models from being used in security-critical areas. In this blog, we will be summarising the basic gradient-based attack mechanisms that have been developed for performing white-box (model parameters and weights are available to the attacker) adversarial attacks on neural networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dauBanmcpBSulOwsQ6rw4g.png" /><figcaption>Example of misclassification due to Adversarial Attacks</figcaption></figure><h3>Gradient based Adversarial Attacks</h3><p>Gradient based adversarial attacks exploit a very simple idea originating from the concepts involved in back-propagation(an algorithm used to train deep neural networks). We will go through back-propagation in a very informal manner in this blog since the main focus here is on summarising gradient based techniques for developing adversarial examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*KbAyutEKkvf5Y_XF.png" /><figcaption>Gradient flow during back-propagation</figcaption></figure><p>For training deep neural networks, the back-propagation algorithm is widely used. The first step here involves calculating the error(using a specific error function) between the desired output and the network output corresponding to a particular input. Now keeping the input constant, the calculated error is used to compute the gradients corresponding to each parameter(also known as <em>weight</em>) of the network. These gradients are used update the weights at each step taking into consideration a specific learning rate.</p><p>Gradient based attacks use this concept to develop a perturbation vector for the input image by making a slight modification to the back-propagation algorithm. Contrary to common practice, while back-propagating through the network, it considers the model parameters(or <em>weights</em>) to be constant and the input to be a variable. Hence, gradients corresponding to each element of the input (for example, pixels in case of images) can be obtained. These gradients have been utilised in various manners to obtain the perturbation vector, such that the new adversarial example has a greater tendency towards being misclassified, provided that it satisfies the constraint of being very similar to the input. Since the input gradients are used to obtain these perturbation vectors, these are known as gradient based attacks.</p><p>Some of these gradient based adversarial attack techniques have been explained below.</p><p><em>A prerequisite for understanding the mathematics behind these methods is a basic knowledge of calculus and the concept of norms of vectors and matrices.</em></p><h4>Fast Sign Gradient Method (FGSM)</h4><p>In their <a href="https://arxiv.org/pdf/1412.6572.pdf">paper</a>, the authors argue that :</p><blockquote>“ .. neural networks are too linear to resist linear adversarial perturbation. LSTMs (Hochreiter &amp; Schmidhuber, 1997), ReLUs (Jarrett et al., 2009; Glorot et al., 2011), and maxout networks (Goodfellow et al., 2013c) are all intentionally designed to behave in very linear ways, so that they are easier to optimize. More nonlinear models such as sigmoid networks are carefully tuned to spend most of their time in the non-saturating, more linear regime for the same reason. This linear behaviour suggests that cheap, analytical perturbations of a linear model should also damage neural networks.”</blockquote><p>Therefore, the authors focus on producing an adversarial example <strong><em>x’ = x + </em>η<em> </em></strong>such that <strong><em>x’</em></strong> is classified incorrectly by the neural network. In order to make <strong><em>x’</em></strong> and <strong><em>x</em></strong> produce different outputs, <strong>η</strong> should be bigger than the precision of the features. For example, if each pixel of an image is represented by 8 bits, any information below <em>1/255 </em>of the dynamic range is discarded. In order to achieve this, the authors use the following scheme for computing <strong>η<em> </em></strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*PAF7Lj2lcMw4kuRqBYhlYA.png" /></figure><p>Here, <strong><em>J</em></strong> is the cost function used to train the neural network, <strong><em>θ</em></strong> represents the parameter of a model, <strong><em>x</em></strong> is the input to the model and <strong><em>y</em></strong> is the target associated with <strong><em>x</em></strong> (for machine learning tasks that have targets). Here, <em>ε</em> decides the size of the perturbation and sign of every element of the perturbation vector(or matrix/tensor) is decided by the sign of the input gradient at the element. This solution is motivated by linearizing the cost function and solving for the perturbation that maximizes the cost subject to an <em>L∞</em> constraint. This linear perturbation technique method reliably causes a wide variety of models to misclassify their input. This method does not require an iterative procedure to compute adversarial examples, and thus is much faster than other considered methods.</p><h4>Basic Iterative Method (BIM)</h4><p>In this <a href="https://arxiv.org/pdf/1607.02533.pdf">paper</a>, the authors suggest a very simple improvement to FGSM. They suggest applying the same step as FGSM multiple times with a small step size and clip the pixel values of intermediate results after each step to ensure that they are in an <em>ε</em>-neighbourhood of the original image. This method is also referred to as Iterative-FGSM or IFGSM. Mathematically, the attacking scheme can be demonstrated as :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*-GyXcCRnEPyDGljUN5aLrg.png" /></figure><p>For the experiments, the value of α<em> </em>used is 1, i.e., the pixel values are changed only by 1 at each step. . The number of iterations were chosen heuristically; it is sufficient for the adversarial example to reach the edge of the <em>ε</em> max-norm ball but restricted enough to keep the computational cost of experiments manageable.</p><p>Both the methods described till now have focused on simply trying to increase the cost of the correct class, without specifying which of the incorrect classes the model should select. <em>A technique for this has been shown in the paper. Since this blog focuses on a basic overview of gradient based adversarial attacks, we are skipping explicit details from the paper. Kindly refer to the paper for knowledge about the same.</em></p><h4>Boosting FGSM with Momentum</h4><p>The momentum method is a technique for accelerating gradient descent algorithms by accumulating a velocity vector in the gradient direction of the loss function across iterations. The memorisation of previous gradients helps to barrel through narrow valleys, small humps and poor local minima or maxima. The momentum method also shows its effectiveness in stochastic gradient descent to stabilize the updates. We apply the idea of momentum to generate adversarial examples and obtain tremendous benefits.</p><p>FGSM generates an adversarial example by applying the sign of the gradient to a real example only once by the assumption of linearity of the decision boundary around the data point. However in practice, the linear assumption may not hold when the distortion is large, which makes the adversarial example generated by FGSM underfit the model, limiting its attack ability. As an improvement, iterative FGSM greedily moves the adversarial example in the direction of the sign of the gradient in each iteration. However, in this case, the adversarial example can easily drop into poor local maxima and overfit the model, which is not likely to transfer across models. The property of producing attacks that can be transferred to other models whose parameters are not accessible to the attacker is known as the <em>transferability</em> of an attack.</p><p>Thus, in this <a href="https://arxiv.org/pdf/1710.06081.pdf">paper</a>, the authors integrate momentum into the iterative FGSM for the purpose of stabilizing update directions and escaping from the poor local maxima. By using momentum, the gradients are updated at each step by accumulating the velocity vector in the gradient direction as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/253/1*jaFOlHzjeoefI68L6meIQA.png" /></figure><p>After gradient calculation, the adversarial sample at the <em>(t+1)-th</em> step is computed as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*u-sY9cgEYQ6CvjVGTLzw_A.png" /></figure><p>The momentum-based method helps retain the transferability of adversarial examples when increasing iterations (by preventing the adversarial example from dropping into a local maxima), and at the same time acts as a strong adversary for the white-box models like iterative FGSM. It alleviates the trade-off between the attack ability and the transferability, demonstrating strong black-box attacks.</p><h4>Projected Gradient Descent (PGD)</h4><p>The Projected Gradient Descent (PGD) attack is essentially the same as BIM (or IFGSM) attack. The only difference is that PGD initializes the example to a random point in the ball of interest (decided by the <em>L∞ </em>norm) and does random restarts, while BIM initializes to the original point.</p><p>In general, Projected Gradient Descent is a well-known method in the optimization literature [<a href="https://arxiv.org/abs/1405.4980">paper</a>]. In that sense, it is better to use “PGD” to refer to this (quite general) method instead of the narrow “IFGSM” terminology. Strictly speaking, the version of PGD that we are talking about is the non-euclidean, <em>L∞-PGD</em> that uses the <em>L∞</em> norm as a distance function.</p><h4>Carlini Wagner Attack with L2 Norm</h4><p>In this approach, the authors propose to generate adversarial samples by considering the following optimization problem</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/178/1*NM95UexFtWEU5V9Yt2oC8w.png" /></figure><p>where <strong>x </strong>is fixed and the goal is to find <em>δ</em> that minimizes <em>D(</em><strong>x</strong>, <strong>x</strong><em>+δ)</em>. Thus, we want to find a small change <em>δ </em>such that when added to an image <strong>x</strong>, the image is misclassified (to a targeted class <em>t</em>) by the model but the image is still a valid image. Here, <em>D </em>is some distance metric (<em>L0, L2 </em>or <em>L∞ </em>according to the paper) and <em>C </em>is the model being used.</p><p>One problem here is that it very difficult for the existing algorithms to solve the constraint <em>C(</em><strong>x</strong><em>+δ) = t </em>due to its highly non-linear nature. Thus, the authors express the problem in a different form suitable for optimization. They define an objective function <em>f </em>such that <em>C(x+δ) = t </em>if and only if <em>f(x + δ) ≤ 0. </em>The paper proposes a list of possible choices of <em>f</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*s6d1chBwX0vI50e8Jf9S8w.png" /></figure><p>where <em>s</em> is the correct classification, <em>(e)+</em> is short-hand for <em>max(e, 0)</em>, <em>softplus(x) = log(1 + exp(x))</em>, and <em>lossF,s(x)</em> is the cross entropy loss for x. Some of the above formula have been adjusted by adding a constant. However, this has been done only so that the function respects the original problem definition and it does not affect the final result. It simply scales the minimization function.</p><p>Now, instead of formulating the problem</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/183/1*qtG8T9IWIz3zvWKReI_KCQ.png" /></figure><p>this alternate formulation is used :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/262/1*-Tr-6DxtQliuA8CAKdtMDw.png" /></figure><p>where <em>c &gt; 0 </em>is a suitably chosen constant.<em> </em>Considering the distance metric <em>D </em>to be the <em>p-th</em> norm of a set of vectors, the optimization problem is reduced to finding δ that solves</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/218/1*We5-gM3sqN66GemNac6PRw.png" /></figure><p>Now, in order to satisfy the constraint in the optimization problem, a change-of-variables is applied and thus, we optimize over <em>w</em>, setting</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/192/1*YrM5aqT9EUk-7jR7xTU-lw.png" /></figure><p>Using this change-of-variable, since <em>−1 ≤ tanh(w) ≤ 1</em>, the constraint <em>0 ≤ x+δ ≤ 1 </em>is satisfied. Hence, this becomes a simple minimization problem now and according to the experiments conducted by the authors, the most effective optimization algorithm for solving this problem is the Adam optimizer.</p><p><em>For more details about searching for the optimal value of c and convergence proofs, kindly refer to the </em><a href="https://arxiv.org/pdf/1608.04644.pdf"><em>paper</em></a><em>.</em></p><h3>Adversarial Training</h3><p>Standard supervised training does not mandate that the chosen model will be resistant to adversarial examples. This property must be encoded into the training procedure somehow and that is where adversarial training comes in.</p><p>It can be shown that by training on a mixture of adversarial and clean examples, a neural network could be regularized somewhat. In general, data augmentation uses techniques such as translation, rotation, etc in order to produce input samples similar to what might be expected in the test set. However, training on adversarial samples is a little different in the sense that it uses inputs that are unlikely to occur naturally but that expose flaws in the ways that the model conceptualises its decision function. Hence, a wide range of literature in the domain of adversarial machine learning uses adversarial training to make models robust against such adversarial examples. And since these adversarial attacks have been observed to be transferable, adversarial training using samples generated from a single model provides robustness to other models performing the same task as well.</p><h3>Conclusion</h3><p>This blog has briefly touched upon some of the gradient based techniques for generating adversarial samples. Other techniques that one must definitely familiarise themselves with is <a href="https://arxiv.org/pdf/1511.04599.pdf">DeepFool</a> and the research paper “<a href="https://arxiv.org/pdf/1804.02391.pdf">Learn to Pay Attention</a>”. I hope this blog serves as a worthy introduction to the field of adversarial attacks.</p><p><em>Thank you for your time in reading this blog. Kindly </em><a href="https://www.linkedin.com/in/siddhant-haldar-2ba70a136/"><em>contact</em></a><em> me if you have any queries about the topic.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=526238660dc9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/gradient-based-adversarial-attacks-an-introduction-526238660dc9">Gradient-based Adversarial Attacks : An Introduction</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Internship Experience at University of British Columbia, Okanagan Campus]]></title>
            <link>https://medium.com/@siddhanthaldar14/internship-experience-at-university-of-british-columbia-okanagan-campus-b93452a252ed?source=rss-78da74ae2256------2</link>
            <guid isPermaLink="false">https://medium.com/p/b93452a252ed</guid>
            <category><![CDATA[internship-experience]]></category>
            <category><![CDATA[kelowna]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[summer-internships]]></category>
            <category><![CDATA[university-internship]]></category>
            <dc:creator><![CDATA[Siddhant Haldar]]></dc:creator>
            <pubDate>Tue, 07 Apr 2020 07:48:31 GMT</pubDate>
            <atom:updated>2020-04-07T07:48:31.400Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the most fulfilling experiences as an undergrad is participating in a summer internship abroad. This is a personal account of the experiences that I had during the summer of 2019 as a visiting research student at University of British Columbia(UBC), Okanagan Campus. I was able to procure this internship through the Mitacs Globalink Research Internship(GRI) program.</p><p>UBC Okanagan is based in a small town called Kelowna in the province of British Columbia. It is a very small town adorned with gorgeous landscapes, surrounded by mountains, lakes and wineries. This was my first trip abroad and I had been thrilled about my 11 week stint in Kelowna.</p><p>I arrived in Kelowna on May 5, 2019 and during my first day there, I settled in and gathered the basic necessities(pillows, blankets, etc.) required for my stay. My apartment was pretty close to the university campus and so, the next morning, I took a bus ride to the campus to meet my adviser and to introduce myself to other members of the lab. After being done with the introductions and the formalities for getting me enrolled as a visiting student, I set out to explore the university campus. UBC Okanagan has a relatively small and beautiful campus with separate buildings for arts, sciences and engineering. There is an enormous library along with a very well equipped gymnasium and numerous fields for students to visit after they are done with school. Furthermore, if you want to grab a bite in between your research or your classes, the campus houses branches of Tim Hortons, Starbucks and plenty of food trucks and other canteens, all of which dispense stunning food at affordable prices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tsMykxQeiOb2sePc3xapJA.jpeg" /><figcaption>The University Campus</figcaption></figure><p>From the second day onward, I commenced working on my project. I was pursuing my internship in the field of computer vision with my project involving the development of novel techniques for small scale object detection in multi-modal images. The lab I was working in comprised multiple PhDs and Master’s students with me being the only undergrad among them. The members of the lab specialised in the fields of computer vision, data science and the applications of machine learning to other disciplines such as electrical engineering and civil engineering among others. There used to be weekly update meetings wherein every member in the lab gave an overview of what they had been working on and these meetings turned out to be very illuminating about the current research going on in the domains that the lab had been working on. It was a very knowledgeable experience altogether.</p><p>My schedule during the weekdays involved waking up in the morning, preparing breakfast and lunch and leaving for the university. I used to work till the evening after which I would either go back to my apartment or go to the supermarket in order to gather the weekly supplies. Occasionally, we set out to have a splendid dinner at one of the better restaurants in town. As time progressed, the workload escalated with me putting in more hours at the lab. However, I followed a “work hard, party harder” kind of policy wherein I put in a lot of effort during the week and went out during the weekends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bDWhNf-w3fb-lWEuSi-cfA.jpeg" /><figcaption>Scenes during weekends</figcaption></figure><p>Kelowna is surrounded by a great many lakes and wineries. Our go-to spot on a usual chilled out day used to be the Downtown. It was an astonishing spot to hangout with friends with stunning restaurants, cafes, ice cream parlours and pubs/bars. One could also go there for a simple walk by the lake with a cup of ice cream to accompany you. Also, I was resolute about exploring the wineries in and around Kelowna. Hence, I went to quite a few of them for wine tastings and for getting to know about the practices concerned with wine making. The wineries were extremely beautiful with astounding sceneries which undeniably called for an amazing trip. Another very good way of spending the weekend involved going on road trips with breathtaking scenes and participating in adventure sports that the place had to offer. We had gone kayaking, sky diving(an out of this world experience), biking on mountains, to the beach for a swim, had barbecues and made it a point to have wonderful dinners at the best restaurants in the town. I had also visited Vancouver and Calgary on a couple weekends to experience the life prevailing in these bustling Canadian cities. Overall, while the weekdays entailed hard work with a great deal of learning, the weekends were all about going crazy and experiencing Canada as a whole and this is what made my time during the summer a memorable one.</p><p>Such internships are an exceptional opportunity for students to experience a different culture, work at the best labs around the world and to get a taste of living as an absolutely independent individual. For the ones looking at going for higher studies in the future, such an experience provides the ideal opportunity to get a taste of life as a graduate student at these universities. It also helps hone one’s skills at financial management along with making them realise the importance of time management. Overall, such an experience is one of the best ways to expose oneself to the world as it will be after graduation. With plenty of learning, making new friends and a good deal of enjoyment, such an experience becomes one of those moments that would bring a smile to your face when you happen to think about it or stumble upon the pictures in the near future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w4QnSGnHUd2b_XTvT27Oyg.jpeg" /><figcaption>Internship in Canada : A wholesome experience indeed</figcaption></figure><p><em>Thank you for your time in reading this blog. I hope this inspires you towards having a similar experience. Kindly </em><a href="https://www.linkedin.com/in/siddhant-haldar-2ba70a136/"><em>contact</em></a><em> me if you have any queries.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b93452a252ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>