<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Smruthi Rao on Medium]]></title>
        <description><![CDATA[Stories by Smruthi Rao on Medium]]></description>
        <link>https://medium.com/@smruthisumanthrao?source=rss-5881b96e7ba9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xU8VDN8uF91LYlhGwnkUwQ.jpeg</url>
            <title>Stories by Smruthi Rao on Medium</title>
            <link>https://medium.com/@smruthisumanthrao?source=rss-5881b96e7ba9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 23:40:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@smruthisumanthrao/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Under the Hood: Scikit-Learn’s Linear Regression.]]></title>
            <link>https://medium.com/@smruthisumanthrao/under-the-hood-scikit-learns-linear-regression-sans-backpropagation-bd78e00a484e?source=rss-5881b96e7ba9------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd78e00a484e</guid>
            <dc:creator><![CDATA[Smruthi Rao]]></dc:creator>
            <pubDate>Fri, 01 Mar 2024 19:43:41 GMT</pubDate>
            <atom:updated>2024-03-04T20:12:42.138Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lkqc68a6b7_TLALs5fmI6A.png" /></figure><p>Backpropagation is a common optimisation technique employed by neural networks. To the uninitiated, it is an elegant solution relying on gradient descent, which drastically reduces number of iterations required to arrive at the optimal solution. The gradient indicates the direction of steepest ascent, so moving in the opposite direction (negative gradient) leads to a decrease in the function value. By repeatedly updating the parameters based on the gradient, the algorithm converges towards the minimum of the function.</p><p>Linear Regression, of course, is a supervised learning technique that essentially determines the linear relationship between dependent and independent variables. The diagram affixed below offers a visualisation that further drives in this difference the difference between linear and non linear regression techniques.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*7YZS7MpHQPs7J1opBGyCjg.jpeg" /></figure><p>While it is possible to express Linear Regression as a perceptron, a very basic neural network unit that performs a linear operation on data, this is considered bad practice due to the unnecessary complexity and computational overhead without providing significant benefits over simpler linear regression techniques. Neural Networks are computationally expensive and are better suited for learning complex, non linear relationships between features.</p><p>That being said, let us consider how to express linear regression as a perceptron as a fun exercise. The weights would have to be coefficients and the y-intercept acts as bias, which are the inputs to the perceptron. The perceptron itself performs the linear transformation, and outputs the resulting value. This describes a single forward pass — which is in most cases, a terrible estimate. The model needs to calculate loss, which is a numerical measure of how close the predicted value is to the actual value, according to the loss function defined — in this case, the simple Mean Squared Error (MSE).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/455/1*2EDmqNZwKGLXL8hqg0y58w.png" /><figcaption>Single Layer Neural Network Perceptron.</figcaption></figure><p>The parameters in the network — weights, and bias, are updated by subtracting the value of the gradient of the respective parameter with respect to the loss function. This gradient is also multiplied by a learning rate to avoid the gradient from being too large and diverging. Essentially, backpropagation. The new values are then sent through the perceptron and a loss is calculated once again, and this process repeats until the loss stabilises, which means it has reached minima. This repeated iteration is how the network ‘learns’ the correct values of parameters which results in a predicted value that is close to the actual value. Let me repeat, the iterative nature of backpropagation is what allows us to achieve good results.</p><p>Scikit learn does not use backpropagation. Neither does it follow an iterative algorithm. Instead, it is somehow able to calculate the correct parameters in one forward pass. How is this possible? We know from the previous neural network example that a single forward pass is usually wildly inaccurate. So how exactly does scikit learn compute the correct values in one go?</p><p>The answer lies in the Ordinary Least Squares method which aims to find the values of the linear regression model’s parameters (i.e., the coefficients) that minimize the sum of the squared residuals. The residuals are the differences between the observed values of the dependent variable and the predicted values of the dependent variable given the independent variables.</p><p>(1). m = (Σ (xᵢ — x̄) (yᵢ — ȳ)) / (Σ (xᵢ — x̄)²)</p><p>(2). b = ȳ — m x̄</p><p>This method is straightforward for linear relationships and has a unique solution if we have enough data points. In contrast, nonlinear problems require iterative solutions and can have multiple possible solutions. Although, it is important to note that OLS is assumes that the data belongs to a normal distribution, and that the features are independent of each other. In other cases, it is advised to use other linear least squares techniques such as Generalized Least Squares, or Weighted Least Squares.</p><p>Hope you enjoyed this article, and happy learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd78e00a484e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[On anomaly detection.]]></title>
            <link>https://medium.com/@smruthisumanthrao/on-anomaly-detection-925d97c11cba?source=rss-5881b96e7ba9------2</link>
            <guid isPermaLink="false">https://medium.com/p/925d97c11cba</guid>
            <category><![CDATA[graph-neural-networks]]></category>
            <category><![CDATA[anomaly-detection]]></category>
            <dc:creator><![CDATA[Smruthi Rao]]></dc:creator>
            <pubDate>Sun, 28 Jan 2024 21:43:24 GMT</pubDate>
            <atom:updated>2024-01-28T21:43:24.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*VNQp3U5cMahnVzgzpt9pQg.png" /><figcaption>In everyone’s favorite movie, ‘Interstellar’, this bookcase holds a special significance. It is the site where the main characters observe a gravitational anomaly.</figcaption></figure><p>An anomaly is an outlier or deviation from normally observed patterns or distributions. It is a seemingly bizzare, assumedly rare event that occurs once in a while just about everywhere, from numbers on a chart or figures on a graph to real-life situations.</p><p>Imagine you’re looking at a graph, and suddenly there’s a weird spike or drop that doesn’t fit the usual pattern. That’s an anomaly, and it makes us dig deeper to figure out what caused it. Or think about everyday life — sometimes there’s a moment or event that breaks the routine and stands out. Maybe it’s a rare bird in your backyard or a surprising twist in a TV show. They can sometimes play an important role in allowing us glimpses into previously undiscovered insights, but can also be a nuisance for various industries including but not limited to manufacturing, healthcare, finance, and agriculture. In these use cases where manual detection is laborious and possibly inaccurate, machine learning can step in to make anomaly detection an automated process.</p><p>Anomalies are not to be confused with novelties, however, which refer to identification of new or unobserved patterns within the data. The difference between the two is subtle, but important for the discussion at hand. Imagine a dataset of Bengal tigers — most instances would be an image of the orange tiger, but a rare white tiger could be considered a novelty. An anomaly on the other hand, would be a random image of a horse — which doesn’t conform to the dataset at all, and is completely unexpected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*hNTdL_9VssEwHBCEjgH6mA.png" /></figure><p>Let’s touch on the three types of anomaly detection: supervised, unsupervised, and semi-supervised.</p><ol><li><strong>Supervised Anomaly Detection</strong>:</li></ol><p>In the realm of supervised anomaly detection, the model is trained on a dataset where both normal and anomalous instances are explicitly labeled. The model learns to distinguish between the two classes, allowing it to identify anomalies with guidance from labeled examples. However, this approach requires a sufficiently large and well-labeled dataset, which might be challenging to obtain in some real-world scenarios.</p><p>2.<strong> Unsupervised Anomaly Detection:</strong></p><p>Unsupervised anomaly detection operates without labeled data. The model learns to identify anomalies by understanding the normal patterns within the dataset. Common techniques include clustering, statistical methods, and density estimation. Unsupervised methods are valuable when labeled anomalous instances are scarce or unavailable, allowing the model to autonomously discern irregularities.</p><p>3. <strong>Semi-Supervised Anomaly Detection</strong>: Semi-supervised anomaly detection strikes a balance between the supervised and unsupervised approaches. In this scenario, the model is trained on a dataset containing both labeled normal instances and unlabeled data that may include anomalies. This setup leverages the available labeled information while allowing the model to generalize to identify anomalies in the unlabeled portion. Semi-supervised methods are particularly useful when obtaining a fully labeled dataset is challenging but some labeled instances are available for training.</p><p>When addressing the intricacies of outlier detection, traditional algorithms encounter challenges, particularly when confronted with datasets featuring images and sequences. Their accuracy and performance tend to diminish in capturing the intricate structures present in these data types. As the data volume scales up, reaching gigabytes, the limitations of traditional methods become increasingly apparent due to the problem of overfitting. As the number of iterations increases, the test loss, which should theoretically fall and then stabilize, actually reaches a minimum and starts increasing due to overfitting of the training data.</p><p>Recognizing the inadequacies of conventional approaches, the necessity for large-scale anomaly detection becomes evident, especially in scenarios involving massive data volumes. This is where the spotlight shines on Deep Anomaly Detection (DAD) techniques, since they have the ability to autonomously learn hierarchical discriminative features from raw data, and adapt to the dynamic and evolving delineation between normal and anomalous behavior.</p><p>Graph Neural Networks (GNNs) are promising in the realm of anomaly detection, particularly when dealing with data that can be represented as a graph or network structure. The inherent design of GNNs caters to the intricacies of graph-structured data, a characteristic shared by various real-world datasets, including social networks, citation networks, and biological networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*OK3KLwfdvzGx1g00o53b7w.png" /><figcaption>Diagram of a graph neural network.</figcaption></figure><p>A key strength of GNNs lies in their ability to comprehend both local and global contextual information. By considering not only the features of individual nodes but also the relationships with neighboring nodes, GNNs excel at capturing nuanced patterns. This feature proves invaluable in detecting anomalies that disrupt the overall structure or patterns within the graph.</p><p>GNNs also shine in their adaptability to irregular data. In scenarios where anomalies manifest as irregular patterns, GNNs can dynamically adjust and identify deviations from established norms. This adaptability is complemented by GNNs’ robustness in handling varying graph structures, making them suitable for scenarios where anomalies may present as structural abnormalities.</p><p>In the realm of semi-supervised learning, GNNs showcase their prowess, leveraging both labeled and unlabeled data. This is particularly advantageous in anomaly detection, where labeled anomalous instances may be limited. The scalability of GNNs further adds to their appeal, as they efficiently process and learn representations even as the size of the graph grows, making them suitable for large-scale applications like cybersecurity, fraud detection, or network intrusion detection.</p><p>Additionally, certain GNN architectures can effectively handle dynamic graphs, where the structure evolves over time. This adaptability proves crucial in scenarios where anomalies may emerge and evolve dynamically. While GNNs offer these compelling advantages, their effectiveness hinges on careful consideration of the data’s nature and the specific characteristics of the anomalies at hand. Tailoring the GNN architecture and training process to the intricacies of the anomaly detection task enhances their overall efficiency. Stay tuned for a GNN implementation of an anomaly detection model, and happy learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=925d97c11cba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[On solar flares and how to predict them (Hint: DeFN)]]></title>
            <link>https://medium.com/@smruthisumanthrao/on-solar-flares-and-how-to-predict-them-hint-defn-a98cdd840a7e?source=rss-5881b96e7ba9------2</link>
            <guid isPermaLink="false">https://medium.com/p/a98cdd840a7e</guid>
            <dc:creator><![CDATA[Smruthi Rao]]></dc:creator>
            <pubDate>Wed, 24 Jan 2024 13:27:31 GMT</pubDate>
            <atom:updated>2024-01-24T13:27:31.209Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*Ol5DXv5CKMvLsST576Iqpg.gif" /><figcaption>Above is the animation captured by the Solar Ultraviolet Imager (SUVI), aboard the GOES East Satellite of a massive solar flare which occured on the far side of the sun, captured on Feb 15, 2022.</figcaption></figure><p>The most spectacular phenomenon related to sunspot activity is the solar flare, which is an abrupt burst of magnetic energy from the sunspot region. They release energy, light and high speed particles into space. The sun has a dynamic magnetic field, which means that it keeps changing and evolving over time. When two magnetic field lines cross over, it leads to a burst of energy so intense that it is a driving force that influences space weather. They’re occasionally accompanied by Coronal Mass Ejections (CMEs) — consisting of plasma, solar wind, and magnetic loops.</p><p>This is all well and good, and extremely interesting to know, but why would we need to predict when solar flares occur? Turns out, although the sun is millions of kilometres away, solar weather affects the earth’s geomagnetic environment and also satellites in space. When a CME interacts with the earth’s magnetosphere, it can trigger geomagnetic storms which can wreck havoc on satellite operations, communication systems and power grids. Additionally, for astronauts on space missions, it is unsafe to be exposed to heightened levels of solar radiation during a solar flare. In short, predicting solar flares is crucial for space weather forecasting in order to mitigate its impact on technology and astronauts as much as possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wf80Jd66ez9FoulLFUb4KQ.jpeg" /><figcaption>A visualisation of the interaction between the magnetic fields of the earth and the sun.</figcaption></figure><p>This is where machine learning comes in! The application of supervised machine learning methods for solar flare prediction has been a hot topic in the past few years. However, the challenge in building a model lies in devising methods for verification and validation. This is due to the fact that models are sensitive to which period in the solar cycle data is collected from (solar cycles are thought to follow an 11 year cycle). The solution to this would be to predict flares in specific periods of the solar cycle in real time. This implementation comes from Nishizuka et. al’s paper titled “Operational solar flare prediction model using Deep Flare Net”.</p><p>The deep flare net (or DeFN, as it is referred to in the paper), is designed to predict solar flares occuring in the following 24 hours after observing magnetogram images. The authors have proposed the use of a time-series CV model for evaluations, which was found to provide an almost unbiased estimate of the true error. The amazing thing about this paper, is that the authors have very generously disclosed the entire architecture and code on github, and all the datasets are publically available. I am yet to try this out, having written this article immediately after coming across this gem of a paper, but lemme know in the comments if you end up implementing :)</p><p>The paper, along with github and dataset links can be found at — <a href="https://earth-planets-space.springeropen.com/articles/10.1186/s40623-021-01381-9">https://earth-planets-space.springeropen.com/articles/10.1186/s40623-021-01381-9</a></p><p>Happy Learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a98cdd840a7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[On the concept of singularity.]]></title>
            <link>https://medium.com/@smruthisumanthrao/on-the-concept-of-singularity-4c79de63908e?source=rss-5881b96e7ba9------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c79de63908e</guid>
            <dc:creator><![CDATA[Smruthi Rao]]></dc:creator>
            <pubDate>Wed, 24 Jan 2024 13:05:12 GMT</pubDate>
            <atom:updated>2024-01-24T13:05:12.358Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Get the down-low on the basis (pun intended) of linear algebra, the most ubiquitous field of mathematics, and how it forms the foundation for machine learning.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*P0mPc2taljUFR46WZVJ2qg.jpeg" /></figure><p>Linear algebra is the field of mathematics that deals with vector spaces and the linear mappings between these vector spaces. Singularity is an important topic in mathematics. Coincidentally, the centre of a black hole is also named ‘singularity’, a single point where density becomes infinite and the laws of physics as we know them break down. This has absolutely nothing to do with out discussion today, but I thought it might be a fun little fact worth mentioning since singularity is used to define various distinct topics in physics and mathematics where certain quantities become infinite or undefined.</p><p>A vector space is the set of all vectors in a mathematical structure that are created by some linear combination of the basis vectors of said structure where the number of basis vectors is called its dimension. The linear mappings between vector spaces is called a linear transformation, and as the name suggests, it conserves the linearity of vectors. That’s a really complicated way to say linear algebra is all about vectors, vector spaces, matrices, and linear transformations, but it’s vital to have a thorough understanding of the terminologies and definitions in mathematics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*nwe_6Xmyv4ce_vYD.jpeg" /></figure><p>Consider this 2D plane example. It is two dimensional, therefore has two basis vectors. Both sets of vectors start from origin, however, they are obviously different, leading to different vector spaces. Vector space 1 can be transformed into vector space 2 by making it undergo a linear transformation.</p><p>In the true spirit of mathematics, let us start off with a few definitions which will make it easier for you to follow along with this article, as well as any formal literature on machine learning. Here goes —</p><ol><li>Vectors: are quantities that possess both magnitude and direction. They can be geometrically visualised as arrows in space. They have as many dimensions as the mathematical structure in which they are defined.</li><li>Matrices: are simply rectangular arrays of numbers, where rows represent dimensions, and columns represent vectors. They are highly versatile, and can be used to represent systems of linear equations, linear transformations, and various other mathematical structures.</li><li>System of equations: A system of equations refers to a set of equations which are simultaneously satisfied by a set of variables, referred to as the solution of the system.</li></ol><p>That should do for now. Let’s talk more about systems of equations. Think of equations as mediums that communicate certain information to you. In a system, each equation does this — it carries some information that we, using our knowledge of linear algebra, must decode. Based on the number of pieces of information we’re getting and whether they make sense when put together and viewed as a coherent system, they may be classified as singular or non singular. There are three potential outcomes to this situation — a single solution (singular system), infinite solutions (non-singular system) or no solution (non singular — system).</p><p>How do we determine whether matrices are singular or not? It totally depends on the information they carry. If for example, a system has three variables, and it carries information about all three, which happens to satisfy all three equations simultaneously, then it has a single solution set, and the system is a singular system. Obviously, if a single solution does not exist, the system is non singular and this can happen in two ways — redundancy, or contradiction. A non singular system is redundant if the information provided is partial, and has been repeated. This is incomplete, and the remaining variables can assume absolutely any value and it will still satisfy the system, which is why redundant systems are said to have an infinite number of solutions. Contradictory systems contradict themselves by implying that a variable can assume two or more different values, which is simply not possible — we require variable values to be consistent across the entire system. Hence, contradictory systems are non singular having no solution.</p><p>To really drill in these concepts, let us visualise a system of two dimensional equations on a plane. Consider the following sets of equations —</p><ol><li>x + y = 2<br>x = y</li><li>x + y = 1<br>2x + 2y = 2</li><li>x + y = 1<br>x + y = 2</li></ol><p>Let us visualize them geometrically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*ttfR8BdjtcL1U8BL.jpeg" /></figure><p>System 1: two lines are meeting. System2: two lines are overlapping. System 3: two lines are parallel.</p><p>Observe how in System 1, the two lines representing our first set of equations intersect at a certain point. What does this mean, physically? Recall that a line is nothing but a collection of points that satisfy a two variable linear equation. Since these lines are intersecting, they pass through the same point which means that this point satisfies both equations simultaneously. Hence, system 1 is singular.</p><p>In system 2, the two equations are of the same line essentially. The lines representing the equations hence overlap at all points which leads to an infinite number of solutions. The information we are gaining from this is redundant, therefore the system is non singular as the solution set is not constrained to a single solution.</p><p>In system 3, notice how the lines are parallel to one another. Geometrically, they will never intersect. Physically, this means that no such point exists which can simultaneously satisfy both equations. Hence, this system is non singular with no solution.</p><p>Now, let’s do something interesting with the lines. Let’s remove all the constants and make them zero. How might this affect singularity? The equations now become-</p><ol><li>x + y = 0<br>x — y = 0</li><li>x + y = 0<br>x + y = 0</li><li>x + y = 0<br>x + y = 0</li></ol><p>Notice how it affects their respective graphs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*ot1wD7qCmu370Guf.jpeg" /></figure><p>We see that the singularity is still preserved irrespective of the constants.</p><p>Since the singularity is preserved irrespective of the constants, we may conclude that singularity depends only on the coefficients of equations and not their constants. This is a very important result in linear algebra, which influences the solvability of linear systems, invertibility of matrices, and the geometric properties of vectors.</p><p>Happy learning, and stay tuned for future articles!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c79de63908e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>