<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhishek Salian on Medium]]></title>
        <description><![CDATA[Stories by Abhishek Salian on Medium]]></description>
        <link>https://medium.com/@abhisheksalian?source=rss-1244b22c8a52------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5221C3Dbs4Gc8Lu0MuW--w.png</url>
            <title>Stories by Abhishek Salian on Medium</title>
            <link>https://medium.com/@abhisheksalian?source=rss-1244b22c8a52------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:51:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abhisheksalian/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating Own PyPI Server in Gitlab for Serving Private Packages]]></title>
            <link>https://abhisheksalian.medium.com/creating-own-pypi-server-in-gitlab-for-serving-private-packages-edd9f66b5bc0?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/edd9f66b5bc0</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pypi]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[pip]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sat, 30 Nov 2024 14:52:16 GMT</pubDate>
            <atom:updated>2024-11-30T14:52:16.485Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GH5eQTH64GMADr0pm_IEUQ.png" /><figcaption><a href="https://www.junia.ai/">AI Generated Image</a></figcaption></figure><p>Often we want to create and distribute an in-house library to users/clients in a private mode, in an easy and accessible manner. As we install any package from a public package registry server, for example, we can install Python packages using ‘pip install package_name’ from the PyPI server, and all the installation processes are automatically taken care of. The only problem with the PyPI registry server is that it has public visibility so we can’t upload private packages to such a server. So in this blog, we will learn an easy way to create our very own PyPI private server with greater control over distribution via Gitlab.</p><p>Excited? So let’s get started.</p><p>Before we begin just a small note, we will be using the PyPI registry setup, which means we will publish a Python package. Feel free to refer to the <a href="https://docs.gitlab.com/ee/user/packages/package_registry/">documentation</a> for other supporting package managers like Maven, npm, etc.</p><p>So to understand the usage and get hands-on with it, we will start by creating a package named ‘<em>my_py_package’</em> inside ‘<em>My_Package</em>’. We will keep the package template design simple. Once we create this package which contains all necessary files and functionalities we will upload it in Gitlab lab by creating a GitLab repository as normally we would do to upload code in the repository. After that, we will do some settings in GitLab so that we can use it like a server as we generally use a PyPI server to install Python packages from it.</p><p>To create a basic package we need the following directory structure</p><pre>|-- My_Package<br>    |-- my_py_package<br>    |   |-- __init__.py<br>    |   `-- main.py<br>    `-- setup.py</pre><ol><li>We will create a package named my_py_package inside <em>My_Package</em> and remember when we do pip install, we would consider <em>my_py_package</em> so <em>pip install my_py_package</em> would be the command to install package.</li><li>Inside the <em>My_Package </em>folder, we will create a file named setup.py which is an important file</li><li>Inside the <em>My_Package</em> folder, we will create the <em>my_py_package</em> folder that contains all functional codes that we want to use when we install and import this library.</li><li>Additionally, we can add more files depending upon the requirement like requirements, License, Readme, utility files, etc.</li><li>For now, we will create two files inside <em>my_py_package</em> folder.</li></ol><ul><li><strong>__init__.py:</strong>- This file is used to mark it as a Python package directory for more refer to this wonderful thread in <a href="https://stackoverflow.com/questions/448271/what-is-init-py-for">stack overflow</a></li><li><strong>main.py:</strong>- This file will contain the functional code that functions, classes, etc.</li></ul><p>Now that we have understood the package structure and files it&#39;s time for coding!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*DUuueR3UxVQFWQGWtBBCdQ.png" /></figure><p>First, we will configure the setup.py file so let us understand the setup.py file.</p><p>For the ones who are new to package creation let me give a short introduction to the setup.py file.</p><p>setup.py helps to turn the package into a pip-compatible <a href="https://realpython.com/python-wheels/">wheel file</a> by using a command which will be discussed later. The setup.py file is like a label or header for your package, which contains a version of a package, author name, package to be installed, and many more necessary things for package installation.</p><p>setup.py contains the following:-</p><pre>import setuptools<br><br><br>setuptools.setup(<br>    # you replace it with your custom package name<br>    name=&quot;dummy_package&quot;, <br>    version=&quot;0.0.1&quot;, <br>    author=&quot;Author Name&quot;,<br>    author_email=&quot;author@example.com&quot;,<br>    description=&quot;A small example package&quot;,<br>    packages=setuptools.find_packages(),<br>    # if any dependencies we want to install then<br>    #install_requires=[&quot;some library&gt;=version&quot;],<br>    classifiers=[<br>        &quot;Programming Language :: Python :: 3&quot;,<br>        &quot;Operating System :: OS Independent&quot;,<br>    ],<br>    python_requires=&#39;&gt;=3.6&#39;,<br>)</pre><p>Once the setup.py file is ready we have to create pypirc file for easy uploading of the package to the GitLab server</p><pre># Create pypirc package for uploading package (for linux users)<br>gedit ~/.pypirc</pre><pre># inside it write following<br>[distutils]<br>index-servers =<br>    gitlab</pre><pre>[gitlab]<br>repository = https://&lt;gitlab base url&gt;/api/v4/projects/&lt;project id&gt;/packages/pypi<br># gitlab base url for default account gitlab.com<br># for enterprise account gitlab.example.com<br>username = username (from deploy token creation)<br>password = token (from deploy token creation)</pre><blockquote>💡 Note: A developer needs to use the above pypirc file so that the deployment can be easily done.</blockquote><p>A username needs to be created in the deploy token section and a password is auto-generated. So let us peep into deploying the token feature of Gitlab.</p><h3><strong>Deploy Token</strong></h3><p>There are many other methods to share the credentials like via personal access token and so on but Deploy tokens offer much more degree of control over reading and writing of the package as per user or group of users.</p><p>Deploy token functionality in Gitlab offers the following:</p><ul><li>We can create a unique username and password (auto-generated) for any individual.</li><li>Defining the scope.</li><li>Expiry of the token.</li><li>Revocable at any point in time.</li><li>No compromising in security.</li><li>Great ease of managing tokens.</li></ul><p>Deploy token method helps in gaining more systematic control in the management of the package distribution.</p><p>Now that we have created our package and understood what a deploy token is, let us create our first deployment token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*12GFLzsviFyhN_lR_N2rZg.png" /></figure><p>Now in Gitlab go to the settings → Repository → Deploy tokens</p><p>Click on <strong>Deploy tokens</strong></p><p>Once we click on Deploy tokens we will land on the following page</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*zCAcbq3cwXRZlwCiPQzhhQ.png" /><figcaption>Image by Author</figcaption></figure><p>Once we click <strong>Create deploy token</strong> by filling in all required fields we will be shown a password do make a note of it since we cannot see the password anywhere later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*Q7EXU_NhYYPkEKgtZK3UMg.png" /><figcaption>Image by Author</figcaption></figure><p>One who builds the package needs to be given write package registry access and all general users can be given<strong> Read_package_registry </strong>access.</p><p>If a developer of the package is writing the package in the registry then the scope should be written registry and the username and password must be added to pypirc file.</p><p><strong>Once we are done with these configurations it&#39;s time to upload the package to the server.</strong></p><p>We need to install Twine to upload the Python package. The following command can be used:-</p><pre>pip install twine</pre><p>Once the twine is installed we have to build the package wheel file using the subsequent command (make sure the command terminal is opened inside the My_Package folder)</p><pre>python3 setup.py sdist bdist_wheel</pre><p>Now let us upload the package to the Gitlab server</p><pre>python3 -m twine upload --repository gitlab dist/*</pre><p>To see whether the package is successfully uploaded or not, we can go to Package &amp; Registry → Package Registry</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bVzeSccg0e019uxj7K7dqQ.png" /></figure><p>Once the library is uploaded we are good to install the library and distribute the package to users. We have to create deploy tokens with the read registry scope for individual users or groups of users so that they can install the package in their system.</p><p>Click on the uploaded package and copy the “pip install ..” command. We need to share this command with the user along with the dedicated deploy tokens that we created for them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*4G_MPe6WKMzncD_fnUnSfg.png" /><figcaption>Image by Author</figcaption></figure><p>Now if we or users want to install the package, simply need to add our token credentials. Project ID will be present in the link only token credentials will be variable depending on allocation.</p><pre>pip install package1 --extra-index-url &lt;https://&lt;username&gt;:&lt;token&gt;@gitlab.com/api/v4/projects/&gt;&lt;project_id&gt;/packages/pypi/simple</pre><p>And package should be successfully installed in the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*ySv-O10xszE-yVgqtUS8dg.png" /></figure><p>Now we can manage, revoke, and distribute private packages to users in a hassle-free way keeping intact the integrity of the library. This blog was w.r.t to python package. But the process remains the same for other Package registries too (refer to <a href="https://docs.gitlab.com/ee/user/packages/package_registry/">documentation</a>).</p><p>I hope you have enjoyed reading this blog.</p><p>Cheers to learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=edd9f66b5bc0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Moving Average Filter: Towards Signal Noise Reduction]]></title>
            <link>https://medium.com/codemonkhq/moving-average-filter-towards-signal-noise-reduction-27f123c13e6c?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/27f123c13e6c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[noise-reduction]]></category>
            <category><![CDATA[signal-processing]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Tue, 31 May 2022 12:44:25 GMT</pubDate>
            <atom:updated>2024-04-23T07:58:44.957Z</atom:updated>
            <content:encoded><![CDATA[<p>This blog is all about understanding the Moving Average (MA) filter in a more discrete and applied way. We will dive deep into various types of moving average filters and will cover the enthralling real-life applications built around these core concepts. <em>This blog was originally published on </em><a href="https://codemonk.io/blog/moving-average-filter/"><em>Codemonk’s blog </em></a><em>website. For better content display and quality reading, I would recommend visiting the blog.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6iM9zM7zLUGTx6Lv.jpg" /><figcaption>Moving from noise to clarity (Credits: <a href="https://www.linkedin.com/in/nidhinalikkal">Nidhin Alikkal</a>, Fellow Codemonker)</figcaption></figure><p>A signal is a physical quantity that varies with time, space, or other independent variables. In signal processing, a signal is a function that conveys information about a phenomenon [<a href="https://en.wikipedia.org/wiki/Signal">1</a>]. A signal can be 1 — Dimensional (1D) like audio and ECG signals, 2D like images, and could be any dimensional depending upon the problem we are dealing with in general.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/0*7Dh9ln4weiUhZVBD.png" /><figcaption>Fig.1: Illustration of various signal Image, Text, &amp; Audio.</figcaption></figure><p>Signals in raw form are not always usable, we need to pass them through the filtering layer to ensure bare minimum quality for further analysis. So first let us try to understand signal filtering and its significance.</p><h3>Signal Filtering</h3><p>Signal filtering is a primary pre-processing step that is used in most signal processing applications. The raw signal is not always in a usable form to perform advanced analysis i.e various noises are present in the raw signal. We have to apply a filter in order to reduce the noise in the signal as a part of pre-processing step.</p><p>There are many pre-processing steps applied one of which is de-noising which is essential when signals are sampled from the surrounding environment. The moving average filter is one such filter that is used to reduce random noise in most of the signals in the time domain.</p><p>Now that we have understood the significance of signal filtering, let us understand the moving average filter.</p><h3>Moving Average Filter</h3><p>Moving Average Filter is a Finite Impulse Response (FIR) Filter smoothing filter used for smoothing the signal from short term overshoots or noisy fluctuations and helps in retaining the true signal representation or retaining sharp step response. It is a simple yet elegant statistical tool for de-noising signals in the time domain.</p><p>These filters are favorite for most Digital Signal Processing (DSP) applications dealing with time-series data. It is simple, fast, and shows amazing results by suppressing noise and retaining the sharp step response. This makes it one of the optimal choices for time-domain encoded signals.</p><p>The Moving Average filter is a good smoothing filter in the time domain but a terrible filter in the frequency domain. In applications where only time-domain processing is present Moving average filters shine, but in applications where information is encoded in both time and frequency or in frequency domain solely it can be a terrible option to choose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iYYBAdF8XH3upBWeLSFj8A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/0*HYa43uT-0uI4NVv0.png" /><figcaption>Fig. 2: Time-Frequency Domain Analysis (<a href="http://play.fallows.ca/wp/wp-content/uploads/sites/4/2017/01/Signal-Analysis.jpg">Source</a>)</figcaption></figure><h3>Types of Moving Average Filter</h3><p>There are various types of moving averages filters but on a broader level simple, cumulative moving average, weighted moving average, and exponentially weighted average filters form the basic block for most of the other variants. There are many moving average filter variants, more or less the fundamental structure boils down to four core types illustrated below figure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*QqT1M2_iCSKEN7qL.png" /><figcaption>Fig. 3: Types of Moving Average Filter.</figcaption></figure><p>This article will cover most of them on a broader level and will show use cases of it, we will also understand their variants which we had used in our research use case, and boosted performance metrics.</p><p>Now let us, deep-dive, into each of these types along with code snippets and some basic mathematical formulation.</p><p>Don’t get intimidated by coding and mathematics I have tried to keep it short and simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/0*fOtkvMDfUSlOkunL.jpeg" /></figure><h3>Simple Moving Average (SMA)</h3><p>This is one of the simplest forms of moving average filter that is easy to understand and apply to the desired application. The main advantage of the simple moving average is that we don’t need exorbitant mathematics to understand it i.e it can be interpreted by its formula itself.</p><p>The con of SMA is that it gives equal weightage to all samples due to which it does not suppress the noisy signal effectively.</p><p>Let’s take an example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfVNUsoJ6_WMKgVuVGBhkQ.png" /></figure><p>Let us implement this simple moving average filter using Python. We will be using the convolution concept for convolving the input signal with all ones with the given window size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*F7K_ez0OwmO27fDakyLRpQ.png" /></figure><pre>import numpy as np<br><br><br>def simple_moving_average(signal, window=5):<br>    return np.convolve(signal, np.ones(window)/window, mode=&#39;same&#39;)</pre><p>We will choose a simple sine wave and superimpose random noise and demonstrate how effective is a simple moving average filter for reducing noise and restoring to the original signal waveform.</p><pre>Fs = 8000 # sampling frequency = 8kHz<br>f = 5 # signal frequency = 5 Hz<br>sample = 8000 # no. of samples<br>time = np.arange(sample)<br>original_signal = np.sin(2 * np.pi * f * time / Fs)   # signal generation<br>noise = np.random.normal(0, 0.1, original_signal.shape)<br>new_signal = original_signal + noise</pre><p>Following is the plot showing the effectiveness of the simple moving average filter for random noise reduction:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*3RrUL4H7fsgAkLva.png" /><figcaption>Fig. 4: Effectiveness of the simple moving average filter for random noise reduction.</figcaption></figure><p>One characteristic of the SMA is that if the data has a periodic fluctuation, then applying an SMA of that period will eliminate that variation (the average always contains one complete cycle). But a perfectly regular cycle is rarely encountered.</p><h3>Cumulative Moving Average (CMA)</h3><p>CMA is a bit deviated from other types in moving average family and usability of this filter in noise reduction is nil. One benefit of CMA is that it also accounts for past data considerably by also accounting for the recent data point, unlike SMA which will be just an average of past data points within a defined sliding window size and equal weights.</p><p>The simple moving average has a sliding window of constant size k, contrary to the CMA in which the window size becomes larger as the time passes during computation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*Q8Cc0uHPybGpJvM6XkuUlg.png" /></figure><p>This is called cumulative since the ‘n+1’ terms will also account for the cumulative of ’n’ previous data points while averaging the points.</p><pre>import numpy as np<br><br><br>def cumulative_moving_average(signal):<br>    cma = []<br>    cma.append(signal[0]) # data point at t=0<br><br>    for i, x in enumerate(signal[1: ], start=1):<br>        cma.append(((x+(i+1)*cma[i-1])/ (i+1)))<br> <br>    return cma</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*hkKo0IDK82oodSWi.png" /><figcaption>Fig. 5: Illustration of Cumulative Moving Average.</figcaption></figure><p>It is very evident that CMA is not at all useful for reducing noise since the retrieved signal is nowhere pristine in shape as w.r.t original signal. This filter is worst to be used for reducing noise pursuit.</p><p>CMA is quite alien in terms of output as compared to its other family members.</p><p>But what we can learn from its formula and concept is we can give certain weights not necessarily equal i.e playing with weights and adjustments that was a setback in the case of SMA. This is where our next topic will come into play Weighted Moving Average.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*9-uf2QxLkIoIqJHT24gLKA.png" /></figure><p>But what we can learn from its formula and concept is we can give certain weights not necessarily equal i.e playing with weights and adjustments that was a setback in the case of SMA. This is where our next topic will come into play Weighted Moving Average.</p><p>One of the interesting applications is in the stock market where data stream arrives in an orderly manner, an investor may want the average price of all trades for a particular financial instrument up until the current time. As each new transaction occurs, the average price at the time of the transaction can be calculated for all transactions up to that point using the cumulative average [ <a href="https://rtmath.net/assets/docs/finanalysis/html/f90144b2-8ccb-4eeb-a622-4bd1ff87feb4.htm#:~:text=For%20example%2C%20an%20investor%20may,point%20using%20the%20cumulative%20average">3</a>].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/0*pRQrNnfahVbcwVfL.png" /><figcaption>Fig. 6: Illustration of CMA application (<a href="https://rtmath.net/assets/docs/finanalysis/html/f90144b2-8ccb-4eeb-a622-4bd1ff87feb4.htm#:~:text=For%20example%2C%20an%20investor%20may,point%20using%20the%20cumulative%20average">Source</a>).</figcaption></figure><h3>Weighted Moving Average</h3><p>Weighted Average works similarly to that of a simple moving average filter, inhere we are considering the relative proportion of each data point within a window length. SMA is unweighted means while WMA contains weight for relative proportion adjustments which makes it even more captivating for application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tcHWCEGTZlmZ512atLKAMw.png" /></figure><p>The purpose of giving weights is to give more importance to a few data points over the other data points. And this weight depends on the application. One of the applications is in image processing to specific the filters we use. Those filter coefficients are nothing but the weights and the local patch of pixel values to which the filters are being applied serves as the data points and the combination serves as the weighted average. There are many filters for e.g blurring filters (nothing but equally weighted filter).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*L1m5Z1IIkW6QBjiV.gif" /><figcaption>Fig. 7: Illustration of weights (3x3 matrix) applied to the image(i.e input samples). Source: <a href="https://editor.analyticsvidhya.com/uploads/99433dnn4.gif">Analytics Vidhya</a></figcaption></figure><p>The characteristics of weights will decide what effects we want given input samples.</p><p>These WMA techniques are also being used extensively in stock markets but more than this its better variant is used like EWA or EWMA.</p><h3>Exponentially Weighted Average (EWA) or Exponential Weighted Moving Average (EWMA)</h3><p>An exponentially weighted average gives more weightage to recent data points and less to previous data points overall. This ensures the trend is maintained by still accounting for a decent portion of the reactive nature of recent data points. In comparison with SMA, this filter suppresses noisy components very well i.e effective and optimal in de-noising signals in the time domain. It is 1st order infinite impulse response(IIR) filter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*cK79pLb25uXmB-gI4j6QkQ.png" /></figure><p>For different applications, the beta coefficients definition changes but overall the meaning remain the same giving more importance to recent data points and less to previously averaged data points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P3d1Es3TocJ_bpI_b01BhA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*zujP_l_40-yj8Rj9.png" /><figcaption>Fig. 8: Weights become smaller as we go back in history.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJtFQdolMVhszUztO5W0Ug.png" /></figure><p>The significance of this EWMA or EMA in deep learning is known by very few people. The variants of this specific filter is been used in Deep Learning optimisers for faster convergence to global minima, because the loss landscape is highly stochastic especially when the data dimension is enormous, for the optimiser to converge faster with the optimal converging path by giving relative importance to the recent path to trace out a future path while making sure the new averaged data points doesn’t deviate too much from past data points. So it tries to maintain momentum while still avoiding any overshoots refer to the below figure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/0*7yD3dV0GQgKt69SG.png" /><figcaption>Fig. 9: With EMA the convergence is faster as it helps in gaining more momentum than without EMA i.e optimization process is faster with EMA than without EMA.</figcaption></figure><p>This type of moving average filter has excellent usage in many other fields like in the stock market where it is considered to be one of the most simple and reliable indicators to do analysis and making decisions. It is also used in many machine learning research.</p><p>In one of our research use cases for efficient peak detection in ECG signals, we used a moving average filter to suppress noise and it incredibly boosts the peak detection performance. But we need to be sure about where we are using this kind of filter, especially where frequency domain operations are involved it’s better to avoid this filtering.</p><h3>Comparative Study</h3><p>Now that we have a good understanding of the Moving Average filter we will dive into more analytical aspects of those filters. We will compare moving average filters w.r.t Savgol filter a time-domain smoothing filter and will compare its performance.</p><p>As we know moving average filters smoothens out the signal. The smoothing effect is inversely proportional to the window length i.e the larger the window length (up to a certain limit) more clearer the signal or more the de-noising. The following figure shows this relationship clearly. We have used Gaussian white noise with sigma = 0.1. From the plot, it is apparently clear that taking 500 points averaging works really well, the signal is quite smoother and resembles that of the original signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*4Qdi09RfGxjOhPqQ.png" /><figcaption>Fig. 10: Effects of variations in Moving Average window size.</figcaption></figure><p>Now let’s understand how effective the moving average filter is when the noise factor is more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*I91PhLHZXZs5Z4KN.png" /><figcaption>Fig. 11: Effects of Moving Average w.r.t to various scale factors of Gaussian Noise.</figcaption></figure><p>We can see in Fig 11. that there is a large amount of white noise which was added to the signal still after applying the moving average filter we are able to retrieve the original signal envelope i.e we are able to filter out the noise effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T2V3qXjbUmh9bZHpkEMfUA.png" /></figure><p>Now let us compare the smoothing effect of the simple moving average filter with that of the Savitzky-Golay filter also known as the Savgol filter. Savgol filter increases the precision of data without much distortion in signal tendency [<a href="https://en.wikipedia.org/wiki/Savitzky%E2%80%93Golay_filter">4</a>]. It is one of the most widely used smoothing signal methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*VBwDNsZ6ksx5WSrl.png" /><figcaption>Fig. 12: Comparison of SMA and Savgol filter.</figcaption></figure><p>Clearly, we can see how the SMA filter retrieves the original signal envelope even with a much stronger white noise addition too. This is suitable for Digital Communication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*5BM53Rd8y_XDqNDX.png" /><figcaption>Fig. 13: Digital Communication flowchart (<a href="https://www.tutorialspoint.com/digital_communication/images/digital_communication.jpg">Source</a>).</figcaption></figure><p>We can characterize the noise in the channel as Gaussian white noise. We can retrieve signal shape by smoothing out the signal or removing Gaussian white noise. In digital communication, it is important to consider the Shannon Channel Capacity Limit [<a href="https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem#:~:text=The%20Shannon%20limit%20or%20Shannon,for%20a%20particular%20noise%20level">5</a>] since moving the average filter is less effective for a low Signal-to-Noise ratio. But moving average filter is a basic filtering technique applied in most communication systems to reduce the impact of white noise.</p><p>Moving Average filters have many potential applications. Since the Moving average filter is an excellent time-domain filter but has an inferior response in the frequency domain it is important to consider the application before applying it. So applications where one is dealing with spectral features (or Frequency domain operations), it is best to avoid applying moving average filters in the spatial or time domain.</p><p>I hope you find this blog educative and interesting in theoretical as well as practical aspects.</p><p><strong>Cheers to filtering noise ahead!</strong></p><h3>References</h3><ol><li><a href="https://en.wikipedia.org/wiki/Signal">Signal Introduction</a></li><li><a href="https://www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_ch6.pdf">Convolution</a></li><li><a href="https://codemonk.in/blog/moving-average-filter/Cumulative%20Moving%20Average">Cumulative Moving Average</a></li><li><a href="https://en.wikipedia.org/wiki/Savitzky%E2%80%93Golay_filter">Savitzky-Golay_filter</a></li><li><a href="https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem#:~:text=The%20Shannon%20limit%20or%20Shannon,for%20a%20particular%20noise%20level">Shannon Limit</a></li></ol><p><em>Originally published at </em><a href="https://codemonk.in/blog/moving-average-filter/"><em>https://codemonk.in</em></a><em> on May 31, 2022.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27f123c13e6c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codemonkhq/moving-average-filter-towards-signal-noise-reduction-27f123c13e6c">Moving Average Filter: Towards Signal Noise Reduction</a> was originally published in <a href="https://medium.com/codemonkhq">Codemonk</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Gentle Introduction to Multi-Objective Optimisation]]></title>
            <link>https://medium.com/codemonkhq/a-gentle-introduction-to-multi-objective-optimization-7994aa0f7130?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/7994aa0f7130</guid>
            <category><![CDATA[pareto]]></category>
            <category><![CDATA[genetic-algorithm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[multi-objective]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sun, 05 Dec 2021 09:05:26 GMT</pubDate>
            <atom:updated>2024-04-23T07:57:47.400Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*6eQAjNY2Po-dbQF3.png" /><figcaption>Loss Landscape [2]</figcaption></figure><p><em>A beginner-friendly introduction to understanding Multi-Objective optimisation core concepts, addressing problems of applying 1D optimisation in Multi-Objective tasks, and the usefulness of multi-objective approaches in many real-life examples. This blog was originally published on </em><a href="https://codemonk.io/blog/a-gentle-introduction-to-multi-objective-optimization/"><em>Codemonk’s blog.</em></a><em> For better content display and quality reading, I would recommend visiting the blog.</em></p><p>Most of the tasks in machine learning have a single objective function, for example, image classification, image captioning, movie rating, etc. But there are several complex problems like Drug discovery, Hyperparameter Tuning, where a single objective function does not suffice to get the optimal solution in the solution space. This is where multi-objective optimisation comes into the picture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1005/1*ZnZLEJBboICHT1lXzjqGyA.png" /></figure><p>Before we get into the brief overview of Multi-Objective optimisation and its incredible applications, let us dive a little bit into understanding optimisation first, followed by single-objective optimisation, and then finally dive into multi-objective optimisation.</p><h3>Optimisation and Single-Objective Optimisation</h3><p>All deep learning algorithms use an optimisation algorithm that helps the network maximise or minimise an objective function depending on the use case. In general, optimisation refers to the task of either minimising or maximising some function <em>F(x)</em> by altering <em>x,</em> where<em> ‘x’</em> is input [1].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*tA3i6F9VcatnBR5jTTls-w.png" /><figcaption>Fig. 1: Input mapped to an output.</figcaption></figure><p>For example, we want to build a dog 🐶 vs cat 😺 image classification model. We have prepared the dataset. We want the model to accurately classify the image, which means we want maximisation of model prediction accuracy; in another way, we can also redefine my objective by saying we want minimisation of the loss. Whether we want to maximise the accuracy or minimise the loss, the objective of the model is achieved. So the objective is only one. This is known as Single-Objective Optimisation.</p><h3>Multi-Objective Optimisation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*6paPS5-lRGrzUb5pwARO0w.png" /></figure><p>In multi-objective optimisation problems, we try to optimise many objective functions simultaneously while trying to find a balance between all competitive objective functions without many trade-offs. However, it is tough to narrow the solution search to one optimal solution since there are many feasible solutions in the solution space, and some problems have intractable solution space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*7FC2lAymXpvf9wEDvThvTA.png" /></figure><p>Thus, whenever we deal with Multi-Objective Optimisation, we need to have some tools to analyse the solution space and make the optimal choice as per the problem. Sometimes for specific use cases, to find an optimal solution, domain knowledge plays a pivotal role in selecting the solution considering the trade-offs. For example, in Drug Discovery, one must consider several variables like stability, effectiveness, toxicity, etc. So, often linear 1D optimisation fails to cover the optimal solution considering many objectives.</p><h3>Problem with Linear 1D Optimisation</h3><p>Now there might be a question, “What is the problem with single optimisation or 1D optimisation?” well, the shortcomings of 1D optimisation on multi-objective tasks are that we often land upon different solutions when trying to optimise one objective function over another. So, for example, let’s say we have two objective functions, so if we apply the 1D optimisation method, we have to apply it to one objective function at a time i.e 1D optimisation on the objective-1 keeping the objective-2 constant or hidden, once the objective-1 is optimised we then apply optimisation on objective-2. So we get an optimal value which might not be the same as following the same process but this time first optimising objective-2 and then objective-1. So rather than a narrow view, we want a Bird’s Eye View of a solution space to find an optimal solution.</p><p>Let’s take the example of buying a car to understand this problem in a more crystal clear way. Let’s say my objective is to buy a good mileage, low-cost car. The X-axis represents ‘Mileage’ as we move towards right from the origin, the better the mileage. The Y-axis represents the ‘Cost’ of the car; as we move upward, cost decreases. So we name X-axis &amp; Y-axis ‘Good Mileage’ &amp; ‘Low-Cost’ to better understand the concept.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/0*GpizoYt-yQpxtUgq.png" /><figcaption>Fig 2. : List of cars (car showroom) plotted as per mileage and cost.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*SM8YDTCiZBPWqPTUK-4KHg.png" /></figure><p>Essentially, we have to minimise cost and maximise mileage for our use case. Applying linear 1D optimisation in multi-objective tasks is like treating one dimension as hidden from an understanding point of view.</p><p>We have the following scenarios -</p><ol><li><strong>Scenario 1</strong>: Applying 1D optimisation on objective-1 i.e. Good Mileage first, followed by application of 1D optimisation on objective-2, i.e. Low-Cost.</li><li><strong>Scenario 2</strong>: Applying 1D optimisation on objective-2, i.e. Low-Cost first, followed by applying 1D optimisation on objective-1, i.e. Good Mileage.</li></ol><p>Let’s make one more assumption for understanding the concept in a much better way, the car showroom is dark, and I am holding a torch to find my desired car, i.e. low cost &amp; good mileage (I know nobody wants to buy their desired car this way 😂).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/0*NLiy-uzKMtsrMHXi.png" /></figure><h3>Scenario 1: Good Mileage followed by Low Cost</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bu8ATQuzqU6KtIdG.png" /><figcaption>Fig 3.: Optimising Good Mileage car first then Low-Cost car</figcaption></figure><ol><li>Referring to the above (Fig. 3(a)), we want a car with good mileage, so we will turn ‘ON’ the torch and move towards the last car we can see i.e. car ‘C3’.</li><li>After we know we have arrived at the best Mileage Car (given the torch visibility) in the showroom, the second objective is to find a low-cost car, so we will turn my torch towards that side (refer to Fig 3. (b)).</li><li>The only car we can find is the car ‘C4’.</li></ol><h3>Scenario 2: Low Cost followed by Good Mileage</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/0*ox98kRD2M9y8QPh_.png" /><figcaption>Fig 4.: Optimising Low-Cost car first then Good Mileage car</figcaption></figure><ol><li>Referring to the above (Fig. 4(a)), we want a car which is low cost so we will turn ‘ON’ the torch and move towards the low-cost car we could see i.e car ‘C5’.</li><li>We know now that we have arrived at the best low-cost car possible (given the torch visibility) in the showroom. The second objective is to find a good mileage car, so I will turn my torch towards that side (refer to Fig 4. (b)).</li><li>The only car we can find is the car ‘C6’.</li></ol><p>Below is the summarised table of the two scenarios discussed so far:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*W59nWugXdZSiHdV4.png" /><figcaption>Table 1: Summarised optimisation result.</figcaption></figure><p>It’s weird, right 🤔? So in scenario 1, we got car C4, and in scenario 2, we got car C6.</p><p>So if we choose to go with the 1D optimisation method, we might get a different solution using a different strategy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*Hqa0p7uz4YnSlN_Z77pprA.png" /></figure><p>We can conclude from Table-1 that we get different optimal solutions for different optimisation strategies. This is the major drawback of 1D optimisation for multi-objective tasks.</p><p>This is the reason why in this type of problem, <strong>Bird’s Eye View</strong> is more beneficial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*-fOvKVzVBdRR68VhGjlugQ.png" /></figure><p>Now that we have understood the setbacks of linear 1D optimisation on multi-objective tasks, let’s understand different approaches to solve the same.</p><h3>Types of Multi-Objective Optimisation Approaches</h3><p>On a broader level, multi-objective optimisation can be achieved using two approaches.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/0*O9pP6EbEvF3SzPVs.png" /><figcaption>Fig 5.: Types of Multi-Objective Optimisation (MOO) approaches.</figcaption></figure><p><strong>Scalarization Approach</strong>: The scalarization approach creates multi-objective functions made into a single solution using weights before optimisation.</p><p>The mathematical formulation of the Scalarization approach is as follows -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/0*CGhYGF4A5M4pk_Ta.png" /></figure><p>The weight of the objective function will determine the optimal solution and show us the performance priority (Refer to [4] for more details).</p><p><strong>Pareto Approach</strong>: There is a dominated solution and a non-dominated solution obtained by a continuously updated algorithm in the Pareto approach. Let’s understand a bit more about this approach.</p><h3>Pareto Approach</h3><p>Before understanding the Pareto concept, let us understand the dominance test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/0*7NRtlvDtORfgHBSR.png" /><figcaption>Fig 6. : Decision Space</figcaption></figure><p>Dominated solutions are solutions that are not the optimal ones, i.e. there exist other points which supersede that point. For example, in Fig. 6, point ‘A’ is dominated by point ‘B’, and point ‘C’ dominates both ‘A’ and ‘B’. The solutions which cannot be dominated by any point in the feasible solution space are called ‘Non-Dominated Solutions’, also known as Pareto Efficient.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*nXrhj_H60mw1B5GU.png" /><figcaption>Fig 7. : Mapping Decision Space to Objective Space [5].</figcaption></figure><p>The Non-Dominated solution set is called Pareto Optimal Solutions/Set. Pareto Front is the boundary defined by the set of all points in objective space or criterion space mapped from the Pareto Optimal Set in decision space, refer to Fig 7. Once we find the Pareto Solution, we can apply other algorithms on top of it rather than computing the intractable solution space we curtailed to a finite set. Pareto optimal solutions are sometimes referred to as “Non-Inferior” solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*qu3oH39MnVikJDYDX0hfvA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/0*yrID1F2Mq_LLTCxo.png" /></figure><p>Broadly the different types of algorithms that are improvements to basic MOO are as follows -</p><ol><li><strong>Classic MOO methods</strong><br>a. Weighted Sum method (Refer to [5] for more details on slide 10)<br>b. ε-constraints method (Refer to [5] for more details on slide 15)<br>c. Weighted metric method (Refer to [5] for more details on slide 18)<br>d. Global Criterion Approach (Refer to [6])<br>e. Goal Programming (GP) (Refer to [6])</li><li><strong>MOO using Genetic Algorithms (MOGAs)</strong> (Refer to [5] for more details on slide 22)</li><li><strong>MO Evolutionary Algorithms (MOEAs)</strong> (Refer to [5] for more details on slide 25)<br>a. Non-Elitist MOEAs<br>b. Elitist MOEAs</li></ol><h3>Applications</h3><p>There are many amazing applications of Multi-Objective Optimisation like Drug Discovery/ Design [8], Ranking problems [9], Routing Optimisations [10], Hyper-parameter Tuning, selection of relays in the ad-hoc networks [4], and many more.</p><p>Specifically, in a drug discovery case, to discover a new drug for a particular disease there are various properties such as thermostability, absorptivity, permeability, binding, potency, toxicity, penetration, etc. These properties are essential as we want the drug to be stable, should not be toxic, and should be highly effective. This is just an example of how each of these properties is an objective function in itself and needs to be treated in a multi-objective fashion to get a desirable chemical drug.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/0*UboD_gwnCf_NdJ_w.png" /><figcaption>Fig 8. : An illustration of Multi-Objective Optimisation usage for Drug Discovery.</figcaption></figure><p>It is captivating to see the potential application of multi-objective optimisation algorithms.</p><p>I hope you understood some of the fundamental underpinnings required to understand Multi-Objective Optimisation problems. Cheers to further exploration in MOO 🥂.</p><h3>References</h3><ol><li>Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press. ISBN: 9780262035613</li><li><a href="https://arxiv.org/pdf/1805.04829.pdf">Image Source</a></li><li><a href="https://www.umsl.edu/~siegelj/information_theory/classassignments/Lombardo/04_intractableproblems.html">Intractable problems</a></li><li><a href="https://www.tandfonline.com/doi/full/10.1080/23311916.2018.1502242">A review of multi-objective optimization: Methods and its applications</a></li><li><a href="https://engineering.purdue.edu/~sudhoff/ee630/Lecture09.pdf">Lecture 9: Multi-Objective Optimization</a></li><li><a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6077796">Overview of multi-objective optimization methods</a></li><li><a href="https://www.sciencedirect.com/topics/engineering/pareto-optimality">Pareto Optimality</a></li><li><a href="https://pubmed.ncbi.nlm.nih.gov/24050140/">Multi-objective optimisation methods in drug design</a></li><li><a href="https://assets.amazon.science/4d/9c/69cbef8346408349385c780cac48/scipub-1195.pdf">Multi-objective ranking optimisation for product search using stochastic label aggregation</a></li><li><a href="https://ieeexplore.ieee.org/document/6214324">Multi-objective routing optimisation using evolutionary algorithms</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7994aa0f7130" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codemonkhq/a-gentle-introduction-to-multi-objective-optimization-7994aa0f7130">A Gentle Introduction to Multi-Objective Optimisation</a> was originally published in <a href="https://medium.com/codemonkhq">Codemonk</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Problem with Taylor Series Approximation]]></title>
            <link>https://abhisheksalian.medium.com/the-problem-with-taylor-series-approximation-a2df76b7bb67?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2df76b7bb67</guid>
            <category><![CDATA[approximation]]></category>
            <category><![CDATA[taylor-series]]></category>
            <category><![CDATA[pade-approximation]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sun, 14 Nov 2021 16:46:31 GMT</pubDate>
            <atom:updated>2021-11-16T15:35:45.945Z</atom:updated>
            <content:encoded><![CDATA[<h4>This article is about the problem with the Taylor series approximation and will study a better approximation method known as Padé Approximation.</h4><p>Approximation theory is a very fundamental concept in application to many scientific fields, the primary reason being accurate modeling of a function into simpler known functions, another reason is easy numerical computation and simpler interpretation. There are different approximates depending on the usage for e.g Fourier series, Taylor Series, Power Series, etc. This blog will be more about Taylor Series and let us try to understand what is the drawbacks of the Taylor series approximation.</p><p>Taylor Series is a ubiquitous way to represent a function in terms of summation of infinite terms of polynomials i.e A function can be represented approximately in terms of polynomials summed up to infinity. For e.g</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/314/0*z_481-RAWxMrBNMu.png" /><figcaption>Fig. 1: Sine Function in Taylor series approximation form</figcaption></figure><p>To get a deeper and intuitive understanding of the Taylor series please refer to the following video link</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3d6DsjIBzJ4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3d6DsjIBzJ4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3d6DsjIBzJ4%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/3e42dd1828caebfbf62496c2d4837c61/href">https://medium.com/media/3e42dd1828caebfbf62496c2d4837c61/href</a></iframe><p>In mathematical form, the Taylor series can be represented by the following polynomial series as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4mQiJAbfUFy-zJbm.png" /><figcaption>Fig. 2: Taylor Series.</figcaption></figure><p>But most often we use the Maclaurin Series which is a reduced form of Taylor Series Expansion i.e we equate a = 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/0*UY9oDy13NNOS0mph.png" /><figcaption>Fig. 3: Maclaurin Series.</figcaption></figure><p>Now, if we use f(x) = sin(x) we get sine series expansion, we use f(x) = cos(x) we get cosine series expansion. So we can use any function only condition is function should be an <a href="https://en.wikipedia.org/wiki/Taylor_series">entire function</a> (logarithm, square root, etc are not an entire function). So we can get the Taylor series for polynomial functions, exponential functions, and trigonometric functions.</p><p>Let’s get a closer look via visualization for the sine function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*DmFh0qh8pufdVCiYM7MAFg.png" /><figcaption>Fig. 4: The first 3 polynomial approximations of Taylor series (degree=5) for the sine function refer to fig. 1.</figcaption></figure><p>In fig 4, we observe the divergence of approximation from the sine function, for the first 3 terms of the equation represented in fig 1. till x=3 the function approximates very well but later starts diverging, and doesn&#39;t resemble a close approximation of sine function wave. Different functions or different polynomial degrees of a function leads to divergence to either towards ∞ or -<strong>∞</strong> for Taylor Series refer below figure<strong>.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*wXaJrtF6TzmErLKj" /><figcaption>Fig. 5: Various degrees of polynomial expansion of sine series.</figcaption></figure><blockquote>So Taylor series with finite polynomial terms cannot extrapolate values of a function for a long range.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/785/1*-d-8TCNSinMiIq4qM5Mp3w.png" /><figcaption>Fig. 6: This table shows the comparative values of the sine function and Taylor series with a certain degree of the polynomial. The red region shows the divergence in value from the true function value.</figcaption></figure><p>Padé Approximation comes to relief for this long-range extreme divergence problem in Taylor Series.</p><p>Padé approximants are derived by expanding a function as a ratio of two power series [1].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*iOqP7C3SyImNP1rHNGQoDA.png" /><figcaption>Fig. 7: Padé Approximation [2].</figcaption></figure><p>[N/M] Padé approximates “M+N” terms of the Taylor series.</p><p>Let see the Padé approximation for sine function will use [2/2] ([N/M])Padé approximation. We can calculate the Padé approximation for sine but to cut down the mathematical calculation on deriving coefficient of [2/2] Padé approximation for sine I will be directly using a <a href="https://www.wolframalpha.com/input/?i=%5B2%2F2%5D+pade+of+sin%28x%29">tool</a> to get the Padé approximant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/0*9sFtqBizYdwEDzZL.png" /><figcaption>Fig. 8: [2/2] Padé Approximation of Sine Function.</figcaption></figure><p>Let&#39;s visualize it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*tYl0B1n5ccQ4oPP3M1b7-Q.png" /><figcaption>Fig. 9: Comparison between the approximation of the Taylor series and the Padé approximation.</figcaption></figure><p>We can observe that Padé approximation although not very precise but still able to capture long-range of a function and closely converges to the original function rather than larger divergence. Therefore, the <strong>Padé approximation is one of the best-known approximations</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*y8EWxrSYAgsmMU_s8QdEgw.png" /><figcaption>Fig. 10: This table shows the comparative values of the sine function approximation using the Taylor series and Padé approximation.</figcaption></figure><p>I would recommend watching the following video which elucidates on Padé approximation in a more intuitive fashion, it also includes a mathematical calculation for calculating Padé coefficients.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FszMaPkJEMrw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DszMaPkJEMrw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FszMaPkJEMrw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/8cfed4b82d95a497c876cc37cc2b47e8/href">https://medium.com/media/8cfed4b82d95a497c876cc37cc2b47e8/href</a></iframe><p>I hope you enjoyed reading this article.</p><p>Thank you.</p><p><strong>References:</strong></p><p>[1] <a href="https://mathworld.wolfram.com/PadeApproximant.html">https://mathworld.wolfram.com/PadeApproximant.html</a></p><p>[2] <a href="https://www.colorado.edu/amath/sites/default/files/attached-files/pade_2.pdf">https://www.colorado.edu/amath/sites/default/files/attached-files/pade_2.pdf</a></p><p>[3] <a href="https://en.wikipedia.org/wiki/Pad%C3%A9_approximant">https://en.wikipedia.org/wiki/Pad%C3%A9_approximant</a></p><p>[4] <a href="https://www.youtube.com/watch?v=szMaPkJEMrw">https://www.youtube.com/watch?v=szMaPkJEMrw</a></p><p>[5] https://www.youtube.com/watch?v=3d6DsjIBzJ4&amp;t=160s</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2df76b7bb67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Marginalization & Reduction in Probability theory]]></title>
            <link>https://medium.com/analytics-vidhya/marginalization-vs-reduction-c421dabddffe?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/c421dabddffe</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[graphical-model]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sun, 05 Jul 2020 19:07:12 GMT</pubDate>
            <atom:updated>2024-02-21T10:16:54.502Z</atom:updated>
            <content:encoded><![CDATA[<h4>In this article, I would be discussing the difference between marginalization and reduction in probability theory</h4><p>If you ever have used <a href="https://en.wikipedia.org/wiki/Graphical_model">Probabilistic Graphical Models (PGM)</a> you might have come across marginalization and reduction of factors in probability space or in general many of these concepts are been introduced in probability theory. We often get confused between marginalization and reduction or rather most of us feel it is one and the same thing but it is definitely not the same. So how is it different? What is marginalization? What is Reduction? So all of this question’s answer will be addressed in this article</p><p>Let’s go through some definitions of various terminologies before diving into the difference</p><h3>Factors</h3><p>A factor is analogous to a function in programming i.e it has arguments and it returns a value. Now arguments in factor are random variables and return outcome value resulted from the argument variables. <a href="https://www.analyticsvidhya.com/blog/2017/03/conditional-probability-bayes-theorem/?utm_source=Backlink&amp;utm_medium=SEO"><strong>Conditional Probability Distribution</strong></a> (CPD) is one type of factor.</p><blockquote>Factor is the most fundamental building block for defining distribution in high-dimensional space.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*Rw2is8J4WcjupdLmuOp9RQ.png" /><figcaption>This is how we express factor mathematically</figcaption></figure><p>Now on this factor, we can do certain operations like product, marginalization, and reduction. We will not go much in-depth for the product of factors. Below is the <a href="https://www.analyticsvidhya.com/blog/2019/10/mathematics-behind-machine-learning/?utm_source=Backlink&amp;utm_medium=SEO"><strong>mathematical expression</strong></a> of product operation in factors we can have ’N’ numbers of random variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*XW6kRf1gOzQL_gXn8j8aSg.png" /><figcaption>Product of two factors resulting in the joint distribution</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/305/1*2EcXDlbFZS55rPtxnWktyw.png" /><figcaption>Example of a factor</figcaption></figure><blockquote>Note:- Sometimes factor values may be unnormalized (i.e not in range [0,1]) but the overall concept and operations remains the same</blockquote><h3>Marginalization</h3><p>Marginalization is a process of summing a variable X which has a joint distribution with other variables like Y, Z, and so on.</p><p>Considering 3 random variables, we can mathematically express it has</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*DSqoNhFSmN7EIT7HUuCJ4A.png" /><figcaption>L.H.S is the marginal distribution of Z &amp; R.H.S is the summation over joint distribution marginalizing Z</figcaption></figure><p>You can refer <a href="https://en.wikipedia.org/wiki/Marginal_distribution">Wikipedia</a> to get more understanding about marginalization</p><h3>Reduction</h3><p>The reduction can be defined as given a certain value of a random variable what is the joint distribution of other remaining variables.</p><p>Mathematically,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/262/1*aM06F8cQYHHZ2YWVMUUeoQ.png" /><figcaption>The above equation is the joint distribution of X &amp; Y given Z = z</figcaption></figure><p>Let us understand this with the help of an example</p><p>In the below figure, we can see that only rows where C = c1 is considered and rest are discarded, this is reduction operation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*0zbCx_IRVgIOG6XOGpNVGw.png" /><figcaption>This is a reduction of a factor with scope = {A, B, C} when C = c1 where A, B, C are binary random variables</figcaption></figure><p>Now in the below figure, we can see how marginalization on C add ups value in the joint distribution of B and C</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*XX7LHH3o2lJF4lxbqaXRWA.png" /><figcaption>This is the marginalization on C of a factor with scope = {A, B, C} where A, B, C are binary random variables</figcaption></figure><p>So this is all about marginalization and reduction. I hope this article was useful to you.</p><p>Thank you.</p><h4>Reference</h4><p>[1] Probabilistic Graphical Model Principles and Techniques by Daphne Koller and Nir Friedman</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c421dabddffe" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/marginalization-vs-reduction-c421dabddffe">Understanding Marginalization &amp; Reduction in Probability theory</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Information Access Modes]]></title>
            <link>https://medium.com/swlh/text-information-access-modes-push-vs-pull-querying-vs-browsing-5b4f2d3abeef?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b4f2d3abeef</guid>
            <category><![CDATA[text-mining]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[information]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sun, 28 Jun 2020 08:07:28 GMT</pubDate>
            <atom:updated>2020-06-29T11:11:16.112Z</atom:updated>
            <content:encoded><![CDATA[<h4>In this article, I will be discussing various text information access modes (Push vs Pull &amp; Querying vs Browsing)</h4><p>In one of my articles, I had discussed the text data based retrieval systems and database retrieval systems (<a href="https://medium.com/@abhishek.c.salian/text-retrieval-vs-database-retrieval-3d13965ea067">link</a>). Now its time to discuss what are various modes to access text information. We all have come across certain software applications or used certain software applications in which we either input certain text to get something or the system prompts (recommends) you certain information about something (like some products). Let us take one simple example of the Amazon website. When we want to buy certain items or items from amazon we simply input the query and we get some items related to our query. Have you ever noticed that after you buy something amazon often recommends you buy other kinds of stuff related to your bought item? Take another example of Netflix, this website also often recommends you movies or series depending on your taste of watch.</p><p>Now let us come to catch from the above examples, some systems pull information from us some push information to us.</p><blockquote>So there are <strong>two modes</strong> to access information one is <strong>Pull </strong>and other one is<strong> Push</strong></blockquote><p>Now let me talk more about Push <em>vs</em> Pull</p><blockquote>Push and Pull of information is w.r.t a system</blockquote><h3>Pull Mode</h3><p>One of the simple example to see this mode of text data access is Search Engine. A user will always input the text or a query to the system and the system will try to find relevant media in context with a given search. So we can say that users take initiative in this mode. Also, the system wants us to input certain information for further steps i.e Ad hoc information is needed for this mode of text access. The below figure is an example of this mode of accessing information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fb7iyicJgMuRIRadg22EZA.png" /><figcaption>Example of querying word “computer” in amazon</figcaption></figure><blockquote>Pull mode has two types Querying and Browsing</blockquote><p>I will discuss querying and browsing after push mode.</p><h3>Push Mode</h3><p>In this mode, the system takes initiative to give us the information or rather recommend us certain information for e.g recommender systems of Amazon, Flipkart, Netflix, etc. In this mode, the system has good knowledge about the user’s need for their interest, and accordingly, the filtering is done to push the information to you. Stable information is a need in this mode, unlike pull mode where ad hoc information is required. The below figure is an example of this mode of accessing information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*ZTDG8jd8QWR7J7umSWYSIA.png" /><figcaption>Example of Amazon’s recommendation system (<a href="https://images.martechadvisor.com/images/uploads/ckimages/images/amazon_577268b881912.jpg">Source</a>)</figcaption></figure><p>Let’s discuss more about querying and browsing.</p><h3>Querying vs browsing</h3><p>Let me explain the difference between both of these using a simple example. Consider you reside in some place ‘A’ and you want to go to a place ‘C’. Now if you know the path from ‘A’ to ‘C’ you will follow that definite path. But what if you don’t know that place ‘C’ but you know about a place ‘B’ which is nearby the place ‘C’ so you will go till place ‘B’ and then ask any person to guide you to the path to ‘C’ and then you will proceed. I know we can simply type the place name in google maps and will get the route 😂 but I am neglecting that fact here. So did you observe something in two approaches, so let me tell you that when you are definite about the path you can simply get to that place, this is analogous to<strong> Querying</strong> and when don’t have a definite path you take intermediary steps which is analogous to <strong>Browsing</strong>.</p><p>Let’s go for some formal definition.</p><h4>Querying</h4><p>In querying users enter a query or keyword and system return relevant documents. Querying is only possible when users are well aware of what query to use to get information. We can also understand this by simple SQL query which returns relevant information upon a query statement.</p><h4>Browsing</h4><p>In browsing, the user navigates through certain relevant documents or information by certain paths. We do browsing when we want to explore certain topics or sometimes we do browsing when we don’t know the exact keyword or query to use. To be able to get exact documents or media we need to do exact queries in a vast space of information (Internet) we need a topical map to tour the information space which is not well defined or well structured as of now.</p><p>I hope you are now able to distinguish between query and browsing and push and push modes.</p><p>Thank you!</p><h4>References</h4><p>[1] C. Zhai and S. Massung, <em>Text Data Management and Analysis: A Practical Introduction to Information Retrieval and Text Mining</em>, ACM Book Series, Morgan &amp; Claypool Publishers, 2016.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b4f2d3abeef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/text-information-access-modes-push-vs-pull-querying-vs-browsing-5b4f2d3abeef">Text Information Access Modes</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Retrieval vs Database Retrieval]]></title>
            <link>https://medium.com/swlh/text-retrieval-vs-database-retrieval-3d13965ea067?source=rss-1244b22c8a52------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d13965ea067</guid>
            <category><![CDATA[text-mining]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[information-retrieval]]></category>
            <category><![CDATA[data-mining]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Abhishek Salian]]></dc:creator>
            <pubDate>Sat, 27 Jun 2020 18:09:29 GMT</pubDate>
            <atom:updated>2020-06-28T20:40:13.036Z</atom:updated>
            <content:encoded><![CDATA[<h4>In this article, I will be discussing the difference between text retrieval and database retrieval.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JDTFe1U6NYd4KTzV.jpeg" /><figcaption><a href="https://miro.medium.com/max/3840/1*RWAmxUsenPRcXDtFVeRaYg.jpeg">Source</a></figcaption></figure><p>In this digital world, text data is everywhere, right from tweets in twitter to parsing some text in documents, everything is associated with the text. Many machine learning based products make effective use of this text data to make amazing technologies based on topics like sentiment analysis, topic modeling, relation extraction, etc.</p><p>As text data is everywhere, therefore, it is important for us to focus and create algorithms that can help us retrieve data in minimal time with optimal relevancy. For e.g, suppose if we type in google browser “Machine Learning algorithms pdf” we would get some relevant documents as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*jED_ycA6xs7NTRleE0ohlw.png" /><figcaption>Resultant relevant documents upon query</figcaption></figure><p>Upon querying we will get the relevant documents in optimal time. So what we can infer from this process of the query is, we want the system to give us results of a query in minimal time, the relevancy of documents also becomes an important factor here, you will not get psychology documents when typing machine learning algorithms because the subject is not that much relevant to the given query. So building one such system is difficult because of several factors and tradeoff which we encounter when we build such systems for a large corpus of documents or textual data retrieval.</p><p>Now knowing the complexity of these systems let us discuss what are the differences between text retrieval and database retrieval. But before that let us go through some formal definition of these retrieval modes.</p><blockquote>Text retrieval is a task where the system would respond to a user’s query with relevant documents. It is a preprocessor for text mining.</blockquote><blockquote>Database retrieval means obtaining data from a database management system such as ODBMS (<a href="https://en.wikipedia.org/wiki/Data_retrieval">wikipedia</a>).</blockquote><p>Now, let us compare both of these over several factors:</p><h3>Structure of Data:</h3><ul><li>Pieces of information are in <strong>unstructured or free text format</strong> in case of text retrieval based systems. According to some <a href="https://blog.bitext.com/the-bulk-of-data-wandering-on-the-net-is-unstructured-data#:~:text=Did%20you%20know%20that%20up,not%20easily%20understandable%20for%20computers.">sources</a>, 80% of the internet text data are unstructured.</li><li>In database systems as most of you would know data is <strong>well structured</strong>. For e.g see how every record is well-structured in below figure</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*bGovuvk0Pc_o4UMvxgwLjA.png" /><figcaption><a href="https://upload.wikimedia.org/wikipedia/commons/f/f2/DVD_Rental_Query.png">Source</a></figcaption></figure><h3>Ambiguity of Data</h3><ul><li>In the case of text retrieval-based systems, we often mine data which are <strong>ambiguous</strong> in nature, for e.g when we mine twitter data we often come across many texts which are ambiguous words and sometimes considering only one modality of information i.e text it is more difficult to understand in which context the sentence has been used.</li><li>Since databases are in a well-structured format the text data stored in this database have well-defined semantics or the level of <strong>ambiguity is very low</strong>. For e.g if we want a list of students in a college database who have opted for a machine learning course we can easily do it because we have well defined semantically segregated columns.</li></ul><pre># SQl query <br>SELECT student_name FROM college<br>WHERE course_name is &quot;machine learning&quot;</pre><h3>Specification of Query</h3><ul><li>When we talk about retrieving text from vast space of information we all have our own queries i.e there is no definite way to get particular information rather we browse the information space and try to get some relevant documents. For e.g see below figure</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nh1MzFvmQMIc22QfFoD2lQ.png" /><figcaption>Semantically same query statements but different relevant documents results</figcaption></figure><p>So the specification for getting certain text data is not well defined or we can say <strong>incomplete specification</strong>.</p><ul><li>For retrieving text from the database we have well-defined statements or queries because of well-structured data. So it has the <strong>complete specification</strong> for the query. For e.g SQL query</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*kZe5ugQe0KM8s4Fj47FgPQ.png" /><figcaption><a href="https://www.w3schools.com/sql/trysql.asp?filename=trysql_op_in">Source</a></figcaption></figure><h3>Results of the Query</h3><ul><li>In the case of text retrieval systems, we get relevant documents as a result of a particular query.</li><li>In the case of database retrieval systems, we have records or data records stored in the database, so when we query we get matched records in database retrieval systems.</li></ul><p>So, these all are some important differences we should know when it comes to understanding the concepts of retrieval systems.</p><p>I hope you find this article useful. Thank you.</p><h4>References</h4><p>[1] C. Zhai and S. Massung, <em>Text Data Management and Analysis: A Practical Introduction to Information Retrieval and Text Mining</em>, ACM Book Series, Morgan &amp; Claypool Publishers, 2016</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d13965ea067" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/text-retrieval-vs-database-retrieval-3d13965ea067">Text Retrieval vs Database Retrieval</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>