<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chirag Samal on Medium]]></title>
        <description><![CDATA[Stories by Chirag Samal on Medium]]></description>
        <link>https://medium.com/@chirag6891?source=rss-17967e561c06------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*0nGiHdoQRatU9WCwDiFsmQ.jpeg</url>
            <title>Stories by Chirag Samal on Medium</title>
            <link>https://medium.com/@chirag6891?source=rss-17967e561c06------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 13:53:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chirag6891/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[G2Net: Unlocking the Secrets of the Universe]]></title>
            <link>https://medium.com/analytics-vidhya/g2net-unlocking-the-secrets-of-the-universe-6a17baa6c756?source=rss-17967e561c06------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a17baa6c756</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[medium]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Chirag Samal]]></dc:creator>
            <pubDate>Sat, 11 Mar 2023 05:40:14 GMT</pubDate>
            <atom:updated>2023-03-11T05:40:14.566Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*APCkYNfktjQ21Aa6ivOu2A.jpeg" /><figcaption>Image credit: NASA</figcaption></figure><h3>Introduction</h3><p>Do you want to be a part of a groundbreaking discovery in astrophysics and astronomy? Are you looking for a chance to contribute to cutting-edge research and hone your data science skills? If you answered yes to both questions, then the G2Net Detecting Continuous Gravitational Waves Kaggle competition is for you!<br>In this competition, our goal is to develop a model that can accurately detect and classify long-lasting gravitational-wave signals by rapidly spinningneutron stars within noisy data. By doing so, we can improve the sensitivity of scientists to these signals, potentially leading to new discoveries in the field. This will give us an opportunity to help us achieve this goal and make a significant contribution to the field of astrophysics and astronomy.</p><p>In this article, we will explore the problem statement, dataset, and our approach to developing a simple baseline model for this competition. Here, I will provide step-by-step guidance and insights into a basic thought process, allowing you to follow along and potentially develop your own model.<br>So, are you ready to contribute to groundbreaking discoveries in astrophysics and astronomy?</p><h3>Context</h3><p>Continuous gravitational waves are ripples in the fabric of space-time that are caused by the motion of massive celestial objects such as black holes and neutron stars. These waves have been the subject of extensive research in the field of astrophysics, and detecting them is crucial to better understanding the universe. The detection of gravitational waves marked a significant breakthrough in astrophysics and astronomy. Since then, scientists have detected gravitational waves from merging black holes and neutron stars, but a second class of gravitational waves, continuous gravitational waves, remains undetected. Continuous gravitational waves are weak yet long-lasting signals emitted by rapidly spinning neutron stars. Detecting them could enable scientists to learn more about the structure of the most extreme stars in our universe.</p><p>The G2Net Detecting Continuous Gravitational Waves Kaggle competition aims to develop a model that is sensitive enough to detect these weak signals within noisy data. In this competition, we aim to create a model that can distinguish between noisy data and the weak but persistent signals emitted by swiftly rotating neutron stars. The task is to detect the presence of a signal in the data given a training set containing time-frequency data from two gravitational-wave interferometers (LIGO Hanford &amp; LIGO Livingston), each data sample with real or simulated noise, and possibly a continuous simulated gravitational-wave signal (CW).</p><p>To overcome the challenge of detecting continuous gravitational waves, the G2Net Kaggle competition provides participants with access to a unique dataset of simulated gravitational-wave signals. This dataset has been specifically designed to mimic the noise and signal properties that are expected in real gravitational-wave observations. To improve the accuracy of their models, participants can also use data augmentation techniques such as flipping and shifting the training data to create new samples. The competition aims to create a model that can identify continuous gravitational waves in noisy data and contribute to the ongoing quest to understand the universe.</p><h3>EDA</h3><p>Exploratory Data Analysis (EDA) is a crucial step in understanding the G2Net dataset. The first step is to plot the distribution of labels to understand the proportion of data with and without a simulated continuous gravitational wave signal.</p><p>The file “target labels.csv” contains the target labels, which are either 0 if there are no gravitational waves present in the data or 1 if there are. (Note the presence of a few files with the suffix -1. The status of these files is currently unknown to physicists.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MiuNWUkS5HQPlnOHn5F43w.png" /><figcaption>Frequency of Labels in the Training Data. Image by the Author</figcaption></figure><p>H1 EDA involves exploring the time-frequency data from LIGO Hanford, whereas L1 EDA involves exploring data from LIGO Livingston.</p><p>Spectrograms and Fourier Transform can be used to gain insight into the data. A spectrogram is a visual representation of the frequency content of a signal over time. It is obtained by dividing the signal into overlapping frames, computing the Fourier transform of each frame, and plotting the magnitude of the resulting spectrum as a function of time. This allows us to see how the frequency content of the signal changes over time.</p><p>The Fourier transform is a mathematical tool that decomposes a signal into its frequency components. It transforms a signal from the time domain to the frequency domain, which allows us to analyze the signal in terms of its constituent frequencies. The Fourier transform of a signal x(t) is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*y_AznJcEdC44PssQ-zlzyA.png" /></figure><p>where F(k) is the frequency-domain representation of f(x) at frequency k, and e^(-2πikx) is the complex exponential at frequency k and time x.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Qanr_zrxxy4uftO8fYRLA.png" /><figcaption>H1 Spectograms. Image by the Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m5kcAH4TveHFy5OVEUNUIA.png" /><figcaption>H1 Fourier Transform. Image by the Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rTQAc1dV3rL7XjePXnDwZw.png" /><figcaption>L1 Spectrogram. Image by the Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t7iN6O273sLfeZkDc8Aaxw.png" /><figcaption>L1 Fourier Transform. Image by the Author</figcaption></figure><p>Additionally, TimeStamp and Frequency Analysis can provide more insights into the variations present in the data. TimeStamp Analysis can be used to study how the signal changes over time, while Frequency Analysis can be used to study how the frequency content of the signal changes over time. These analyses can help identify specific patterns or features in the data that may be important for detecting gravitational waves. Understanding these patterns can help in developing effective signal-processing techniques and machine-learning models for the detection of continuous gravitational waves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g4rI4kEblkq-dFWMW2KThQ.png" /><figcaption>TimeStamp Analysis. Image by the Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*aAe9CNtNfuQGnJ_rVgrzDw.png" /><figcaption>Frequency Analysis. Image by the Author</figcaption></figure><h3>Dataset Description</h3><p>The dataset for this competition includes time-frequency data from two gravitational-wave interferometers (LIGO Hanford &amp; LIGO Livingston). Each data sample contains real or simulated noise and a simulated continuous gravitational wave signal. Each sample has Short-time Fourier Transforms and GPS time stamps for each interferometer, which are not always contiguous in time. The simulated signals have eight randomized parameters and are characterized by the location, orientation, frequency, and spin-down of the hypothetical astrophysical source. The typical amplitudes of the resulting signals are lower than the detector noise. The data also includes timestamps and frequency data for each sample.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8TbpVd60XfXsI-CpavmLdw.png" /><figcaption>Dataset Structure. Image by Edward Crookenden</figcaption></figure><h3>Data Augmentation</h3><p>Data augmentation is a powerful tool for increasing the size of a small dataset, as it allows us to generate new variations of the same data by rotating, flipping, and shifting the input images. By using data augmentation, we can train our model more effectively by providing it with more training examples, which leads to better generalization and improved accuracy. However, it’s essential to ensure that any variation of the image does not violate the physical principles behind the input data.</p><p>Here, we will discuss two common data augmentation techniques that we can apply to our SFT images.</p><p><strong>1. Horizontal Flipping:</strong></p><p>One simple technique is to flip the SFT images horizontally. The intuition behind this technique is that flipping the image horizontally doesn’t change the underlying physics of the signal, but it does add more diversity to our training data.</p><p><strong>2. Vertical Shifting:</strong></p><p>Another useful technique is to shift the SFT images vertically by a certain number of pixels. The intuition behind this technique is that the location of the signal in the image shouldn’t affect the prediction of our model.</p><p>In the context of SFTs, we can use horizontal and vertical flipping to create new images that have the same features as the original but are flipped. This method can help the model learn the features from different angles and improve its performance. Additionally, we can shift the images vertically by a specific number of units to generate a new set of images that have the same features but are slightly different. This technique can help the model learn to detect gravitational waves, even when they are shifted, thus improving the model’s accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*v-9YKoSl-87yd08UIMPO8Q.png" /><figcaption>Visualizing Augmented Image. Image by the Author</figcaption></figure><h3>Baseline Model</h3><p>In this section, I want to share some insights on the baseline model used for the G2Net challenge. The model is built using TensorFlow and uses a convolutional neural network (CNN) to detect gravitational wave signals in the SFT images. The training data is loaded from a directory containing SFT images in HDF5 format.</p><p>Before feeding the images to the model, we preprocess them by applying various transformations such as reshaping, normalization, and augmentation. The CNN model uses <strong>EfficientNet B3</strong> as the base model with ‘imagenet’ weights for pretraining. EfficientNet B3 has been shown to perform well on image classification tasks and provides a good starting point for the G2Net challenge.</p><p>The input image size is (360, 360) with two channels, and the output shape of the image is (360, 360, 2). We add a dropout layer with a rate of 0.2 to the last layer of the model. The shuffle size for the dataset is set to 128, and the batch size for training is set to 16, while the test batch size is set to 64. The model is trained for 200 epochs with a learning rate of 0.001 using binary cross-entropy loss and the Adam optimizer. The training is stopped early if the validation loss does not improve for ten epochs, which is known as early stopping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3aLAeIkFGbRYfQBG8D675w.png" /><figcaption>A pictorial representation of the relationship between loss and accuracy over epochs.</figcaption></figure><p>The final results show that the model achieves an accuracy of 0.8059 on the training data and 0.7909 on the validation data at epoch 41 out of 200. The loss for training data is 0.4092, while for the validation data, it is 0.4284. We restore the model weights from the end of the best epoch, and the training is stopped at epoch 41 because the validation loss does not improve for ten epochs.</p><h3>Summary</h3><p>In conclusion, the G2Net challenge is an exciting opportunity to explore the depths of our universe and detect long-lasting gravitational wave signals. We have explored the data preprocessing and augmentation techniques used in a baseline model for the challenge, which uses TensorFlow and EfficientNet B3 as the base model. Overall, the baseline model provides a solid starting point for the G2Net challenge. However, there is still room for improvement by further tuning the hyperparameters or using more advanced techniques such as transfer learning or ensemble methods.</p><p>As we continue to push the boundaries of our knowledge of the universe, the G2Net challenge represents an important step toward detecting and studying continuous gravitational waves. With the power of AI and machine learning, we can uncover the secrets of the universe and further our understanding of the world around us.</p><p>You can access the complete code here in this <strong>GitHub Repository: </strong><a href="https://github.com/chiragsamal/Detecting-Gravitational-Waves"><strong>Detecting Gravitational Waves</strong></a></p><h3>References</h3><p>As I wrap up this post, I would like to thank all the wonderful people who shared their expertise and knowledge on the G2Net challenge. Without their contributions, this article would not have been possible.</p><p>If you’re interested in diving deeper into the G2Net challenge, I highly recommend checking out the following resources:</p><ul><li>The insightful EDA provided by Ayush Thakur and Jose Caliz in [G2Net] <a href="https://www.kaggle.com/code/ayuraj/g2net-understand-the-data">Understand the Data</a> and <a href="https://www.kaggle.com/code/jcaliz/g2net-eda-that-gives-you-insights">G2Net: EDA That Gives You insights</a>.</li><li>The helpful starter code and EDA provided by Edward Crookenden in <a href="https://www.kaggle.com/code/edwardcrookenden/g2net-getting-started-eda">G2Net Getting Started + EDA</a>.</li><li>The data preprocessing and augmentation techniques shared by MaharshiPandya in <a href="https://www.kaggle.com/code/maharshipandya/g2net-data-and-augmentation">G2Net Data and Augmentation.</a></li></ul><p>And, of course, a big shoutout to the creator of the baseline model used in this post, George Chirita, whose code can be found <a href="https://www.kaggle.com/code/crischir/g2net-tf-baseline-360">here</a>.</p><p>I hope this post has been informative and helpful for those interested in the G2Net challenge. As always, I welcome any suggestions and feedback and look forward to continuing the discussion with you all. Happy learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a17baa6c756" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/g2net-unlocking-the-secrets-of-the-universe-6a17baa6c756">G2Net: Unlocking the Secrets of the Universe</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to land a research internship: A Guide for Students]]></title>
            <link>https://medium.com/analytics-vidhya/how-to-land-a-research-internship-a-guide-for-students-58c8a6d02c71?source=rss-17967e561c06------2</link>
            <guid isPermaLink="false">https://medium.com/p/58c8a6d02c71</guid>
            <category><![CDATA[research]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[internships]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Chirag Samal]]></dc:creator>
            <pubDate>Fri, 03 Mar 2023 09:27:45 GMT</pubDate>
            <atom:updated>2023-03-03T09:27:45.115Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*Ea_sQy1GXT-vV77KZkLXdA.png" /></figure><h3>Introduction</h3><blockquote>Research internships can be an excellent way for undergraduate and graduate students to gain valuable research experience and explore their interests in a specific field. However, the application process can be confusing and challenging. In this article, I will share my experience in applying for research internships and provide some tips and advice for students interested in pursuing research opportunities.</blockquote><p>My Experience: My name is Chirag Samal, and I have completed my under graduation from IIIT Naya Raipur. I have experience interning at Stanford University and the Indian Institute of Science (IISc), Bangalore. Throughout my undergraduate studies, I have applied to various foreign and Indian universities for research internships and have been accepted to several, including Carnegie Mellon University, the University of Tokyo, IIT Delhi, IIT Varanasi, and the University of Toronto, among others. Through this experience, I learned about two methods of applying for research internships.</p><h4>Methods of Applying:</h4><p>There are generally two methods of applying for research internships:</p><ol><li><strong>Applying through various programs:</strong> Some universities and organizations offer programs specifically for research internships, such as MITACS Globalink for research positions in Canada and India Connect@NTU for research internships in NTU Singapore. You can find a comprehensive list of these programs on this website GitHub repository maintained by <a href="https://github.com/himahuja/Research-Internships-for-Undergraduates">Himanshu Ahuja</a>.</li><li><strong>Applying via cold-emailing professors:</strong> This method is time-consuming and tedious but can be an effective way of finding research internships. Start by figuring out the area of research you’re interested in and looking for professors who work in fields similar to yours. Visit their websites and look for projects that you find interesting. Finally, craft a thoughtful email introducing yourself, explaining your interest in their research, and asking if they have any available research positions.</li></ol><h3>Tips for Cold-Emailing Professors:</h3><p>When emailing professors, keep the following tips in mind:</p><ol><li>Start your email with a clear and concise subject line.</li><li>Introduce yourself in the first paragraph and explain your interest in their research.</li><li>Show that you have done your research by referencing specific projects or papers they have worked on.</li><li>Be polite and professional in your tone and language.</li><li>Keep your email short and to the point but provide enough information to demonstrate your qualifications and interest.</li><li>Follow up politely if you don’t hear back after a week or two.</li></ol><p>To help you write a cover letter, <strong>I have shared a template from my cover letter</strong>, which you can modify according to your needs:</p><p>Dear [Researcher’s Name],</p><p>I am [Your Name], a [Your Academic Standing] from [Your University/Institution]. I am writing to express my interest in the possibility of a research project/internship under your guidance for [Length of Internship] in [Researcher’s Lab/Group]. I came across your work on [Research Topic], and I am impressed by the contributions you and your team have made to the field.</p><p>I have always been interested in [Your Research Interests], and after reading some of your publications and research projects, I am confident that your lab is the perfect place to pursue my research interests.</p><p>I have had the opportunity to work as a [Your Previous Position] at [Previous Institution/Lab], where I have developed strong skills in [Your Skills]. I have also had the opportunity to be a co-author on a research paper under review at [Conference/Journal] on [Paper Topic].</p><p>Some of my significant academic achievements include [Academic Achievements].</p><p>I have attached my resume to this email, which includes a link to my LinkedIn and GitHub profiles. Additionally, I have a letter of recommendation available upon request. I am open to both on-campus and remote opportunities.</p><p>Thank you for considering my application. I look forward to the opportunity to discuss my qualifications further.</p><p>Sincerely,</p><p>[Your Name]</p><p>[University/College]</p><p>[Phone Number]</p><p>[Email Address]</p><h3>Conclusion</h3><p>In conclusion, applying for research internships can be challenging, but it is also a valuable and rewarding experience. Whether you apply through programs or cold-email professors, it is important to be persistent and professional in your approach. Remember to showcase your qualifications and interests, and follow up politely if you don’t hear back. With dedication and hard work, you can secure a research internship and gain valuable experience in your field. Best of luck on your journey!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58c8a6d02c71" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/how-to-land-a-research-internship-a-guide-for-students-58c8a6d02c71">How to land a research internship: A Guide for Students</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Netflix Appetency: A beginner’s guide]]></title>
            <link>https://medium.com/analytics-vidhya/netflix-appetency-a-beginners-guide-9c2b9c0835f6?source=rss-17967e561c06------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c2b9c0835f6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[netflix]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Chirag Samal]]></dc:creator>
            <pubDate>Thu, 09 Jun 2022 03:22:34 GMT</pubDate>
            <atom:updated>2022-06-09T03:42:47.237Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X3fxJQySIif__8SvuHMX3w.png" /><figcaption>Image source: Netflix</figcaption></figure><p>Data Science plays an essential role in most online services and helps bring more customers and keep the existing ones happy. The reason why this is because, with Data Science, you get a more realistic picture of your consumers’ taste in the form of graphs and charts that take not just one metric but several as input. This crucial piece of information helps you mold your products and services to look one-of-a-kind to your customers, attracting them to your platform.</p><p>With a company like Netflix brimming with data, it’s always a wise decision to put that pile of data to good use. By incorporating concepts like data analysis, machine learning, statistics, and deep learning, Data Science can help Netflix and any business grow exponentially regardless of sector.</p><p>Data Science plays a critical role in deciding the functioning of Netflix and presents them with newer opportunities to grow.</p><p>In this article, we will classify consumers according to their appetite to subscribe to Netflix using <a href="https://www.kaggle.com/competitions/netflix-appetency/">Netflix Appetency</a> dataset from Kaggle. The training set consists of an id column, the customers’ features, and a target column: target.</p><h3>Importing Libraries</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/301c567e5176b5b7eb7aeade03b09a54/href">https://medium.com/media/301c567e5176b5b7eb7aeade03b09a54/href</a></iframe><p><strong>Load Dataset</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b098329f6cec61c23e2674a88312bfbf/href">https://medium.com/media/b098329f6cec61c23e2674a88312bfbf/href</a></iframe><ul><li>train.csv - the training set. it consists of an id column, the customers&#39; features, and a target column: target.</li><li>test.csv - the test set. it consists of everything in train.csv except the target.</li><li>sample_submission.csv - a sample submission file in the correct format target=1 means that the customer subscribes to Netflix</li></ul><p><strong>Target type and count</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e62d5a3c31b6aec2b3bb6d87f42c910f/href">https://medium.com/media/e62d5a3c31b6aec2b3bb6d87f42c910f/href</a></iframe><figure><img alt="Target distribution of the dataset" src="https://cdn-images-1.medium.com/max/720/1*rpgwzyB-CWNJwCeJgybtCw.png" /><figcaption>Target distribution. Image by the author</figcaption></figure><h3>Data Cleaning</h3><p>Data cleaning is the most important task that should be done as a data science professional. Having wrong or bad quality data can be detrimental to processes and analysis. Having clean data will ultimately enhance overall productivity and allow you to make the best decisions possible.</p><p>We will drop features:</p><ul><li>Features with more than 10% null values</li><li>Features with single categorical features</li><li>Features with single values features</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/efb512c3303dd9d4d0b00f66ab4aa211/href">https://medium.com/media/efb512c3303dd9d4d0b00f66ab4aa211/href</a></iframe><p><strong>Top features distribution</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6baea2b46216a7b4810a81f07fc8c9b/href">https://medium.com/media/c6baea2b46216a7b4810a81f07fc8c9b/href</a></iframe><figure><img alt="Feature distribution" src="https://cdn-images-1.medium.com/max/1024/1*uvL5CGvV4R36lSZGKSXG0A.png" /><figcaption>Top 9 features distribution. Image by the author.</figcaption></figure><p><strong>Correlation Heatmap</strong></p><p>The graphical representation of a correlation matrix represents the correlation between the top 9 variables.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca64e1912822a64a5fc62c2eed8c521e/href">https://medium.com/media/ca64e1912822a64a5fc62c2eed8c521e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qVd9eWbLniDgew67hxlEVg.png" /><figcaption>Correlation Heatmap. Image by the Author</figcaption></figure><h3>Preprocessing</h3><p><strong>Label Encoding</strong></p><p>We will use Label Encoding to encode categorical columns in the dataset to convert the labels to numeric columns.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a6e1368a50003c0ed3162fb200f1f15/href">https://medium.com/media/1a6e1368a50003c0ed3162fb200f1f15/href</a></iframe><h3>Machine Learning Model Training</h3><p><strong>XGBOOST</strong></p><p>We will use the XGBoost machine learning model to train our dataset.</p><p>(XGBoost) is a scalable and enhanced version of the gradient boosting algorithm (terminology alert) that is optimized for efficacy, computational speed, and model performance. It is a member of the Distributed Machine Learning Community and an open-source library. XGBoost is a unique combination of software and hardware features meant to improve conventional boosting techniques with precision and speed.</p><p>To avoid overfitting we will use cross-validation.<br>Cross-Validation is a process in which a test set is kept for final evaluation but the validation set is not required when executing CV. The basic technique, known as a k-fold CV, divides the training set into k smaller sets (other approaches are described below, but generally follow the same principles). For each of the k “folds,” the procedure is as follows:<br>A model is trained using the folds as training data, and the model is then validated using the remaining data (i.e., it is used as a test set to compute a performance measure such as accuracy).<br>The average of the values is the performance measure produced by k-fold cross-validation.</p><p>We will use <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html#sklearn.model_selection.StratifiedKFold"><strong>StratifiedKFold</strong></a> which is a variation of <em>k-fold</em> which returns <em>stratified</em> folds: each set contains approximately the same percentage of samples of each target class as the complete set. We will use a stratified 5-fold cross-validation.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f1ad36295ba59851614750b9c262f60a/href">https://medium.com/media/f1ad36295ba59851614750b9c262f60a/href</a></iframe><p>The evaluation metric used in the competition is AUC (Area under Curve). The ROC-AUC score function calculates the area beneath the receiver operating characteristic (ROC) curve, also known as ROC-AUC. The curve information is summarised in one number by computing the area under the roc curve.</p><p>The AUC value ranges from 0 to 1. The AUC of a model whose predictions are 100 percent incorrect is 0; the AUC of a model whose predictions are 100 percent correct is 1.</p><p>The mean AUC which we receive after 5-fold Stratified K-fold cross-validation in the XGBoost Machine Learning model is 0.7853.</p><p>Let’s plot the top feature importance from the machine learning model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/27bcc9aefea4a8ddec181ed5b8104ce2/href">https://medium.com/media/27bcc9aefea4a8ddec181ed5b8104ce2/href</a></iframe><figure><img alt="Feature Importance" src="https://cdn-images-1.medium.com/max/1024/1*i2ymwvwwvEfcnMs56CjGqA.png" /><figcaption>Feature Importance. Image by the author.</figcaption></figure><h3>Summary</h3><p>In this article, we discussed Netflix Appetency, a Kaggle competition to classify consumers according to their appetite to subscribe to Netflix. We discussed a beginner-level tutorial to understand the problem statement and trained an XGBoost Machine Learning model to train the dataset. The mean AUC which we got after training our model is 0.7853.</p><p>You can access the full code here in this GitHub Repository: <a href="https://github.com/chiragsamal/NeuralProphet-Tutorial">Netflix Appetency Tutorial</a>.</p><h3>References:</h3><ul><li><a href="https://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics">Sklearn Metrics</a></li><li><a href="https://www.kaggle.com/code/tarique7/netflix-appetency-1st-place-solution">Netflix Appetency — 1st Place Solution</a></li><li><a href="https://www.kaggle.com/code/mehrdadsadeghi/netflix-appetency-consumer-classification">Netflix appetency — Consumer Classification</a></li><li><a href="https://www.kaggle.com/code/gopidurgaprasad/netflix-appetency-stater-eda-model">Netflix Appetency : Stater EDA + Model</a></li><li><a href="https://www.kaggle.com/code/hamzaouammou/netflix-3rd-place-basic-solution">Netflix 3rd Place Basic Solution</a></li></ul><p>Thanks for reading; I will be writing more similar posts soon. Let’s get involved in discussions, and suggestions are always welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c2b9c0835f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/netflix-appetency-a-beginners-guide-9c2b9c0835f6">Netflix Appetency: A beginner’s guide</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Time Series Tutorial using NeuralProphet]]></title>
            <link>https://medium.com/analytics-vidhya/time-series-tutorial-using-neuralprophet-e918a1b437ed?source=rss-17967e561c06------2</link>
            <guid isPermaLink="false">https://medium.com/p/e918a1b437ed</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <dc:creator><![CDATA[Chirag Samal]]></dc:creator>
            <pubDate>Mon, 21 Feb 2022 03:36:29 GMT</pubDate>
            <atom:updated>2022-02-25T15:17:39.560Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W9JXFMEKgW9Aqxt5CkCXjg.png" /><figcaption>NeuralProphet. Image Credit: NeuralProphet official Repository</figcaption></figure><p>In this article, we will build a time series forecasting model using NeuralProphet. NeuralProphet is a neural-network-based Time-Series model, inspired by <a href="https://github.com/facebook/prophet">Facebook Prophet</a> and <a href="https://github.com/ourownstory/AR-Net">AR-Net</a>, built on PyTorch. In this article, we will use the AirPassengers Dataset that provides monthly totals of US airline passengers from 1949 to 1960.</p><h3>Tutorial of this article:</h3><ol><li><strong>Introduction to NeuralProphet</strong></li><li><strong>Advantages of NeuralProphet</strong></li><li><strong>Changes from Prophet</strong></li><li><strong>Installation of NeuralProphet</strong></li><li><strong>Importing Libraries</strong></li><li><strong>Import Input Data</strong></li><li><strong>Preprocessing of the dataset</strong></li></ol><ul><li>Renaming Columns.</li><li>Changing data type.</li></ul><p><strong>8. Visualize the dataset</strong></p><p><strong>9. Training NeuralProphet Model</strong></p><ul><li>Visualize the forecast by the model</li><li>Visualize the components</li><li>Visualize the parameters</li><li>Visualize Training and Validation Loss</li></ul><p><strong>10. Conclusion</strong></p><p><strong>11. References</strong></p><h3>Introduction to NeuralProphet</h3><p>NeuralProphet is a PyTorch implementation of a user-friendly time series forecasting tool for practitioners based on Neural Networks. This is heavily influenced by Prophet, a popular Facebook-developed forecasting tool. NeuralProphet is built on a fully modular architecture that allows it to grow in the future as new components are added. This is a user-friendly forecasting tool that maintains the original goals of Prophet, such as interpretability and configurability, while also adding new features, such as automatic differencing capabilities, by using PyTorch as the backend.</p><h3>Advantages of NeuralProphet</h3><p>NeuralProphet revolutionizes time series modeling and forecasting:</p><ul><li>Auto-regression and covariates are supported.</li><li>Hyperparameters related to training are automatically selected.</li><li>Fourier term seasonality at various time scales, including yearly, daily, weekly, and hourly.</li><li>Piecewise linear trend with automatic changepoint detection as an option.</li><li>Forecast component plotting, model coefficient plotting, and final prediction plotting.</li><li>Global modeling assistance.</li><li>Future and lag regressors</li><li>Regularization reduces coefficient sparsity.</li><li>Python package that is both user-friendly and powerful.</li></ul><h3>Changes from NeuralProphet</h3><p>In comparison to the original Prophet, NeuralProphet has a number of additional features. The following are the details.</p><ul><li>Optimization using Gradient Descent with PyTorch as the backend.</li><li>Using AR-Net to model time series autocorrelation.</li><li>Using a separate Feed-Forward Neural Network to model lagged regressors.</li><li>FFNNs with configurable non-linear deep layers.</li><li>Adaptable to different forecast horizons (greater than 1).</li><li>Losses and metrics that are unique to you.</li></ul><h3>Installation of NeuralProphet</h3><p>We can directly install neuralprophet using pip:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/74922b15b374e96ed692cf4c21c5cc1f/href">https://medium.com/media/74922b15b374e96ed692cf4c21c5cc1f/href</a></iframe><h4>Import Libraries</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c86ac5e5fb5acc5f68d49f89c8e5125b/href">https://medium.com/media/c86ac5e5fb5acc5f68d49f89c8e5125b/href</a></iframe><h4>Import Input Data</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5687b903afd520ad1c7e889a2cb21820/href">https://medium.com/media/5687b903afd520ad1c7e889a2cb21820/href</a></iframe><p>Summary of the dataset</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WzT5byfxpSVxC_iHL2xWtg.png" /><figcaption>Summary of the dataset</figcaption></figure><h3>Preprocessing of the dataset</h3><p>NeuralProphet expects only two columns by default- a column called ‘ds’, which stands for dates, and another column called ‘y’, which represents the value that we’re trying to predict.</p><h4>Renaming Columns</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0fac69dcdebe45901f8b8276177aeed/href">https://medium.com/media/e0fac69dcdebe45901f8b8276177aeed/href</a></iframe><h4>Changing data-type</h4><p>The important thing to note here is that, the ds column must be of the datetime type. But, we can see that it is of object data type. Now, because the ds column is not of the datetime type. So, we’ll need to convert it into datetime type.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e692cc0a3874eb7782f1af4f7bd8ad81/href">https://medium.com/media/e692cc0a3874eb7782f1af4f7bd8ad81/href</a></iframe><h4>Visualize the dataset</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*7ye4Ms5j1j-Ew5r4fOaR1g.png" /><figcaption>Visualize the dataset</figcaption></figure><h3>Training a NeuralProphet Model</h3><p>We’ll make a simple model with neural_prophet for this dataset, which can be fitted by creating an object of the NeuralProphet class. Here, the frequency of the dataset is ‘month’. So we will use the change the frequency parameter accordingly. All the parameters are set by default by NeuralProphet. Model validation for NeuralProphet can be done in two ways. Users can split the dataset manually to validate after the model fitting like below by specifying the fraction of data to be used for validation in the argument valid_p. In this article, we will use the first method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4952d43dbd2897672a486b505112520d/href">https://medium.com/media/4952d43dbd2897672a486b505112520d/href</a></iframe><p>We can make predictions using the fitted model once it has been fitted.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e6a2b4b79b83172c508054dac49a1c50/href">https://medium.com/media/e6a2b4b79b83172c508054dac49a1c50/href</a></iframe><h4>Visualize the forecast by the model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*fQt5G0XPKH46Y26788QjGQ.png" /><figcaption>NeuralProphet forecast plot. Image by the author.</figcaption></figure><h4>Visualize the components</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*rAc3Jd4JEXbfBZ5efFkq4g.png" /><figcaption>NeuralProphet model components plot. Image by the author.</figcaption></figure><h4>Visualize the model parameters</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*byXyDn9B2f4n_Kgw6L7L9A.png" /><figcaption>NeuralProphet model parameters forecast plot. Image by the author.</figcaption></figure><h4>Visualize Training and Validation Loss</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W_TfKTZ-nvBsRPWuMhsThQ.png" /><figcaption>Model Loss using Mean Absolute Error</figcaption></figure><h3>Summary</h3><p>In this article, we discussed NeuralProphet, a Python package that uses Neural Networks to model time series. The library’s backend is PyTorch. In this article, we developed a prediction model using time-series data on Air Passengers. NeuralProphet is built on a fully modular architecture based on AR-Net, making it scalable to incorporate new features in the future.</p><p>You can access full code here in this GitHub Repository: <a href="https://github.com/chiragsamal/NeuralProphet-Tutorial">NeuralProphet Tutorial</a></p><h3>References:</h3><ul><li><a href="https://github.com/ourownstory/neural_prophet">NeuralProphet GitHub Repository</a></li><li><a href="https://facebook.github.io/prophet/">Prophet Documentation</a></li><li><a href="https://arxiv.org/pdf/1911.12436.pdf">AR -Net Paper</a></li></ul><p>Thanks for reading; I will be writing more similar posts soon. Let’s get involved in discussions, and suggestions are always welcome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e918a1b437ed" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/time-series-tutorial-using-neuralprophet-e918a1b437ed">Time Series Tutorial using NeuralProphet</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build the first Flask Python web app framework]]></title>
            <link>https://medium.com/analytics-vidhya/https-medium-com-chirag6891-build-the-first-flask-python-e278b52473f3?source=rss-17967e561c06------2</link>
            <guid isPermaLink="false">https://medium.com/p/e278b52473f3</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Chirag Samal]]></dc:creator>
            <pubDate>Wed, 06 Nov 2019 02:23:12 GMT</pubDate>
            <atom:updated>2019-11-11T12:16:38.076Z</atom:updated>
            <content:encoded><![CDATA[<h4>When developing a web app in Python, chances are you’re leveraging a framework. There are a number of frameworks for Python, from which we will now learn <a href="http://flask.pocoo.org/">Flask</a>. This article will be your complete definitive guide for developing fully functional websites with the Flask web framework.</h4><h3><strong>Breakdown of this Article</strong>:</h3><p>1) Install flask</p><p>2) Create a Machine Learning Model</p><p>3) Saving Machine Learning Model</p><p>4) Creating an API using Flask</p><p>5) Create a front-end application</p><p>6) Hosting on the personal server</p><h3>Environment setup</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZDXzEwWqdDwu95G374AM0g.jpeg" /><figcaption>REST API using Flask</figcaption></figure><p><strong>Flask is a web framework which </strong>provides us with tools, libraries and technologies that allow us to <em>build a web application</em>. This web application can be any web pages or a blog or go as large as a commercial website.</p><p><strong>In Anaconda Distribution install Flask:</strong></p><p>pip install flask</p><h3>Explanation of Data Flow using Flask</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*t0MLLv0UhYoETR8Y_ZaSmw.png" /><figcaption>Data Flow using Flask</figcaption></figure><p><strong>Steps to be taken:</strong></p><p>1) Train the Machine Learning Model</p><p>2) Save the trained machine learning model object as a pickle file (serialization)</p><p>3) Build a flask environment that will have an API endpoint which would encapsulate our trained machine learning model and enable it to receive inputs (features) through GET requests over HTTP/HTTPS and then return the output after de-serializing the earlier serialized model</p><p>4) Upload the flask script along with the trained Machine Learning Model</p><p>5) Make requests to the hosted flask script through a website or any other application capable of sending HTTP/HTTPS requests</p><h3>Machine Learning Model</h3><h4>Train our Machine Learning Model</h4><p>Create a Python file model.py</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uCUj0Suo6kMSROdtCswE9Q.png" /><figcaption>Machine Learning Model</figcaption></figure><p>Here, after running the above code, a file called “model.pkl” is created using dump which is the trained machine learning model that can be transferred anywhere and used after de-serialization as given using load.</p><h3>Flask setup</h3><p>Let’s first set up the flask server on the local host and later deploy it on python.</p><p>The following script starts the flask server on localhost and default port (5000) making the URL: <a href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a> (http://127.0.0.1:5000/)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NY9c4B4ZSM7Yojk50jXo2A.png" /><figcaption>app.py</figcaption></figure><p>In-App Route app.route decorator is used for specifying the flask app route over the web.</p><ol><li>“/” simply means the home that is “http://127.0.0.1:5000/” (http://127.0.0.1:5000/”)</li><li>2) “/predict/” means <a href="http://127.0.0.1:5000/predict/">http://127.0.0.1:5000/predict/</a> (http://127.0.0.1:5000/predict/)</li></ol><h3><strong>Routing</strong></h3><p>After initializing the app.py file, we have to tell the Flask what we want to do after the web page loads. The python code <a href="http://twitter.com/app">@app</a>.route(“/”, methods = [“GET”,” POST”]) tells the Flask what to do when we load the home page of the website. In methods: GET method is the type of request our web browser will send to the website when it accesses the URL of the web page. Don’t worry about the POST method as it doesn’t have much relevance for us in this deployment process.</p><h3>Setting up the HTML file</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JbONG9LD6Ri-m6Rz8OhdMg.png" /><figcaption>index.html</figcaption></figure><h4>Save model.py, app.py and index.html</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*9eRbSWpqPKCC2dVLTGeoNw.png" /><figcaption>Run app.py in anaconda prompt</figcaption></figure><p>Copy the URL <a href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a> (http://127.0.0.1:5000/) and paste it into your browser</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*vq-WemLKQF5lFi-jZWbLaw.png" /></figure><p>After a few secs, this would be shown in the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BzVYPLm_l2HhKSIzBd5LBg.png" /><figcaption>Machine Learning API</figcaption></figure><h4>Input Data into different Categories</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d2nP5EN0utxXinTWJ1FqEQ.png" /></figure><h4>Then Press Predict button</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K1aB482e9Eqsd2iiJ_KuVA.png" /></figure><p>You can see that we Predicted our profit and notice that the URL of the page changed to <a href="http://127.0.0.1:5000/predict">http://127.0.0.1:5000/predict</a> (http://127.0.0.1:5000/predict).</p><p>This is my first article hope you like it and the Github link for all the files is in this link: <a href="https://github.com/chiragsamal/flask_tutorial">https://github.com/chiragsamal/flask_tutorial</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e278b52473f3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/https-medium-com-chirag6891-build-the-first-flask-python-e278b52473f3">Build the first Flask Python web app framework</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>