<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manan Agarwal on Medium]]></title>
        <description><![CDATA[Stories by Manan Agarwal on Medium]]></description>
        <link>https://medium.com/@manan_agarwal?source=rss-2c6915d07485------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*nBMAFF9TOnIwRwkuqJSUuQ.jpeg</url>
            <title>Stories by Manan Agarwal on Medium</title>
            <link>https://medium.com/@manan_agarwal?source=rss-2c6915d07485------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 19:03:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@manan_agarwal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Heart Anomaly Detection by Analysing Stethoscope sounds using Deep Learning]]></title>
            <link>https://medium.com/@manan_agarwal/heart-anomaly-detection-by-analysing-stethoscope-sounds-using-deep-learning-62871fe2fbe5?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/62871fe2fbe5</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[heart-disease]]></category>
            <category><![CDATA[audio]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Mon, 18 Nov 2019 20:01:44 GMT</pubDate>
            <atom:updated>2019-11-18T20:01:44.640Z</atom:updated>
            <content:encoded><![CDATA[<p>Heart disease is the leading cause of death globally, resulting in more people dying every year due to cardiovascular diseases (CVDs) compared to any other cause of death [World Health Organization, 2017]. Rural mortality rates have surpassed those of urban areas as 75% of rural primary-care is handled by unqualified practitioners owing to an acute shortage of doctors.</p><p>Currently, there are two effective ways to monitor one’s heart condition; electrocardiogram (ECG) and echocardiogram. However, both methods are relatively expensive for mass inspection and require technical expertise in using them. Our goal is to develop a reliable, fast and low-cost system that can be used by untrained frontline health workers or anyone with internet access, to help determine whether an individual should be referred for expert diagnosis, particularly in areas where access to clinicians and medical care is limited. This will also help in early diagnosis of CVDs which will drastically decrease the potential risk factors of these deaths [1].</p><p>In this work, we take stethoscope sounds and even waveforms recorded using the microphone of a mobile phone as input and apply deep learning to the task of automated cardiac auscultation, i.e. recognizing abnormalities in heart sounds. We describe an automated heart sound classification algorithm that combines the use of time-frequency heat map representations with a deep convolutional neural network (CNN).</p><h3>Dataset</h3><p>This dataset was originally used in a Machine Learning challenge for the classification of heartbeat sounds by Mr Peter Bentley [2]. The dataset is divided into 2 sets depending on the sources from where it was collected. Set A (set_a.csv) data was collected from the general public via the iStethoscope Pro iPhone app and Set B (set_b.csv) from a clinical trial in hospitals using the digital stethoscope DigiScope.</p><p>In this dataset there are 4 classes of heartbeat sounds:</p><ol><li>Normal: healthy heart sounds</li><li>Murmur: extra sounds that occur when there is turbulence in blood flow that causes the extra vibrations that can be heard</li><li>Extrahls: heartbeats with an additional sound</li><li>Extrasystoles: are additional heartbeats that occur outside the physiological heart rhythm and can cause unpleasant symptoms</li></ol><p>As a part of the project, we were required to collect heartbeat from local clinics and hospitals to test our model. However, the couple of hospitals we talked to denied permission to do so on the grounds of acquiring an IRB approval.</p><p>The Institutional Review Board (IRB) deals with the review of behavioural research which involves human participants. The IRB approves and monitors all studies involving human participants to ensure that the experiment or survey is designed to meet legal and ethical concerns and does not involve unnecessary risks for the participants. All proposed research involving ‘human subjects’ must be reviewed and approved by the IRB before recruitment and data collection can start. The IRB ensures that the participants are adequately informed of what their participation entails (e.g.: risks and benefits). This includes written and signed informed consent in most cases.</p><p>Despite the above-mentioned constraint, we were successfully able to collect 58 data samples from small local clinics for testing our model.</p><h3>Pre-Processing</h3><p>The heart sounds recorded by digital stethoscope and the mobile phone microphone often has background noise. The preprocessing of heart sounds is an essential and crucial step for automatic analysis of heartbeat recordings. We have cut out all the audio files that have a duration of fewer than 3 seconds because they do not contain enough data points to accurately classify heartbeats. The recordings have to be converted to some fixed length prior to training, we slice the heart sounds into fixed-length segments of length 3 sec. To increase the size of the dataset we are slicing large files into multiple smaller files while still retaining their original label (i.e. normal or abnormal) We are also cutting about half a second from the start and end of all the audio files because the noise is due to the contact of the microphone with the body.</p><h3>Feature Extraction</h3><p>The raw audio data used .wav type (Waveform Audio File Format) was in the amplitude vs time form in the time domain. We transformed this one-dimensional time-series signal into a two-dimensional heat map that captures the time-frequency distribution of the signal. This representation of the spectrum of frequencies of a signal as it varies with time is called a spectrogram. Since the data was collected using different instruments (ie. digital stethoscopes and mobile phone microphone) which results in varying amplitude ranges, converting the data to the frequency domain leads to more accurate results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TgiuLlFCz9-gN3zoQpKSeQ.png" /></figure><p>In the frequency domain, we have used the following for feature extraction:</p><ul><li><a href="https://en.wikipedia.org/wiki/Mel-frequency_cepstrum"><strong>Melspectrogram</strong></a>: It represents an acoustic time-frequency representation of a sound. It is basically a spectrogram with the Mel Scale as its y-axis. This Mel Scale is constructed such that sounds of equal distance from each other on the Mel Scale, also “sound” to humans as they are equal in distance from one another.</li><li><strong>MFCC (Mel Frequency Cepstral Components) </strong>[3]: We chose to use Mel Frequency Cepstral Coefficients to perform this transformation, as MFCCs capture features from audio data that more closely resembles how human beings perceive loudness and pitch. MFCCs are commonly used as a feature type in automatic speech recognition.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AxND474IxXUe2tc9FjtBxw.png" /></figure><p>From <a href="https://librosa.github.io/librosa/">librosa</a> library in python, we used the inbuilt function for the above as follows:</p><ol><li>The preprocessed sound files are transformed into magnitude spectrogram and then mapped onto the mel-scale by using the inbuilt feature method of librosa to get the mel-scaled spectrogram.</li><li>A cepstral analysis is performed on the Mel-Spectrum to obtain Mel-Frequency Cepstral Coefficients (MFCC) by passing the log-power Melspectogram as an argument to the MFCC function.</li><li>Thus the audio file is now represented as a sequence of Cepstral vectors. These Cepstral vectors are then given to the model for anomaly detection</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s4jjH4MlAI90J18P6ezNqg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0kGB_d5jTxJSFLuFEkKg1g.png" /><figcaption>Visualising the data and features</figcaption></figure><h3>Model</h3><p>The original one-dimensional time series data is transformed into a two-dimensional time-frequency representation (i.e. spectrogram), which allows each heart sound segment to be processed as an image. The Convolutional Neural Network (CNN) is one of the neural network architecture specifically used for image classification. Just like other neural network methods, CNN is also inspired by human brain tissue. Convolution neural network is mainly composed of two parts, feature extraction, and classification.</p><p>The network architecture of a convolutional neural network that accepts as input a single channel 40x130 MFCC heat map and outputs a binary classification, predicting whether the input segment represents a normal or abnormal heart sound.</p><p>Convolutional Neural Network in this study uses 4 convolution layers, 4 max-pooling layers, 4 dropout layers, 1 global average pooling layer and finally a dense layer. The activation function in convolution layers uses Rectifier Linear Unit (ReLU) algorithm. The ReLU algorithm has advantages in time efficiency for training and testing.</p><p><strong>The dropout layer</strong>: The term “dropout” refers to dropping out units (both hidden and visible) in a neural network. It is a very efficient way of performing model averaging with neural networks. Model averaging is a natural response to model uncertainty. The dropout layer allows for regularization by randomly setting some neurons in previous layers to zero during training.</p><p><strong>Max Pooling</strong>: The objective of Max pooling is to down-sample an input representation. It helps in reducing the dimensionality and alleviate feature extraction. It reduces the computational cost-reducing the number of parameters to learned.</p><p><strong>Dense layer</strong>: Here every input is connected to every output by weight. And we are using softmax as the non-linear activation function after this layer.</p><p>Adam method is used for the optimization process to update the weight on the Convolutional Neural Network. This method has efficient computation (memory and time), invariant to gradient scaling and suitable when applied to large data or parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ac4cbQXZUKGFvfS_-2KDw.png" /><figcaption>Model summary</figcaption></figure><h3>Conclusion</h3><p>This study was designed to detect an abnormal heartbeat sound using Stethoscope sounds and heartbeats recorded through the microphone of a mobile phone. Classification of heartbeat sounds was conducted using a Convolutional Neural Network. We did not use any other time sequence based Neural Networks such as RNNs since the temporal behaviour of the heartbeat was repeated within the window of observation and different sequential patterns were not needed to be learnt.</p><p>The work presented here is one of the first to apply deep convolutional neural networks to the task of automated heart sound classification of heartbeat sound recorded through a stethoscope. We developed a novel algorithm which first transforms the one-dimensional time-series input into a two-dimensional time-frequency Melspectrogram. It then trains a 4-layer CNN architecture on the MFCC obtained from the Melspectrogram. The trained network automatically distinguishes between normal and abnormal heartbeat sound inputs. The epoch values used were 100, 150, 200, 250 and 300. The best results were obtained with 300 epoch at 0.001 learning rate applied on batches of 128. The training accuracy is <strong>89.73</strong>, while the testing accuracy rate is <strong>84.04</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V76kJzByItMcHrzBLp2p6g.png" /></figure><p>I recorded my own heartbeat using the microphone of my mobile phone. On classifying it using the trained model, I got “normal heartbeat” with 96% confidence.</p><p>For further information, you can refer to the <a href="https://github.com/MananAgarwal">code</a> on Github.</p><h3>References</h3><p>[1] [Z.-J. Yang, J. Liu, J.-P. Ge, L. Chen, Z.-G. Zhao, and W.-Y. Yang, 2011] “Prevalence of cardiovascular disease risk factor in the Chinese population: the 2007–2008 china national diabetes and metabolic disorders study,” European heart journal, vol. 33, no. 2, pp. 213–220, 2011.</p><p>[2] [Bentley P. and Nordehn G. <em>et al.</em>,2011] The Classifying Heart Sounds Challenge 2011 <a href="http://www.peterjbentley.com/heartchallenge/index.html">http://www.peterjbentley.com/heartchallenge/index.html</a>&quot;</p><p>[3] [Davis and Mermelstein, 1980] Steven Davis and Paul Mermelstein. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE transactions on acoustics, speech, and signal processing, 28(4):357–366, 1980.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62871fe2fbe5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why an Internship Over the Summer Might Be Just What You Need?]]></title>
            <link>https://medium.com/summer-in-a-startup/why-an-internship-over-the-summer-might-be-just-what-you-need-ef4f31f2094c?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef4f31f2094c</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[summer-internships]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[internships]]></category>
            <category><![CDATA[jobs]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Wed, 02 Jan 2019 19:51:15 GMT</pubDate>
            <atom:updated>2019-01-02T19:51:15.273Z</atom:updated>
            <content:encoded><![CDATA[<p>A hard disk full of high-end games, movies, TV shows, a riveting music collection and all that you could possibly find on your college’s Intranet is what most people look forward to when their summer vacations come close. But what if you could finish all the things on your summer vacations’ bucket list along with an internship <em>that matters</em>, and getting a chance to learn new things and apply your skills in the real world?</p><p>You might be wondering why one should intern during their vacations after a boisterous semester and that too in a startup. Let’s look at why a startup might be the best way to make your summer productive and also have fun while you’re doing so.</p><p>An internship allows you to gain experience of working in an organisation while studying in college. Internships provide to you the right opportunities to figure out your career options. It gives you a playground to test your skills and improve them to meet the industry standards. Many such opportunities even help with jobs down the line. Internships during college breaks can be a really rewarding experience for the journey ahead.</p><p>At a startup, you will learn things that few other places can teach. What you will take away might change the way you solve problems.</p><h4>1. Test Drive your Knowledge and Skill</h4><p>Whether you are a techie or non-techie, an internship in a startup gives you an opportunity of doing something you’re passionate about. You can try your knowledge and skills to find out your true passion, Would I be happier with marketing or advertising? Can I really sit for 10 hours behind a screen coding? Am I good with designing? These are the questions which present the answers themselves. Startups allow you to apply your skills to a broader spectrum of things, which is never the case when you’re interning in a big MNC. This gives your work much more meaning and makes you feel more responsible. Remember, skills dominate everything else.</p><h4>2. Building a good network</h4><p><strong>“</strong>It’s not about what you know; It’s about who you know.” An internship in a startup gives you the opportunity to increase your network, expand your professional branding and connect with great mentors. The benefits of networking show up gradually. It helps you know people across the industry, bring in better future prospects, etc. You can turn to these people for career advice in the future or use them as your reference to land better internship/job opportunities.</p><h4>3. Time Management</h4><p>Your internships bring in your first work experience. And, it is during your internship only that you realize managing your time between academics and co-curricular activities was barely a task. Handling individual and group tasks and finishing them before the deadline makes you learn how to balance everything with efficient time management.</p><h4>4. There’s no readymade solution to everything</h4><p>Till now, there was almost nothing that you couldn’t find in one of the books in the library or on Google, isn’t it? But internships teach you that it doesn’t work that way, especially in the startup world. At work, there will be substantial and unanticipated sector-related problems for which there won’t be any discovered solution yet. You will need to find one yourself, based on your knowledge, intelligence, and contemporary trends. And it would be this problem-solving attitude that will help you step-up higher on the ladder of success.</p><h4>5. Every Experience Counts</h4><p>Internship in a startup provides you with a lifetime of experiences to explore new things, opportunities and life lessons that you will never forget. It provides you with the opportunity to look into various profiles. If you really liked a particular job profile at the end of your internship, you can plan a career in it. Also, interning in a startup provides you with the experience of working in an energetic and fun environment with highly talented individuals.</p><p>Considering all of this, Center for Entrepreneurship and Leadership (CEL) brings you the right platform to land summer internships in some of India’s most successful startups. Stay tuned on our <a href="https://www.facebook.com/bits.sis/">facebook page</a> for more updates on our project, Summer In A Startup (SiS).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef4f31f2094c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/summer-in-a-startup/why-an-internship-over-the-summer-might-be-just-what-you-need-ef4f31f2094c">Why an Internship Over the Summer Might Be Just What You Need?</a> was originally published in <a href="https://medium.com/summer-in-a-startup">Summer in a Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 4.02: Wrapping up]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-4-02-wrapping-up-66346255bc6c?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/66346255bc6c</guid>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[astropy]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[sphinx]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Mon, 13 Aug 2018 16:24:38 GMT</pubDate>
            <atom:updated>2018-08-13T20:27:23.336Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World!</p><p>The obscenity of time dumbfounds me as I find myself staring down the barrel of this final blog for my GSoC project. The past three months were filled with coding, debugging, optimising, taking feedback and working on suggestions. It feels like I just started contributing to <a href="http://www.astropy.org/">Astropy</a>, indeed how time flies!</p><p>For all those who haven’t been following <a href="https://medium.com/@manan_agarwal">my blog</a>, my GSoC project “Learn Astropy” is with Astropy, an organisation under the umbrella of <a href="https://openastronomy.org/">OpenAstronomy</a>. Astropy is a python library which contains key functionalities and common tools needed for performing astronomy and astrophysics with python.</p><p>Astropy is a very welcoming and active community of like-minded people sharing common passion and interest for Astronomy. The community was very responsive to all my pull requests and doubts. I would like to thank the entire Astropy community, especially my mentors <a href="https://github.com/kelle">Kelle</a>, <a href="https://github.com/adrn">Adrian</a> and <a href="https://github.com/eteq">Erik</a> for being to so helpful and supportive at every step.</p><h3>My GSoC Project</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*GVjUn3GqB5rJ2fYY" /></figure><p>Over the summer I worked on “Learn Astropy”, a website for the Astropy Learn ecosystem. The Learn ecosystem includes the Astropy documentation and associated examples, Astropy tutorials, and guides. The goal was to make the Astropy Learn ecosystem more organised, accessible and efficient by adding features like dynamic search that works across all of the Learn resources. Astropy aims to provide users packages that improve reuse, interoperability, and interface standards for python astronomy and astrophysics packages. Learn Astropy will help both amateurs as well as professional astronomers and astrophysicists in exploring and implementing the various utilities and functionalities provided by Astropy. It will also simplify the process for anyone who is learning to work with python astronomy/astrophysics packages or anyone developing astronomy software.</p><p>The project involved optimising the dynamic search, making custom Sphinx directives, indexing and tagging of Astropy Learn material. Work was also done on user interface design and user experience. I worked primarily with javascript and python, and used Sphinx for generating the html build files from jupyter notebooks. The project “Learn Astropy” was a very good learning experience as it gave me an exposure to open source development and at the same time allowed me to work with a community passionate about Astronomy and Computer Science.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n8wIr2_ARDB7QjaV" /></figure><ul><li>GSoC Proposal: <a href="https://drive.google.com/file/d/13q3YYJCG4yBtY9t_Dcb8o_pL2AgA36qH/view?usp=sharing">Learn Astropy</a></li><li>Repositories: <a href="https://github.com/MananAgarwal/astropy-tutorials/tree/new-theme">Astropy-Tutorials(new theme)</a> and <a href="https://github.com/MananAgarwal/Learn-Astropy-Testing">Learn-Astropy-Testing</a></li><li>Commits: <a href="https://github.com/MananAgarwal/astropy-tutorials/commits/new-theme">List of github commits</a></li><li>Pull Requests: <a href="https://github.com/astropy/astropy-tutorials/pull/213">#213</a> and <a href="https://github.com/astropy/astropy-tutorials/pull/247">#247</a></li></ul><p>It was truly a Eureka moment when my pull request successfully passed both Travis build and Circleci build.</p><h3>Features</h3><ul><li>Dynamic Search across all the Astropy Learn material: A feature that searches for the query in all the Astropy Learn material simultaneously and displays the results in a unified list. The results are displayed on placards which are colour coded for different learn materials, have a thumbnail and an abstract to give the user a much better idea of the content. This search feature is implemented by extending and tweaking the default Sphinx full text search. Currently the dynamic search looks through the Astropy tutorials and documentation. This will be extended to Astropy examples and guides. I have covered this more extensively in <a href="https://medium.com/@manan_agarwal/gsoc-diaries-3-01-f048541d523b">this blog</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3VF_naU6HnmR6lvu" /></figure><ul><li>Filter by keyword: A list of relevant keywords are displayed on the sidebar which have a corresponding check-box. These check-boxes can be selected to apply filters by that keyword which narrows down the search results. It is possible to select multiple filters. This helps the user to easily find what he/she is looking for. To further improve the efficiency I have made these filters searchable. More on this <a href="https://medium.com/@manan_agarwal/gsoc-diaries-3-02-b39f8218415">here</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y0Yx_hocKM9L8Mew4RvzzQ.png" /></figure><ul><li>Navigation sidebar: All the tutorials have a table-of-contents sidebar, which has clickable links to the corresponding section. This gives the user an overall idea about the contents of the page and helps him/her to efficiently navigate through the page.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*pJmiqdi6F5iq3BQMwvWFrw.png" /></figure><p>Other than these, there are some small features like scroll retention, intuitive user interface, thumbnails, toggle for guides placard, etc.</p><h3>What next?</h3><p>GSOC 2018 may be coming to an end but my passion for Astronomy isn’t going anywhere. I want to contribute more to the Astropy community. The successful completion of the intended work has opened up new avenues to extend my work, like making a Sphinx plugin to extract examples from the Astropy documentation and developing a unified theme for all of Astropy. I have already started working on them and will continue even after my GSoC period ends.</p><p>All in all my GSoC experience was a memorable one and one that I will cherish. What made this special was the work, the learning and the people who came together to help me complete this amazing journey.</p><p>Signing off…</p><p>PS: Huge shout out to Numfocus for promoting <a href="https://numfocus.org/blog/google-summer-of-code-2018-part-4-final">me</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66346255bc6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 4.01: Bringing everything together]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-4-01-b9e4ae033c68?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9e4ae033c68</guid>
            <category><![CDATA[search]]></category>
            <category><![CDATA[sphinx]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Thu, 02 Aug 2018 22:01:01 GMT</pubDate>
            <atom:updated>2018-08-11T13:14:15.273Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World!</p><p>We are already more than halfway through the third and final coding phase of GSoC 2018. I have managed to successfully integrate the new Astropy tutorials theme with the existing Sphinx build process.</p><h3>How to make the new build?</h3><p>To make the Sphinx build of the Astropy tutorials with the new theme</p><pre>git clone <a href="https://github.com/MananAgarwal/astropy-tutorials.git">https://github.com/MananAgarwal/astropy-tutorials.git</a><br>cd astropy-tutorials<br>git checkout new-theme<br>pip install sphinx-bootstrap-theme</pre><p>Now you are all set to build the Astropy tutorials like you normally do i.e.</p><pre>git submodule init<br>git submodule update<br>source activate tutorials<br>sudo make html</pre><p>Voila! Astropy tutorials with the new theme are ready. ( If you are facing some error, most probably you just need to update nbconvert.)</p><h3>Implementation</h3><p>Files related to the new sphinx-theme are added in a ‘themes’ folder inside tutorials. Static files like Astropy logo, banner background, stylesheet are added in the ‘_static’ folder. I have then linked these inside the conf.py file to override the default configurations.</p><pre>html_theme_path = [path.abspath(path.join(path.dirname(__file__), ‘themes’))]</pre><pre>html_theme = ‘tutorials-theme’</pre><pre>def setup(app):<br> app.add_stylesheet(‘bootstrap_sandstone.css’)<br> app.add_stylesheet(‘astropy.css’)</pre><p>All the HTML build configurations are in the theme.conf file except for the sidebar configurations which is in the conf.py file.</p><p>While implementing this I was challenged by a number of weird errors and learned a lot of things the hard way like the importance of virtual environments, compatibility of different packages, etc. Some of these errors that are worth pointing out as they might trouble other developers also are:</p><h4>1. Build process fails due to some error in the tutorial</h4><p>Some of the tutorials have code which is intended to give an error as the output. These are to explain the usage and syntax of different packages. nbconvert is used to take care of this so that instead of throwing an error and stopping the build process, they are added to the tutorial build as output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*ZBxEAnyHxujf1GYcQtu3CQ.png" /></figure><p>This can be solved by updating the nbconvert to the latest release from its official GitHub repo.</p><pre>$ git clone <a href="https://github.com/jupyter/nbconvert.git">https://github.com/jupyter/nbconvert.git</a><br>$ cd nbconvert<br>$ pip install -e .</pre><h4>2. Module not found error (inside the virtual environment)</h4><p>Build process fails because it was unable to find a module. If you try installing this module inside the virtual environment then it will throw an error stating that the module is already installed. Weird right. It is surprising to find that by default this build does *not* use the same python environment we are running sphinx in. This is because the nbconvert machinery depends on Jupyter kernels to create a separated environment to run each notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/0*-jfKR_NztDivmoi9" /></figure><p>To solve this Erik has added the feature to explicitly state the environment we want nbconvert to run in. <a href="https://github.com/astropy/astropy-tutorials/pull/244">Pull request</a></p><p>To use a specific environment, you will need to use the ``jupyter kernelspec`` or ``ipykernel install`` command to create a named kernel for your favored environment. Then pass it into sphinx using the ``NBCONVERT_KERNEL`` environment variable. Something like::</p><pre>$ python -m ipykernel install — user — name astropy-tutorials — display-name “Python (astropy-tutorials)”</pre><pre>$ NBCONVERT_KERNEL=astropy-tutorials make html</pre><p>You can also resolve this by simply installing the module globally outside the virtual environment.</p><p>With less than two weeks remaining in GSoC 2018, I am working on the Astropy Examples Gallery. Seems like I’ll be soon making a pull request!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9e4ae033c68" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 3.02: All about tags]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-3-02-b39f8218415?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/b39f8218415</guid>
            <category><![CDATA[tag]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[sphinx]]></category>
            <category><![CDATA[search]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Mon, 30 Jul 2018 13:36:29 GMT</pubDate>
            <atom:updated>2018-08-11T14:05:40.675Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World!</p><p>I have successfully implemented the tags feature for the Astropy tutorials. Now when you visit the tutorials landing page, you will see all the tutorials on placards with the title and an abstract. You can find all the tags in a sidebar on the left-hand side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xQDQT7KXsThVg0kpt-7eRg.png" /><figcaption>Astropy Tutorials page</figcaption></figure><p>Each tag has a corresponding checkbox which you can select to filter results. Multiple tags can also be selected at the same time. For example I have selected both Astroquery and coordinates in the screenshot below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NX2pfoIuvQftoUYf3XsGrQ.png" /><figcaption>Astropy Tutorials page with multiple filters selected</figcaption></figure><h3>Implementation</h3><p>I added the tag “filterTutorials” to all the tutorial’s .rst file so that I have a keyword which is unique to tutorials. I then make a normal search query for it to display all the tutorials. I will similarly add the other relevant tags to all the tutorials to implement the filter-by-tags feature. Currently, when you select a tag, it just makes a full-text search in the tutorials for that tag and shows the results.</p><p>Next I am working on integrating the new Astropy tutorials theme with the existing Sphinx build mechanism.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b39f8218415" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 3.01: It’s taking shape]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-3-01-f048541d523b?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/f048541d523b</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[sphinx]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[gsoc-diaries]]></category>
            <category><![CDATA[astropy]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Wed, 27 Jun 2018 09:06:31 GMT</pubDate>
            <atom:updated>2018-08-11T14:03:36.537Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World!</p><p>It has been some time since I gave you any updates on Learn Astropy website, so here it is the Diary 3.01. Why 3 and not 2? Because the first coding phase of GSoC 18 has ended and in fact, we are already two weeks into the second phase. By the way, I have successfully passed the first evaluation of GSoC 2018..yay!</p><p>If you remember, in the last diary I mentioned how I improved the search results for Astropy tutorials but was yet to figure out how to incorporate Astropy Documentation. To implement this I initially tried making a search request to <a href="http://docs.astropy.org/en/stable/">Astropy Documentation</a> and then parsing through the HTML to get the relevant search results and finally injecting these results into the Learn Astropy website’s search page. This seemed to be a plausible solution, but when I sat down to implement this I ran into a ton of problems. The one which brought an end to this solution was that the Astropy Documentation&#39;s HTML response got collected even before the search results came in. So basically, I was getting the skeleton of the search page before the results were even inserted through an AJAX call. While going through the network traffic I found the API call which was returning the JSON object to be injected into the Astropy Documentation’s search page. So now, I directly send a search query to this API and inject the top results from the JSON object into the Learn Astropy search page. But as this needs making a GET request to a different server, it is blocked by the Cross-Origin Resource Sharing (CORS) restriction of the web browser. We will have to find a workaround for this problem. One obvious solution is to host both documentation and Learn Astropy website from the same place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QNS2INbsM0zZHt2Qa4jrMw.png" /><figcaption>search results when queried for “theme”</figcaption></figure><p>The results with an orange border on the left are from the Astropy tutorials and the ones with a yellow border are from the Astropy Documentation. The text above the search result placards notifies the user about the number of results found from each category.</p><p>Guides, tutorials, documentation and examples now have their respective landing pages other than the main landing page of Learn Astropy website. I have also made some changes in how the individual tutorial page look. Earlier the text of the tutorials was preceded by the table of contents like on a Wikipedia page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LMKx0nNRe4uq7hvWxgmwiA.png" /><figcaption>Old tutorial page</figcaption></figure><p>Now, the table of content appears on a sidebar which sticks to the screen even when the page is scrolled down, so that the table of contents is always visible to the user. Also, you may have noticed that I have swapped the position of guides and tutorials placards. This is to arrange the categories in a proper hierarchy and to retain the colour scheme even when the guides are removed from Learn Astropy website (until guides are made).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pJmiqdi6F5iq3BQMwvWFrw.png" /><figcaption>New tutorial page</figcaption></figure><p>I am currently working on the tags and filters feature. I have started implementing it in the tutorials landing page. When completed, it will have all the Astropy tutorials listed on the right with a small abstract and an image, and corresponding filters in a sidebar on the left. These filters will be in the form of checkboxes. So when a users checks one or more filters, tutorials relevant only to those filters will be visible. Surprisingly, there is no inbuilt or easy way of implementing tags in Sphinx. I’ll talk more about the implementation in the next GSoC Diary. Till then happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f048541d523b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 2.02: Skeleton is ready]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-2-02-4dd30376609a?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/4dd30376609a</guid>
            <category><![CDATA[sphinx]]></category>
            <category><![CDATA[astropy]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[search]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Sat, 09 Jun 2018 17:51:11 GMT</pubDate>
            <atom:updated>2018-08-11T14:01:59.640Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World!<br>The first coding phase of GSoC 2018 ends tomorrow, and it will be followed by the first evaluation. The last two weeks were quite productive and had lots of interesting things happening in the tech world like Apple’s WWDC 2018 (I was really hoping for a new MacBook Air…sigh) and Microsoft acquiring Github.</p><p>I have made significant progress in the project since the last GSoC diary, this was not without hurdles though, but then that’s what makes GSoC fun. So the last time I wrote the GSoC diary, I was having troubles making Sphinx display the content in a way that I wanted it to. But guess what, after making my own Sphinx theme using Jinja and Bootstrap I am now able to completely control the display. I still need to work on the responsiveness of the display but it works perfectly on desktops or any other devices which have a big screen (and an internet connection). The next step was to build Astropy tutorials using this new theme. My mentors really liked the new theme, and we discussed on adding some new features and the utility of these features. We decided to add a clickable table of contents section on the top of each page. This will help in navigating through the page and will also provide a good overview of all the topics covered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eulv1Hz_EihiJDPFqvbfQA.png" /><figcaption>Screenshot of the Learn Astropy landing page</figcaption></figure><p>The most interesting part was working on the search functionality. Sphinx provides a basic full-text search capability, but this needs some tweaks before it starts giving you some smart results. I played around with the function, changing parameters and weights to improve the quality and order of search results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gciVa2RaOEj0YHXA" /></figure><p>Can you spot a problem in the above screenshot? Yes, it is returning HTML tags as a part of the context. That doesn’t look pretty. I wanted the context to have some instance of the search query (highlighted in yellow) and no HTML tag or styling symbol from the .rst file. What the current script was doing was simply searching for the query and dumping a string having the first instance of this query. So my task was to make this script return a more relevant instance of the query without any useless tag an symbol. This is what the end result was:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GCJ3Nv_pCnmNHyyLp4cAWA.png" /></figure><p>This is not as smart as a Google search result but definitely a significant improvement from the previous results. I will be adding tags and metadata to further improve the quality of results. Now that the search page is displaying good search results from Astropy tutorials, my next task is to get the search results for the same query from Astropy documentation to be incorporated into the same results list. I have a few ideas on how to make this work, but I’ll have to implement these (and many more) before finding an optimal solution. This is my task for the coming week.</p><p>Will share the solution I found in the next GSoC diary. Till then happy coding! And all the best to all my fellow GSoCers for the first evaluation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4dd30376609a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 2.01: First Coding Phase]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-2-01-3844137a778b?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/3844137a778b</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[gsoc-diaries]]></category>
            <category><![CDATA[sphinx]]></category>
            <category><![CDATA[gsoc]]></category>
            <category><![CDATA[astropy]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Tue, 29 May 2018 11:33:09 GMT</pubDate>
            <atom:updated>2018-08-11T13:05:08.713Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World,</p><p>I completed two weeks of coding with Astropy yesterday, which means we are already halfway through the first Coding Phase of GSoC! These two weeks began with me developing the landing page of Learn Astropy Website using bootstrap 3. This was not a functioning webpage, as it had many parts hardcoded which must have been dynamically rendered based on the search query. But it gave us an idea of how the page will look like on a web browser and how the overall user interaction will feel like. I realised that to smoothen the user experience I had to make some minor changes like placements of some components, prioritising information hierarchy, tweaking the shade of some colours so that it grabs users attention subconsciously.</p><p>The next task was obviously to make this dummy page into a functioning (at least partially) webpage. So for this, I had planned on using Sphinx documentation tool, which makes it easy to create intelligent documentation. It was originally created for making the Python documentation. As this was my first time using Sphinx, it took me some time to understand its workflow. I realised it the hard way that Sphinx focuses primarily on the functionality of a documentation and not on how presentable can we make it look. Though it provides some built-in themes, none of them satisfies the design standards for this decade and it is not easy to edit these. So I had to make my own theme which was consistent with the mockups. As I wanted some structural changes also, I had to first create a sphinx template and for that, I ended up learning Jinja which is a templating engine. Sounds like a lot right! But trust me, these are all easy to understand but the problem is lack of intuitive interface and a developer community to update these to the new standards. Luckily I found a GitHub repository of customisable sphinx theme using bootstrap. This should reduce my work exponentially, as now bootstrap can take care of responsiveness and consistency. I have started building my own template and custom theme for Learn Astropy. Last week was not so productive for me, but hopefully, I’ll cover up this week now that I have developed decent understand of Sphinx.</p><p>I better start coding :) .See you guys in the next GSoC diary. Happy Coding folks!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3844137a778b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GSoC Diaries 1.01: The journey begins!]]></title>
            <link>https://medium.com/@manan_agarwal/gsoc-diaries-1-01-e6fb1c0180b4?source=rss-2c6915d07485------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6fb1c0180b4</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[gsoc-diaries]]></category>
            <category><![CDATA[astropy]]></category>
            <category><![CDATA[gsoc]]></category>
            <dc:creator><![CDATA[Manan Agarwal]]></dc:creator>
            <pubDate>Wed, 16 May 2018 17:47:28 GMT</pubDate>
            <atom:updated>2018-08-12T09:11:33.192Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OSiVlCsvAcknfAOu." /></figure><p>Hello World,</p><p>After spending numerous nights scouting for the perfect organisation among the 212 participating in GSoC and the perfect project from the plethora of projects offered, I managed to submit two proposals. And now, after 2-months of toiling hard, innumerable coffee breaks and a month long unsettling application review period, I finally got the mail from Google congratulating me on getting selected for GSoC 2018!!! Doing a project under GSoC was a dream I had since my first year at college and now that it has finally come true, I am really looking forward to the upcoming 3-months of learning, designing, coding, debugging, optimising and getting feedback of the mentors.</p><p>For all those who don’t know about <a href="https://summerofcode.withgoogle.com/">Google Summer of Code</a> (GSoC). It’s an international annual program by Google which provides an opportunity for college students to contribute to free and open-source software projects over the summer and earn some money at the same time.</p><p>What’s GSoC diaries? It is a series of blogs that I’ll be writing about my experience, learning, coding, basically everything related to my GSoC 2018 project “Learn Astropy Website” with <a href="http://www.astropy.org/">Astropy</a> under the umbrella organisation <a href="https://openastronomy.org/">Open Astronomy</a>. This is my first blog ever and I am also not much of a writer, so bear with me.</p><h3>My GSoC organisation</h3><p>As I have previously stated, I am doing my GSoC project with <a href="https://openastronomy.org/">Open Astronomy</a>, which is a collaboration between open source astronomy and astrophysics projects that are used by researchers and engineers around the world to study our universe either by analysing the data obtained from amazing instruments like the <a href="http://www.nasa.gov/hubble">Hubble Space Telescope</a>, the <a href="https://www.skatelescope.org/">Square Kilometer Array</a> or the <a href="http://sdo.gsfc.nasa.gov/">Solar Dynamic Observatory</a>, etc.</p><p>Among the <a href="https://openastronomy.org/members/">12 organisations</a> that currently form OpenAstronomy, I am working with Astropy, which is a community-driven package intended to contain much of the core functionality and some common tools needed for performing astronomy and astrophysics with Python. <a href="http://astropy.org/">I</a>t provides common tools such as celestial coordinates, image processing, tabular data reading and writing, units and support for astronomy-specific file formats. The Astropy Project is fiscally sponsored by <a href="http://www.numfocus.org/">NumFOCUS</a>.</p><h3><strong>My GSoC project</strong></h3><p>So what am I working on? Concisely, it’s a website for the Learn Astropy ecosystem which will consolidate Astropy documentation, Astropy tutorials, guides and examples in a single dynamically searchable web page. You will get to know more about it in my upcoming posts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GVjUn3GqB5rJ2fYY" /></figure><h3><strong>Community Bonding</strong></h3><p>The community bonding period ended on the 14th May and now we have got to the part where we write some code. The community bonding period is for the students to connect with the organisation and become a part of it. I got to interact with some great coders and astrophysicists. We had discussions on the features of the website and workflow of the project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vTaAVdT1xQEc2uig." /></figure><p>We brainstormed on different mockups and designs of the website. Where should the search bar be placed? What should be the colour scheme? Would the red color read as “danger/warning/stress” red, especially given that the Tutorials should be the most accessible of the products. Or will the red quickly grasp the attention of the user to tutorials and make it more accessible. All these discussions on what is best for good user experience made the team more clear on what we are looking for from the website and how we see the Learn Astropy Website functioning for the users.</p><p>I am looking forward to having a great summer ahead, while I work with Kelle and Adrian (my mentors for this project) on the Learn Astropy Website.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6fb1c0180b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>