<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nuno Ramos Carvalho on Medium]]></title>
        <description><![CDATA[Stories by Nuno Ramos Carvalho on Medium]]></description>
        <link>https://medium.com/@nunorc?source=rss-4ff11b0113ad------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*EotHlYe4BtivhUvHbLJIJg.jpeg</url>
            <title>Stories by Nuno Ramos Carvalho on Medium</title>
            <link>https://medium.com/@nunorc?source=rss-4ff11b0113ad------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:10:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nunorc/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hunting Exoplanets in Kepler Data — Introduction to lightkurve]]></title>
            <link>https://medium.com/@nunorc/hunting-exoplanets-in-kepler-data-introduction-to-lightkurve-87ece24aa5f6?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/87ece24aa5f6</guid>
            <category><![CDATA[kepler]]></category>
            <category><![CDATA[astronomy]]></category>
            <category><![CDATA[exoplanets]]></category>
            <category><![CDATA[astrophysics]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Mon, 16 Mar 2020 19:54:05 GMT</pubDate>
            <atom:updated>2020-03-16T19:54:05.750Z</atom:updated>
            <content:encoded><![CDATA[<h3>Hunting Exoplanets in Kepler Data — Introducing the lightkurve package</h3><p>Exoplanets are planets that live outside our Solar System. This post briefly introduces processing Kepler data using the <em>lightkurve </em>Python <em>package, </em>to study exoplanets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Ovs1KzgljkGScNKpFYnQQ.jpeg" /><figcaption>Image credit: <em>Gemini Observatory/NSF/AURA/Artwork by Joy Pollard.</em></figcaption></figure><h4>Introduction</h4><p>The <a href="https://lco.global/spacebook/exoplanets/transit-method/">transit metho</a>d is currently one of the most popular indirect methods used to discover exoplanets. In a nutshell, it consists in analysing the brightness variation of the parent star when the planet crosses the light path of the object with respect to the observer. The now retired <a href="https://www.nasa.gov/mission_pages/kepler/main/index.html">Kepler </a>telescope was launched by NASA to detect planets orbiting other stars and is responsible for helping in detecting most of the exoplanets known today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*IIefg2kiENGyU2Y34TxUEQ.gif" /><figcaption>A planet transit causing the star brightness to dim, image credit: <a href="https://imgur.com/43f17Ke">https://imgur.com/43f17Ke</a>.</figcaption></figure><p><a href="https://github.com/KeplerGO/lightkurve"><em>Lightkurve</em></a><em> </em>is a Python package for Kepler &amp; TESS time series analysis, providing a user-friendly way to analyse astronomical flux time series data. The remaining of this post illustrates how to use this package to study the light curve for <strong>Kepler-427 b</strong>. We start by importing the package:</p><pre>import lightkurve as lk</pre><h4>Kepler-427 b</h4><p><a href="https://exoplanets.nasa.gov/exoplanet-catalog/5261/kepler-427-b/">Kepler-427 b</a> is a gas giant orbiting a G-type start, with mass of 0.29 <em>Jupiters</em>. The first step to analyze this systems’ Kepler observations, is to retrieve the data for the system. We could retrieve the raw pixel files (more on this in a future post) but to keep it simple we choose to get the light curve file, already built by the Kepler pipeline. To download the light curve file, we simply call the package search_lightcurve function, giving as argument the target system, and the quarter that we are interested in (Kepler observations are divided in quarters):</p><pre>lcf = lk.search_lightcurvefile(&#39;kepler-427&#39;, quarter=6).download()</pre><p>This downloads the required data files, and from the lcf object we can retrieve many artifacts, including two light curves: for the Simple Aperture Photometry (SAP) flux, and for the Pre-search Data Conditioning PDCSAP flux (that includes some corrections for star activity and instrumental errors), both light curves are readily available. Using the the following code we can retrieve and visualize the PDCSAP curve:</p><pre># retrieve the PDCSAP light curve from the file<br>lc = lcf.PDCSAP_FLUX</pre><pre># plot the light curve<br>lc.plot()</pre><p>The lc object now stores the PDCSAP light curve, and the following figure illustrates the corresponding plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*4KlEo8P9KotPJ-0jMV1UPA.png" /><figcaption>Kepler-427 PDCSAP light curve for Quarter 6.</figcaption></figure><p>By eyeballing the plot we can identify a repeating dim in the flux, a possible exoplanet transit. Next, we create a normalized <em>periodogram</em> (useful for characterizing periodicity in unevenly sampled time series) that will help identifying the signals period with the to_periodogram function using the <a href="https://docs.astropy.org/en/stable/timeseries/bls.html">Box Least Squares</a> (BLS) method:</p><pre># create the periodogram from the light curve<br>pg = lc.to_periodogram(method = &#39;bls&#39;)</pre><pre># plot the periodogram<br>pg.plot()</pre><p>The following figure illustrates the <em>periodogram </em>plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*FCRnr8RPaxHYZxehr3uv0A.png" /><figcaption>Kepler-427 PDCSAP light curve periodogram plot.</figcaption></figure><p>From a visual analysis of the plot, we identify a strong periodic signal, with a period of a little over 10 days. We can ask the <em>periodogram</em> pg object for the period at the maximum frequency:</p><pre>print(&#39;Period at max power: {:.3f}&#39;.format(pg.period_at_max_power))<br># Period at max power: 10.297 d</pre><p>Now that we have a pretty good idea of the period of the peak of our transit, we can fold all the observations over this period, i.e. overlap all the observations of the transit, using the fold function from the light curve object lc, given the period:</p><pre># fold the curve given the period<br>fold = lc.fold(period = pg.period_at_max_power)</pre><pre># plot the fold<br>fold.plot()</pre><p>The following image illustrates the folded light curve:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*uu40E4vpx23Mx9Nvo23Wkg.png" /><figcaption>Kepler-427 folded light curve for the inferred period.</figcaption></figure><p>And there we go, we can no proceed in characterising this exoplanet orbit around the hosting star, give that we have a pretty good idea of the period and can also estimate the estimates of planet diameter from the depth of dim of the star flux. Or maybe fitting a model to the observed data.</p><p>The <em>lightkurve </em>package provides a lot of useful functions to quickly process Kepler data, the full documentation is available from its’ <a href="https://docs.lightkurve.org/">homepage</a>. More useful and interesting functionalities are available, this post is just a brief introduction, we’ll discuss more complex operations in future posts.</p><p>A complete notebook with the code and illustrated results for this post is available <a href="https://colab.research.google.com/drive/10FlxeWrYzmX8zHHI-XaptD3smD_2jiD8">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87ece24aa5f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Question-Answering Model Fine-Tuned for Portuguese]]></title>
            <link>https://medium.com/@nunorc/question-answering-model-fine-tuned-for-portuguese-801bb05b6119?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/801bb05b6119</guid>
            <category><![CDATA[portuguese]]></category>
            <category><![CDATA[pln]]></category>
            <category><![CDATA[question-answering]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Mon, 19 Aug 2019 19:44:28 GMT</pubDate>
            <atom:updated>2019-08-19T19:44:28.109Z</atom:updated>
            <content:encoded><![CDATA[<h4>This post introduces <a href="https://github.com/nunorc/qaptnet"><strong>qaptnet</strong></a>, an instance of Google’s <a href="https://github.com/google-research/bert">BERT</a> model, fine-tuned for performing question-answering tasks in Portuguese text.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YIOu0pqXimD65uH5JBAPFA.jpeg" /><figcaption>Luís de Camões, “Os Lusíadas”</figcaption></figure><p>The model is trained on a <a href="https://github.com/nunorc/squad-v1.1-pt">dataset</a> of contexts, questions, and corresponding answers written in Portuguese. The output of the model is the span of text in the context that answers the proposed question. For example. given the following context:</p><pre>context = &#39;Arquitetonicamente, a escola tem um caráter católico. No topo da cúpula de ouro do edifício principal é uma estátua de ouro da Virgem Maria. Imediatamente em frente ao edifício principal e de frente para ele, é uma estátua de cobre de Cristo com os braços erguidos com a lenda Venite Ad Me Omnes. Ao lado do edifício principal é a Basílica do Sagrado Coração. Imediatamente atrás da basílica é a Gruta, um lugar mariano de oração e reflexão. É uma réplica da gruta em Lourdes, na França, onde a Virgem Maria supostamente apareceu a Santa Bernadette Soubirous em 1858. No final da unidade principal (e em uma linha direta que liga através de 3 estátuas e da Cúpula de Ouro), é um estátua de pedra simples e moderna de Maria.&#39;</pre><p>And the following question:</p><pre>question = &#39;A quem a Virgem Maria supostamente apareceu em 1858 em Lourdes, na França?&#39;</pre><p>The model infers the span on the context that yields the answer to the question:</p><pre>&gt;&gt;&gt; ptnet.query(context = context, question = question)<br>&#39;Santa Bernadette Soubirous&#39;</pre><h4>The Model</h4><p>Bidirectional Encoder Representations from Transformers (BERT) is a state-of-the-art model for many NLP tasks, e.g. classification, named entity recognition, and question-answering. In a nutshell the model is divided in two major sections: 1) a pre-trained transformer that acts as a language model, and 2) a section fine-tuned for the task at hand, as illustrated by the following figure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BERtOa_tIfE1kRNAoJKtGA.png" /><figcaption><a href="https://arxiv.org/abs/1810.04805">https://arxiv.org/abs/1810.04805</a></figcaption></figure><p>The input to the model in the context and the question, and the output is the starting and ending token of the span of the answer in the context. Google provides different flavors of pre-trained weights for the model, mainly single language or multilingual, and with different number of hidden layers. For training <a href="https://github.com/nunorc/qaptnet">qaptnet</a> the multilingual, case sensitive, 12 hidden layers, weights were used to bootstrap the model, and the train part of the dataset described in the next section was used to fine-tune the model. Implementation, training and deployment is done using the <a href="https://github.com/huggingface/pytorch-transformers">PyTorch-Transformers</a> package.</p><h4>The Dataset</h4><p>To train the model for Portuguese a collection of questions and contexts, with the corresponding answers. To build this dataset we translated the <a href="https://rajpurkar.github.io/SQuAD-explorer/">SQuAD</a> dataset from English to Portuguese using the Google Translate API. The original dataset contains a collection of around 60000 pairs question answer. of around 400 topics from Wikipedia. In general the automatic translation result was good enough for a first version, mainly because the dataset snippets of text are small enough to yield good results from an automatic translation and the type of texts (Wikipedia articles) is not very complex or abstract.</p><h4>Wrapping Up</h4><p>The final version of the model achieves around 50% accuracy on the development part of the dataset, which is interesting enough for a first exploratory attempt. Further improvements to the dataset translation, and fine-tuning the model hyper-parameters, can easily improve the accuracy score. <a href="http://pln.pt/">PLN.pt</a> provides access to the current version of model via it’s <a href="http://pln.pt/api">API</a> for easy access. Examples and how to query the API are available from the <a href="https://github.com/nunorc/qaptnet">qaptnet</a> repository.</p><p>Current versions of the model and the dataset are available in their repositories on GitHub:</p><ul><li><a href="https://github.com/nunorc/squad-v1.1-pt">https://github.com/nunorc/squad-v1.1-pt</a></li><li><a href="https://github.com/nunorc/qaptnet">https://github.com/nunorc/qaptnet</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=801bb05b6119" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Galaxies Morphology Classification Using Convolutional Neural Networks]]></title>
            <link>https://medium.com/@nunorc/galaxies-morphology-classification-using-convolutional-neural-networks-ed5d7d82fbf?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed5d7d82fbf</guid>
            <category><![CDATA[cnn]]></category>
            <category><![CDATA[astrophysics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Mon, 17 Jun 2019 20:38:10 GMT</pubDate>
            <atom:updated>2019-06-17T20:38:10.542Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5wWQoroIpglvQsC9" /><figcaption>Photo by <a href="https://unsplash.com/@o1234567890?utm_source=medium&amp;utm_medium=referral">Inactive.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The Universe is made of clusters of millions of galaxies of different shapes and sizes. The study of galaxies and their evolution plays an important role in understanding how the Universe works. The classification of galaxies it’s useful while studying how galaxies are formed, and how they evolve. Current surveys of the sky produce huge amounts of data, manually labeling galaxies’ morphological features can be time consuming, and prone to errors, so people are trying to come up with clever ways to tag these automatically. The goal of this work is to build a model that can predict the type of galaxy given an image in the visible spectrum.</p><p>The <a href="https://github.com/nunorc/galaxy-convnet">galaxy-convnet</a> GitHub repository includes notebooks for the creation of the dataset, and the training of the model with and without data augmentation, discussed in the next sections.</p><h4>The Dataset</h4><p>The goal is to be able to classify images in the visual spectrum. We are aiming at 3 types of classes only: spiral, elliptical and somewhere in-between (lenticular). The initial dataset is available from the<a href="https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge/overview"> Galaxy Zoo challenge</a> that contains a total of 61578 images. Each image is a galaxy, and for each image volunteers answered a set of questions to help identify special characteristics of the galaxy (e.g. spiral or disk features).</p><p>In order to train our classification model we need a set of images for each of the three classes of interest. So we devise a path in the decision tree to group images of the same class. For example, we consider elliptical galaxies, the images where 80% of the people considered the galaxy smooth (opposed to having features/disk), and that at least 40% considered the galaxy completely round. A similar approach was devised for the other classes, the following figure illustrates some examples for each type of galaxy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*GuDV0Y7Yat3MftI6JvKI_w.png" /><figcaption>Examples of galaxies for each class.</figcaption></figure><p>A Python notebook illustrating the creation of the dataset used is available in the <a href="https://github.com/nunorc/galaxy-convnet">galaxy-convnet</a> GitHub repository.</p><h4>The Model</h4><p>To build a model to predict the galaxy type we used a CNN, due to it’s usually good results for classification images tasks. The following figure illustrates the sequence of convolutional, pooling and fully connected layers that are composed together to build our model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/1*STxeD7OvWwkw0KE_Z0JwNA.png" /><figcaption>CNN to predict type of galaxy from a color 150x150 image.</figcaption></figure><p>We divided the dataset created in two sets: training and testing, using the test set to validate the model at each training epoch. The following plot shows the accuracy of the training and validation set when training the model for 100 epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xFUTS8AhSFAevd2vOXZhvg.png" /><figcaption>Model training and validation accuracy and loss function plots.</figcaption></figure><p>It seems to indicate that we are starting to have a bit of an over-fitting, we are approaching 100% accuracy in the training accuracy, while the accuracy in the validation set has stagnated around 85%. To try to overcame this problem we used a technique usually refereed to as <em>augmented data</em>, i.e. we generate batches of images by randomly applying some image transformation increasing the variance of the training set for each training epoch. The same plots as before are illustrated in the following figure but this time training the model using data augmentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IU3LqpfcHj-KJ2OkCVvuJA.png" /><figcaption>Model training and validation accuracy and loss function plots using augmented data.</figcaption></figure><p>From a visual analysis of the plots we see that when using augmented data the validation accuracy climbs steady with the training accuracy, giving an hint that the model it not over-fitting to the training data.</p><h3>Testing the Model</h3><p>Finally we do some testing of the model with some random images found online. Given an image the model outputs a list of 3 values, where each value is the probability of the galaxy to belong to the elliptical class, the lenticular class, or the spiral class respectively. The following images illustrate the model ability to predict the morphological class of images not seen in the training/test sets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*jVkR1md5FdHXMQz4rvuFIA.jpeg" /><figcaption>Model prediction: [1, 0, 0], which means an elliptical galaxy.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/1*slZ__5bxZb1IAjTnHtfdyg.png" /><figcaption>Model prediction: [0, 0, 1], which mean a spiral galaxy.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed5d7d82fbf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[World Cities Clustering Based on Places of Interest]]></title>
            <link>https://medium.com/@nunorc/world-cities-clustering-based-on-places-of-interest-d0d11bc92867?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0d11bc92867</guid>
            <category><![CDATA[k-means]]></category>
            <category><![CDATA[cluster-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Sat, 04 May 2019 15:54:17 GMT</pubDate>
            <atom:updated>2019-05-04T15:54:17.276Z</atom:updated>
            <content:encoded><![CDATA[<p>The goal of this analysis is to group cities from around the world, that offer similar trends of points of interest to their visitors. This kind of model can be useful for travel agencies devising targeted promotions for returning customers, or if you are planning your next vacations for example. The main intuition motivating this is: if you are a traveler and you enjoyed visiting a city there is a strong chance of also enjoying other cities that offer the same trends of points of interest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_RPq8sApbNj-RY4W" /><figcaption>Photo by <a href="https://unsplash.com/@serjosoza?utm_source=medium&amp;utm_medium=referral">sergio souza</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>The Data</h4><p>The initial data set is very straightforward, is simply a list of cities around the world. To make things more interesting and comprehensive we started with the <em>Euromonitor International</em> top 100 city destinations ranking fro 2015, which includes the top 100 leading cities in the world in terms of international tourist arrivals. Then, for each city we discover it’s geographical location: latitude and longitude. And finally, we retrieve information from the <a href="https://developer.foursquare.com/">Foursquare API</a> concerning the points of history (venues) around a 50 km radius of the city location. The final dataset is build by grouping the venues dataset by city, while adding the number of venues in each category. The following figure illustrates some rows in the dataset, and some of the categories (columns) that represent the venues for each city.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*ftIAwClViglnFoaryG1FRw.png" /><figcaption>First rows and some columns in the dataset.</figcaption></figure><h4>The Algorithm</h4><p>For clustering the cities we choose to use k-means, an unsupervised clustering algorithm based on minimizing the distance of every observation in the data set to the center of a initially defined number of clusters. One of the drawbacks of k-means is that you need to specify the number of clusters before hand. A common technique used to come up with a sensible number of this is the <em>elbow technique</em>, that basically consists running the algorithm with different numbers for k, and measure the inertia (a possible measure of quality) of the resulting clusters. When we plot this usually there is a clear change in the steepness of the line, that tells us that after that particular number of clusters the gain in information is significantly decreased. The following plot illustrates the computed inertia using a different number of clusters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*QtXA_g_-vE5D2xjQPRVtAQ.png" /><figcaption>Inertia per number of clusters k.</figcaption></figure><p>As we can see from the plot there is no clear <em>elbow</em> so we choose to use 8 clusters to group our data, it seems a sensible number to start, and it seems to be a slight change in the plot around that number.</p><h4>The Results</h4><p>The following world map figure illustrates the final results of running the k-means algorithm for 8 clusters, where each city marker is colored according to the assigned cluster, i.e. cities with the same color belong to the same cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*I91IcoO-5NOceryVtfvrbA.png" /><figcaption>World map with all the cities in the dataset colored by clusters.</figcaption></figure><p>The final world map in HTML is available <a href="https://nunorc.github.io/IBM-DS_Coursera_Capstone/">here</a>. A full Python notebook including all the analysis is available <a href="https://github.com/nunorc/IBM-DS_Coursera_Capstone/blob/master/Final-Notebook.ipynb">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0d11bc92867" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Pulsar Stars Using Neural Networks]]></title>
            <link>https://medium.com/@nunorc/predicting-pulsar-stars-using-neural-networks-ecb3a527336b?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/ecb3a527336b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[astronomy]]></category>
            <category><![CDATA[astrophysics]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Sat, 19 Jan 2019 15:26:48 GMT</pubDate>
            <atom:updated>2019-01-19T15:26:48.034Z</atom:updated>
            <content:encoded><![CDATA[<p>Neutron stars are dense and hot objects, they provide unique opportunities to study the laws of physics in extreme conditions. Pulsars are highly magnetized neutron stars that rotate very fast and yield detectable periodic strong radio emissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*IfOF_QNqAwFJY0tbU7UlZQ.jpeg" /><figcaption>Image Credit: European Southern Observatory (ESO).</figcaption></figure><p>Modern pulsar surveys produce large volumes of data, the process of manually labeling candidates is laborious and time consuming. Hence researchers are currently trying to study and come up with approaches for automatic identifying candidates.</p><h4>The Dataset</h4><p>The HTRU2 dataset describes a sample of pulsar candidates collected during the High Time Resolution Universe Survey. The dataset contains a total of 17 898 observations, where 1 639 are positive examples, and 16 259 are negative. For each candidate (observation) 8 continuous variables, and a<br>binary classification variable, are available:</p><ol><li>the mean of the integrated profile;</li><li>the standard deviation of the integrated profile;</li><li>the excess kurtosis of the integrated profile;</li><li>the skewness of the integrated profile;</li><li>the mean of the DM-SNR curve;</li><li>the standard deviation of the DM-SNR curve;</li><li>the excess kurtosis of the DM-SNR curve;</li><li>the skewness of the DM-SNR curve;</li><li>and, the target class, where the values used are 1 for candidates<br> identified positively as pulsars, and 0 otherwise.</li></ol><p>All the variables are available for every observation, Figure 1 illustrates a sample of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*CBBZS27FLU2T3pik3JVsuA.jpeg" /><figcaption>Figure 1: Dataset sample illustration.</figcaption></figure><p>More information about the dataset is available <a href="https://archive.ics.uci.edu/ml/datasets/HTRU2">here</a>.</p><h4>The Model</h4><p>The goal is to build a model that is able to identify a pulsar given a set of input features, we approach this as a classification problem. To implement the model we use a 2-layer neural network, illustrated in Figure 2, with a fully connected hidden network with 6 nodes using a RELU activation function, and output layer with a sigmoid activation function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZSKvmXpemI2PFXCzc6j2sQ.png" /><figcaption>Figure 2: two layer Artificial Neural Network for predicting pulsars.</figcaption></figure><p>Next we fit the model to the dataset, using an Adam optimizer, and a binary cross entropy loss function.</p><h4>Evaluation</h4><p>The goal of the model created in previous sections is to correctly predict the identification of a pulsar given the defined features. Given enough training data the model should capture the inherent patterns of the data that identify a pulsar, generic enough that can be used to identify new cases, never seen by the model or training process. The evaluation of the model helps measuring the ability of the model to correctly generalize to unseen data.</p><p>Since the two classes of observations on the dataset are unbalanced, i.e. the number of positive observations (~ 10%) is much smaller than the negative ones (~ 90%), measuring accuracy is not a good approach to evaluate the model. To get a better grasp of the model performance we take a look at the ROC curve, a plot where the x-axis yields the number of false positive rate, and the y-axis yields the true positive rate, illustrated in Figure 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*6OWsH1lsQ8tptQh-PQzu7Q.png" /><figcaption>Figure 3: Evaluating the model using a ROC curve, while predicting the classification on the test set.</figcaption></figure><p>An area of 1 under the line would represent a perfect classifier, i.e. all the observations were correctly predicted, an area of 0.5 represents a worthless classifier. In this case an area of 0.981 usually portrays a very good to excellent classifier.</p><p>The complete implementation of the model, including the evaluation and training stages discussed are available in a <a href="https://www.kaggle.com/">Kaggle </a>kernel <a href="https://www.kaggle.com/nunorc/predicting-pulsar-stars-using-a-neural-network">here</a>, the dataset was originally published <a href="https://www.kaggle.com/pavanraj159/predicting-a-pulsar-star">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ecb3a527336b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Goodbye Kepler, and thanks for all the fish]]></title>
            <link>https://medium.com/@nunorc/goodbye-kepler-and-thanks-for-all-the-fish-da0690a31763?source=rss-4ff11b0113ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/da0690a31763</guid>
            <category><![CDATA[astronomy]]></category>
            <dc:creator><![CDATA[Nuno Ramos Carvalho]]></dc:creator>
            <pubDate>Wed, 31 Oct 2018 19:53:03 GMT</pubDate>
            <atom:updated>2018-10-31T19:53:03.480Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Td3-Y-IYxpYjRQEFEq8UJg.jpeg" /><figcaption>Artist concept. Image credit: NASA</figcaption></figure><p>After nine years of hard work the planet hunter Kepler, a space telescope launched by NASA in 2009, is retiring. The space telescope ran out of fuel after completing 2 missions, providing a total of 678 GB of data.</p><p>Detecting planets outside our solar system directly is hard, because when compared to their hosting stars, planets are very faint light sources in the sky. So, most of the planets are detected indirectly. Kepler contributed greatly for our current view of the universe, because it has helped to show that planets around stars is a common thing, and that there is a great diversity of planets out there.</p><h4>Detecting Exoplanets by Transit</h4><p>Detection by transit is one of the major indirect methods used nowadays to discover exoplanets. In a nutshell, consists in analysing the brightness variation of the parent star when the planet crosses the light path of the star with respect to the observer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*yfAAiYvPvfLbte10AJ6HhA.gif" /><figcaption>The change in brightness caused by a planet transit. Image source: <a href="https://imgur.com/43f17Ke">https://imgur.com/43f17Ke</a></figcaption></figure><p>For this phenomenon to be detectable, the planetary orbital plane needs to be in line-of-sight with the observer’s point of view. Also, a single transit observation is not enough, more observations of this brightness variation are required. But planets may have long orbital periods along their hosting stars, which means you may have to wait a bit before having at least 2 ou 3 measurements.</p><p>The <a href="http://exoplanet.eu/">Extrasolar Planets Encyclopaedia</a> catalog includes a total of confirmed 3874 exoplanets, spread across 2892 planetary systems. Planet transit methods are clearly the most common methods for detecting exoplanets, as illustrated by the following plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*gZkWhUD12hQfLtgzCsu3UA.png" /><figcaption>Number of confirmed exoplanets by detection method.</figcaption></figure><p>From these, a total of 2327 confirmed exoplanets were discovered by Kepler observations. This emphasizes the contribution of this spacecraft to the current knowledge about planets (and their characteristics) outside our solar system. And there is still plenty of work to do will all the data gathered.</p><p>So, thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da0690a31763" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>