<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pippin Lee on Medium]]></title>
        <description><![CDATA[Stories by Pippin Lee on Medium]]></description>
        <link>https://medium.com/@pippinlee?source=rss-28ff78fed88e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*312pRa3Jh6ESE7Es.jpeg</url>
            <title>Stories by Pippin Lee on Medium</title>
            <link>https://medium.com/@pippinlee?source=rss-28ff78fed88e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:02:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pippinlee/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Finding Supernovae with XGBoost]]></title>
            <link>https://medium.com/space2vec/finding-supernovae-with-xgboost-3275f2639b06?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/3275f2639b06</guid>
            <category><![CDATA[supernova]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[astronomy]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[space]]></category>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Fri, 16 Feb 2018 20:29:53 GMT</pubDate>
            <atom:updated>2018-02-16T20:32:31.381Z</atom:updated>
            <content:encoded><![CDATA[<p>In our previous update, we explored different astronomy problems and their feasibility. With our limited knowledge in this space (pun totally intended) it made sense to try and pick an initial problem with metrics that were well defined. A large part of our curiosity with the space2vec project is asking whether modern techniques (think deep learning/neural nets) in machine learning can be useful in astronomy. So we looked for work done using classical methods that could be improved on, in hopes of having early comparisons to document.</p><blockquote><strong>Side Note: </strong>These comparisons are crucial for people learning to build their first model, how else would you know how well you are doing?</blockquote><p>We searched <a href="https://arxiv.org/">arxiv.org</a>, a website researchers use to post their work. During our search we found a paper that discusses building more efficient ways of detecting supernova (an exploding star), using images from the Dark Energy Survey. Titled <a href="https://arxiv.org/abs/1504.02936"><em>Automated Transient Identification in the Dark Energy Survey</em></a><em> (ATIDES),</em> the paper outlines a model for detecting supernovae from images. It explains that the current methods heavily depend on the human eye for detection of transient astronomical events.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/0*oAJtkoGBUOm1OxQa." /></figure><p><em>ATIDES</em> laments the labour intensive work of looking at images to discern if they do indeed contain supernova. The problem it points to is that this manual work does not scale well as datasets get larger over the next decade. The paper improves quite well on this notion, using a <a href="https://medium.com/@Synced/how-random-forest-algorithm-works-in-machine-learning-3c0fe15b6674">random forest</a>, a classical machine learning algorithm, to classify each image as either interesting (likely to be a supernova), or not (non-objects or artifacts).</p><h3>Overview of ATIDES Solution to Identifying Transients</h3><p>ATIDES code and data: <a href="http://portal.nersc.gov/project/dessn/autoscan/">http://portal.nersc.gov/project/dessn/autoscan/</a></p><ul><li>Features created (manually): 38</li><li>Images: 898,963</li><li>Images of supernovae: 454,092</li><li>Images not of supernovae: 444,871</li><li>Images and FITS files can be found under “Images”</li><li>Feature engineered CSV can be found under “Features”</li></ul><blockquote><strong>Side-note</strong>: FITS files are a structured means of storing data along with metadata (data about the data) such as date and time of the information gathered.</blockquote><p>Let’s first try to understand the paper and its metrics so we know what we’re hoping to beat. The central dataset in the paper is the first release of data from the <em>Dark Energy Survey Supernova program (DES-SN)</em> paired with hand-crafted data<em>.</em></p><blockquote><strong>Side-note: </strong>Hand-crafting data, often referred to as feature engineering, is the process of looking at the data available and converting it into data that might be easier to feed into a model or let the model pick up on. In this case, the researchers took the images and extracted information such that the model would be able to pick up on certain things it wouldn’t be able to before.</blockquote><p><em>“The Dark Energy Survey (DES) is an international, collaborative effort to map hundreds of millions of galaxies, detect thousands of supernovae, and find patterns of cosmic structure that will reveal the nature of the mysterious dark energy that is accelerating the expansion of our Universe. DES began searching the Southern skies on August 31, 2013.”</em></p><p>Source: <a href="https://www.darkenergysurvey.org/the-des-project/overview/">https://www.darkenergysurvey.org/the-des-project/overview/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vEXCKoU29hxaVpUO." /><figcaption>Cerro Tololo Inter-American Observatory, high in the Chilean Andes. <a href="https://www.darkenergysurvey.org/">Source</a></figcaption></figure><p>The dataset contains 444,871 images of human-verified artifacts, and 454,092 simulated transient events projected onto real images of galaxies. Essentially half the dataset contains manufactured images of supernovae, which seems weird, but <em>data augmentation</em> has become a popular technique for bettering a model. Data — both in quality and quantity — is a big part of the equation when considering the accuracy of a model.</p><blockquote><strong>Side-note:</strong> Data augmentation has been explored quite a bit, from simply changing the data that you have, to trying to create new data based on what you have (<a href="https://hackernoon.com/how-do-gans-intuitively-work-2dda07f247a1">GANs</a> are a popular new way of doing this). This is seen in the image recognition field; if you have 10,000 colour images of cats and get the same 10,000 images in black and white, your dataset is now 20,000. Your model should be able to understand what a cat is, even if the colour is different.</blockquote><p>Supernovae are rare, so there’s a lack of supernova images that can be used for training. <em>ATIDES</em> make use of the few images of supernovae that they have, and overlay them on different galaxies. By using many different backgrounds (essentially adding unique space noise) with each supernova picture, researchers were able to build a strong dataset for their model.</p><p><strong>When data is sparse, is it a good idea to use “simulated images”?</strong></p><p>Yes — to a certain extent. When the data is lacking, some data is better than no data. It’s worth understanding that the model we build can only be as good as the data. So if half the dataset includes simulated images we should keep that in mind when doing production<em> inference</em>.</p><blockquote><strong>Side-note: </strong>Inference is when we feed data through our model without updating it. This is used to see how the current model is doing without having it learn the data that is used (an important part of neural nets that will be talked about soon).</blockquote><p>With the knowledge that good data and model go hand-in-hand, we could see a huge improvement happening if we spent time creating higher quality simulated data. As we mentioned in a previous side-note, GANs could be helpful here.</p><p><strong>Our goal is differentiating between images containing artifacts and those containing transient objects (eg: supernova).</strong></p><p>In total, the survey will cover 1/8th the entire sky using a telescope at the <a href="http://www.ctio.noao.edu/noao/">Cerro Tololo Inter-American Observatory</a> in Chile. The ATIDES paper uses a subset of this data and focuses on the supernova aspects of the Dark Energy Survey, referred to as DES-SN.</p><p>The collection of algorithms used in ATIDES is referred to as <a href="http://portal.nersc.gov/project/dessn/autoscan/">Autoscan</a>, a Python package for artifact rejection.</p><p>The paper describes how with the help of astronomers they were able to <a href="https://en.wikipedia.org/wiki/Feature_engineering">feature engineer</a> 38 noteworthy characteristics of images.</p><p>ATIDES uses feature engineering since the data is in image format and thus needs to be quantifiable in order to run within a random forest algorithm.</p><p>Here’s what a small sample of the images looks like once they’ve been converted into a format that’s easier to do data analysis on (feature engineered). Each row is an image, and each column is a feature (only 14 of 38 features shown).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w5Ib8QvBjjy8NgDX." /><figcaption><strong>First 10 Rows of the Feature Engineered Data</strong></figcaption></figure><p>You can see an <strong>ID</strong> column which is how we know which object the data-point refers to and an <strong>OBJECT_TYPE</strong> column which is a binary indicator of whether the data-point contains a supernova (0) or not (1). In data science the column that you train your model on is called the target. In this case <strong>OBJECT_TYPE </strong>is our target because it tells our model what the correct answer is in the question “is this a supernova or not?”.</p><p>We have to make sure that we take both of these columns out of the data that gets fed into the model (something that is shown in our code). The feature names aren’t too user friendly but the paper explains most of them. We did note that we found some naming convention issues between the data set and paper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NbSCRsq2SWn7lzza." /><figcaption><strong>Autoscan’s Feature Library</strong></figcaption></figure><p>With respect to exploring other model approaches, the paper also attempts using <a href="http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf">Support Vector Machines</a> and <a href="https://en.wikipedia.org/wiki/AdaBoost">Adaboosted decision trees</a> but random forest are found to have the best accuracy. The random forest algorithm used in ATIDES is from the popular <a href="http://scikit-learn.org/stable/">scikit-learn</a> Python library (which makes a lot of things very easy and accessible).</p><blockquote>Side-note: scikit-learn is the swiss army knife of data science. For running off-the-shelf models it’s will make your life much nicer. It’s a great starting point, but also a standard for quickly getting common classical machine learning models like logistic regression.</blockquote><p>How does random forest work? At a high level it creates a set of <a href="https://www.quora.com/What-is-an-intuitive-explanation-of-a-decision-tree">decision trees</a> from the data and classifies the image based on a ‘vote’ among the decision trees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MHXfDV7s_CHW9uEF." /><figcaption><strong>Helpful Visualization for a Random Forest</strong></figcaption></figure><p>The random forest algorithm produces weightings for its features. For example in Autoscan, <em>r_aper_psf</em> was considered most important. This is useful because it informs the astronomer, whose domain knowledge supplied the features, which features are most relevant to identifying an object of interest.</p><p>Given this, it’s neat to think of machine learning helping us see things we cannot see even if we were experienced astronomers — much like how x-rays machines help doctors see and understand things that normally wouldn’t be possible.</p><h3><strong>Understanding The Target and Metrics</strong></h3><p>We spent a good amount of time understanding how the paper presents its results, which had a lot to do with how they classify results using the below confusion matrix.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/0*Rdw8cHF1be9-Dbst." /></figure><p>The random forest’s performance was evaluated on the missed detection rate (MDR) and the false positive rate (FPR).</p><p><strong>MDR:</strong> represents the amount of objects that the model missed; where the model said “this is not a supernova” but it actually was.</p><p><strong>FPR:</strong> represents the amount of objects that the model falsely labeled as a supernova; where the model said “this is a supernova” but it really wasn’t.</p><p>Generally, if an algorithm is more conservative about identifying images as transients, it will miss more transients (higher MDR), but register fewer artifacts incorrectly as transients (lower FPR). It is preferable to have both low MDR and FPR. Both are defined below (with <strong>F</strong> being “false”,<strong> T</strong> being “true, <strong>p</strong> being “positive”, and <strong>n</strong> being “negative”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/0*zDagP9w6hsTT32mm." /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/0*I83xSZ0aXpGN9EAJ." /><figcaption><strong>Definition of MDR and FPR</strong></figcaption></figure><p>This gets a little tricky when we introduce the <em>threshold</em>. To explain how the threshold is used here we have to explain what the model outputs. Since we’re trying to see if a supernova is present (0/positive) or is not present (1/negative), the model output should be something similar. However, we want to know <strong><em>how</em></strong><em> </em>sure the model is — this means we output a probability (or a number that can be used similarly), that is a number between 0 or 1. The thresholds used in the paper’s figure below (0.4, 0.5, and 0.6) define <strong><em>how</em></strong> sure the model needs to be for us to consider it’s confidence as a decision.</p><p>A helpful way to understand how these two rates are related is try out this interactive ROC curve explanation: <a href="http://www.navan.name/roc/">http://www.navan.name/roc/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/0*t5ni33XhI0cDVm5V." /></figure><p>This figure represents <em>ATIDES</em> results for all three thresholds. This ideally means a curve on the inside (left) of their curve.</p><h3>Our Approach and Baseline Model</h3><p>When building a model, the most important first step is the data. We spent a good amount of time exploring and getting comfortable with the data to get an understanding of what is described and how.</p><p>The next step was to build a baseline model. What does baseline mean? A model that is simple or naive — that can act as a bottom bar that our actual model should be able to beat. It’s a model that allows us to quickly check that our setup is working and ensure data is being properly imported.</p><p>We have done this a few ways, and although our baseline uses the feature engineered data (which we want to remove the need for in the future), it was helpful to check that our development environment was working and we could do something competitive.</p><p>Ok — a quick tease, the ATIDES paper mentions that further exploration could be done using Convolutional Neural Nets (CNNs), which would allow skipping the manual feature engineering steps needed in their process. A valid critique ATIDES adds is that CNNs (and neural networks in general) do suffer from a lack of transparency of what the model is doing. While we won’t be discussing CNNs here, it will take our entire next update.</p><p>We decided for our baseline model that we would use the feature engineered data from ATIDES<em> </em>and see how XGBoost would perform compared to the paper’s use of random forest. XGBoost is a common algorithm usually used for regression and classification problems. It uses the concept of gradient boosting on-top of decision trees. If you’re looking for a more in-depth info of the inner working of XGBoost you can can check out <a href="http://xgboost.readthedocs.io/en/latest/model.html.">http://xgboost.readthedocs.io/en/latest/model.html.</a> XGBoost is often used in data science competitions and is really quick to get up-and-running.</p><p>First we needed to set up an environment that could handle a reasonable amount of compute.</p><h3><strong>Development Environment</strong></h3><p>We didn’t do much initial assessment of different compute systems since we were already very comfortable using Amazon’s Web Services (AWS).</p><p>Our initial needs were:</p><ul><li>Quickly explore the data</li><li>Use jupyter notebooks, Tensorflow, or Keras</li><li>Handle anywhere from ~2.5–50GB of data</li></ul><p>We spun up a <a href="https://aws.amazon.com/ec2/instance-types/p2/">P2 AWS instance</a>, using the <a href="https://aws.amazon.com/marketplace/pp/B077GCH38C">Deep Learning AMI</a>. This gives a bunch of tools out of the box like PyTorch, Keras, and Tensorflow and some half decent GPUs to do the grunt work. Lots of options!</p><blockquote><strong>Side-note:</strong> In the name of education we’ve open sourced all our code! We talk about this in the <strong>Code</strong> section.</blockquote><h3><strong>XG-Boost VS Random Forest: detecting supernovae</strong></h3><p>Finally, let’s talk about our super awesome (and totally, really complicated to run) first model!</p><p>If you look at the code you can see how simple it is to get something like this up and running. Even though this was a quick baseline model, it was used for a few reasons:</p><ol><li>We had spent a <em>ton</em> of time defining and understanding the problem, which made us long for actually getting something up and running</li><li>It forced us to familiarize ourselves with the actual data rather than just the problem</li><li>We can have some simple baseline metrics to compare our future models against</li></ol><p>With that said we can now look at our baseline model!</p><p>We’ve highlighted the winning score in the table below in green. Although it would have been just as easy to say we lost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*ieLVFNuYLpvoSU-9." /><figcaption>Our XGBoost model VS paper’s random forest</figcaption></figure><p>This gives us quite a bit of insight and helps us validate some assumptions that we had:</p><ol><li>The feature engineered data is good and holds a lot of predictive power</li><li>Newer techniques (XGBoost in this case) are fairly competitive to certain problems with minor tuning</li><li>This is going to require more work than an out-of-the-box solution</li></ol><h3><strong>Code</strong></h3><p>We keep talking about making this a helpful tool for beginners and that’s great let’s do something that is actually useful to learn the <em>coding</em> part of this.</p><p>To do this, we will open source<em> everything </em>that ends up making up the completed codebase. We have created a GitHub repo to host everything that you can use.</p><p>Sadly, we won’t be going into how to use these tools if you aren’t familiar with them… What we do have time for is giving a list of tools to Google:</p><ul><li>GitHub</li><li>Python</li><li>Pip/Conda</li><li>iPython/Jupyter</li></ul><p>The list of Python libraries can be found in the “requirements.txt” file in the repository.</p><p><a href="https://github.com/pippinlee/space2vec-ml-code">https://github.com/pippinlee/space2vec-ml-code</a></p><p>Data in a pickle file can also be found here: <a href="https://drive.google.com/open?id=1Pa4-imVbK7yfZuCX3mfF-mMae1eyhQqo">https://drive.google.com/open?id=1Pa4-imVbK7yfZuCX3mfF-mMae1eyhQqo</a></p><h3><strong>Conclusion</strong></h3><p>These past two weeks were all about understanding the data and the problem. We looked at the paper’s random forest model and feature engineering to classify supernovae.</p><p>We looked at understanding the paper’s metrics and how they achieved them, spinning up a P2 instance on AWS to quickly get model building and developing a half-decent baseline model.</p><p>In our next post our focus will be on using a modern machine learning technique that avoids this labour intensive process of defining features beforehand, and instead feeding images directly into the model with a Convolutional Neural Net.</p><p><strong><em>You can follow our work here: </em></strong><a href="https://twitter.com/space2vec"><strong>https://twitter.com/space2vec</strong></a></p><h3><strong>Resources</strong></h3><p>Understanding XGBoost <a href="https://machinelearningmastery.com/gentle-introduction-xgboost-applied-machine-learning/">https://machinelearningmastery.com/gentle-introduction-xgboost-applied-machine-learning/</a></p><p><a href="http://blog.kaggle.com/2017/01/23/a-kaggle-master-explains-gradient-boosting/">http://blog.kaggle.com/2017/01/23/a-kaggle-master-explains-gradient-boosting/</a></p><p><a href="http://colah.github.io/posts/2014-07-Conv-Nets-Modular/">http://colah.github.io/posts/2014-07-Conv-Nets-Modular/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3275f2639b06" width="1" height="1" alt=""><hr><p><a href="https://medium.com/space2vec/finding-supernovae-with-xgboost-3275f2639b06">Finding Supernovae with XGBoost</a> was originally published in <a href="https://medium.com/space2vec">space2vec</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting space2vec: using machine learning in space]]></title>
            <link>https://medium.com/space2vec/starting-space2vec-using-machine-learning-in-space-9640680ad516?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9640680ad516</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[astronomy]]></category>
            <category><![CDATA[space]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Sun, 11 Feb 2018 19:45:07 GMT</pubDate>
            <atom:updated>2018-02-11T19:45:07.788Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The goal of these updates is to document our research into applying modern machine learning techniques in astronomy. We’ll write about what works and also what doesn’t. We write software by trade so these posts should hopefully give insight into HOW modern astronomy and machine learning works as we better understand them ourselves.</em></p><p><em>If you have any questions about our work we’d love to hear from you: pl@deeplearni.ng</em></p><h3>How It Started</h3><p>“Clustering the entire goddamn universe”.</p><p>It was early July this past summer when Cole first started flying these words around the DeepLearni.ng office. We were discussing dream projects to work on within the field of <a href="http://neuralnetworksanddeeplearning.com/">deep learning</a>, and a quick survey of the room confirmed our initial suspicions… space is pretty freaking awesome — but how would deep learning techniques actually work in a field with so much history and tradition behind their data analysis toolkit?</p><p>This question led us to spend the next few months diving deep into the space field, and talking with astronomers to see if there may be a use for <strong>any</strong>machine learning in astronomy. After all — astronomy is a data problem and data is what drives machine learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*SPLO8QoamOHVjeLFXKLz7g.gif" /><figcaption>Lightcurves as seen from Earth</figcaption></figure><h4>What Are We Doing?</h4><p>The aim of this blog will be to document our progress — unfiltered stories, code, data — and how machine learning and astronomy can work together to better understand the skies above us. It will hopefully act as a guide and motivation on how <strong>you</strong> can travel down a similar path too.</p><h4>About Us</h4><p>We come from various backgrounds (computer science, web development, pure mathematics) but are extremely passionate about two things:</p><p>1. Space: It’s amazing and we love it. Space forces us to ask big questions, leaving us with a child-like curiosity. We don’t have a formal background in astronomy but we ❤ space and part of the process will be learning about the science of our skies.</p><p>2. Cutting through the hype: Neural networks don’t solve all data science problems. If our research shows that neural networks can’t be easily applied in astronomy, we’ll document it. We think it’s important to show the value of different applications and datasets — and that also means covering where things don’t work.</p><h4>Our Initial Research</h4><p>Although we had initial ideas of problems to tackle, such as using newer techniques from the academic world to “cluster the entire goddamn universe”, we knew we had to be practical. This forced us to approach our research with a question:</p><p><strong>Can we find a <em>real, annoying</em> problem in the astronomy community where we can help by applying modern machine learning techniques?</strong></p><p>As we began our research, we were very aware of our lack of astronomy knowledge, but to our delight the astronomers we spoke with were all wonderfully welcoming (the conversations will be an entire post themselves!). We also knew our depth of understanding would be shallow at best — therefore it was important to have astronomers explain quite a few technical aspects of astronomy to make sure we didn’t get stuck at 100 level questions. We started with the following:</p><h4>What tools are modern astronomers using (i.e. computer applications, programming languages)?</h4><p>We were amazed to find that a large portion of the astronomy community has followed the open science movement and is using open source technologies — this means we’re right at home with the many open source Python libraries available. We’ll be publicly posting all these resources in a separate post as we go forward.</p><p>We also found a super friendly community of astronomers eager to discuss how the whole astronomy field has changed over the last dozen years or so. In short, the data and discoveries have grown exponentially! Just look at exoplanet discoveries over the last 50 years (<a href="https://www.lsst.org/">and it’s about to get even wilder</a>!)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*Z-_d3zP6uwnX0TjOzq4yOA.gif" /><figcaption>Discovery of exoplanets over time</figcaption></figure><h4>Data is one of the biggest pieces in machine learning problems, so how is the data?</h4><p>Astronomy is quickly becoming a big data problem. Projects like the <a href="https://www.lsst.org/">LSST</a> will produce 15–30 terabytes of data a night. This is nuts, <strong>i</strong>, and immediately stuck out to us — especially as part of our work on DeepLearni.ng’s upcoming product <a href="http://deeplearni.ng/frontiers/">Frontiers</a>, is trying to perfect massive data pipelines.</p><p>We also quickly learned the common ways that exoplanets (planets outside of Earth’s solar system) are discovered using data from different satellites. We’ll have another post talking about this subject later on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*579QYUQ9-9fEIVx4LcbTvA.gif" /><figcaption>What does satellite data look like? This is an example of K2’s data</figcaption></figure><h4>Are astronomers using machine learning?</h4><p><strong>Yes</strong>, but not much. Astronomy has traditionally had a human component to identify objects. One of the more interesting projects we found was Galaxy Zoo which crowdsources identification of galaxies from the <a href="http://www.sdss.org/">Sloan Digital Sky Survey</a>. Most examples of machine learning tend to be on the classical side, using techniques such as K-means clustering or <a href="https://en.wikipedia.org/wiki/Logistic_regression">logistic</a> <a href="https://en.wikipedia.org/wiki/Logistic_regression">regression</a> (technical stuff that will be in a post about our models).</p><h4>Are astronomers using modern techniques like neural nets?</h4><p><strong>Not really.</strong> For instance a quick look for mentions of “neural nets” or “deep learning” in relation to astronomy on <a href="https://arxiv.org/">arxiv.org</a> doesn’t come up with more than a few results. After speaking to a Machine Learning PhD turned astronomer from the University of Toronto, we learned there’s a reason for this — you can’t just throw a standard model from scikit-learn at the available data, you need to fundamentally understand what you are doing on both the astronomy and modeling side. This is where we’re hoping to shed the most light since this is how we operate as a company — make sure we understand the problem from many different perspectives, then start building a custom model that uses these insights. We will always try to come back to this point, <strong>why can’t you just throw some out-of-the-box model at it?</strong></p><h4>Deciding on a Problem</h4><p>During our research we found this paper: <a href="https://arxiv.org/abs/1504.02936">Automated Transient Identification in the Dark Energy Survey</a>.</p><p>It outlines using a machine learning model (random forest) with data from the Dark Energy Survey Supernova program (DES-SN) to make identifying transients more efficient and accurate. It provided data, code, and good documentation of the model they built. We decided it’d be a good initial challenge to see if we can beat the paper’s model with a deep neural network.</p><p>Our next post will talk more about the task-at-hand, our plan for approaching the problem, as well as model architectures. Stay tuned!</p><p>We’d like to give a big thank you to <a href="https://twitter.com/angryastropanda?lang=en">Nathalie Ouellette</a>, <a href="https://twitter.com/dstndstn">Dustin Lang</a>, <a href="https://twitter.com/exoplaneteer?lang=en">Dan Foreman-Mackey</a>, and <a href="https://twitter.com/rossfadely?lang=en">Ross Fadely</a> for taking their time to answer our beginner questions and giving valuable input to getting this project going!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9640680ad516" width="1" height="1" alt=""><hr><p><a href="https://medium.com/space2vec/starting-space2vec-using-machine-learning-in-space-9640680ad516">Starting space2vec: using machine learning in space</a> was originally published in <a href="https://medium.com/space2vec">space2vec</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Shipping a Fellow]]></title>
            <link>https://medium.com/the-open-journalism-project/shipping-a-fellow-47fb10fa83de?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/47fb10fa83de</guid>
            <category><![CDATA[code]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[medium]]></category>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Wed, 26 Aug 2015 19:15:42 GMT</pubDate>
            <atom:updated>2015-08-26T19:39:06.989Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*9WeZla87KROqYyIGElWV3A.png" /></figure><p>Good news — today the <a href="http://openjournalismproject.com/">Open Journalism Project</a> is announcing the start of a Fellowship program.</p><p>A few months ago when Asad and I <a href="https://medium.com/@pippinlee/the-open-journalism-project-better-student-journalism-fb39f4f701bb">wrote about the experience and research</a> we’d done in understanding problems that student newsrooms are facing — we got many thoughtful responses. We were able to talk to students quite literally scattered across the globe, from Pakistan and India, to small-town USA, and big cities in Europe. These were students that shared our frustration with the lack of opportunities at their universities to tinker at the intersection of code and journalism.</p><p>While we’ve written, talked, and advised students newsrooms — <strong>we’ve been lacking in our methodology</strong>. At best we’re able to recommend similar setups we’ve seen at professional newsrooms. We haven’t been able to clearly point to what has (and hasn’t) worked<em>. </em>We can’t share <em>how</em> we got a student newsroom curious about programming or obsessing over data<em>. </em>In part this is because nothing like this exists in Canada — we’re carving out a new and important path in journalism education. Today we’re announcing our fellowship program, something that is very much an experiment — but we believe if we do it in an open way, it will help show us and everyone <em>what</em> student newsrooms can be.</p><p>From our research:</p><blockquote><strong>We don’t know what we don’t know</strong></blockquote><blockquote>We spent much of the rest of the school year asking “what should we be doing in the newsroom?”, which mainly led us to ask “how do we use digital platforms to tell stories?” It was a straightforward question that led to many more questions. Out in the real world, traditional journalists were struggling to keep their jobs in a dying print world. They wore the same design of shoes that we were supposed to fill. Being pushed to repeat old, failing strategies and blocked from trying something new scared us.</blockquote><h3>Why?</h3><p>We’re kickstarting the process of getting Canadian student newsrooms thinking about the web. The Open Journalism Fellowships will seek students with strong computer science, web, data, or interactive skills by funding them as fellows to work at their campus student newsroom. We’re asking fellows to play a critical role as a colleague, mentor, and coder — but we know they’re up to the challenge. Our model is heavily inspired by our friends at the <a href="http://knightlab.northwestern.edu/">Knight Lab at Northwestern University</a>, as well as the <a href="https://opennews.org/what/fellowships/">Knight Mozilla OpenNews Fellowship</a>.</p><p>We’re focused on student newsrooms because that’s the part of the ecosystem we know best, and also the part where we can have the most influence. We believe student newsrooms are the perfect place to experiment because of how much flexibility and control over the process students have. That said, for this project to be successful we realized we’d also need to have the perspective of the professional newsroom, so we’ve brought on <a href="https://twitter.com/juruwolfe">Julia Wolfe</a>, news developer and designer for the Globe and Mail, and <a href="https://twitter.com/timalamenciak">Tim Alamenciak</a>, from TVO, to keep fellowships in line with what professional newsrooms are doing.</p><h3>Who’s involved</h3><p>For this first year we’ve partnered with the <a href="http://queensjournal.ca/">Queen’s Journal</a>, at Queen’s University, which is being helmed by editors-in-chief Anisa Rawhani and Sebastian Leck. Our first Open Journalism fellow will be Max Bittker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*szZIQn9jdAq43umHzPnaNg.jpeg" /><figcaption>Max Bittker (photo: Megan Gamble)</figcaption></figure><p><a href="https://twitter.com/maxbittker">Max</a> is a developer interested in digital empowerment and delight, Twitter bots, generative worlds, roasted vegetables, and neural networks. He’s a fourth-year computer engineering student at Queen’s University hailing from Rochester, New York. He looks forward to being the first Open Journalism Fellow and building skills, bridges, and weird stuff with the Queen’s Journal. You can find him here: <a href="http://maxbittker.github.io/">maxbittker.github.io</a>.</p><p>In addition to The Journal, we’ve brought together UBC’s <a href="http://ubyssey.ca/">Ubyssey</a>, the <a href="http://mcgilltribune.com/">McGill Tribune</a>, uOttawa’s <a href="http://thefulcrum.ca/">The Fulcrum</a>, and UofT’s <a href="http://thevarsity.ca/">The Varsity</a> to help build a strong community of students experimenting with telling stories on the web.</p><p>Our decision to make our first year of the fellowship program small is with the understanding that if we can work towards a model that fits student newsrooms, we’d be happy to expand in future years to more Canadian campuses.</p><h3>Collaborators</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*CwoYV9JXgobrjV4smGQRrA.png" /><figcaption>Aniqah Mair (photo: Khalil Mair)</figcaption></figure><p>We’re also fortunate to have <a href="https://github.com/aniqah">Aniqah Mair</a> and <a href="https://github.com/jpittis">Jake Pittis</a>, both second year Computer Science students who will be collaborators with the project at Queen’s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*g6ix2lWw7vqZMo8UZX2E7Q.png" /><figcaption>Jake Pittis (photo: Hack Western)</figcaption></figure><p>Both Aniqah and Jake have backgrounds in creating friendly environments for learning and programming — key pieces to the workshops, events, and mentorship we’re planning for the Queen’s Journal for the upcoming school year.</p><h3>Goals and requirements</h3><p>We understand that as the Fellowships progress we’ll learn how to make the experience for both the student newsrooms and fellows better, so there may be adjustments to our requirements.</p><p><strong>Requirements for the fellow:</strong></p><ul><li>10 hours per week in the newsroom</li><li>Plan in collaboration with the newsroom + community for media week</li><li>Stipend + additional support around hardware, software, and mentoring</li><li>Running workshops locally to increase computer literacy</li><li>Helping us design and build curriculum as open-sourced manuals, which includes documenting process of “going-digital” in blog posts</li><li>Run for a full year with the last month budgeted for transition and reflection</li></ul><p><strong>What daily life looks like for fellows:</strong></p><ul><li>Treated as journalists</li><li>Have access to meetings involving story planning</li><li>Work with section heads/digital-editor to develop and showcase web-worthy stories</li><li>Commit/publish/document code to Github weekly</li></ul><p>We’ll have frequent updates here at The Open Journalism Project publication <a href="https://medium.com/the-open-journalism-project">https://medium.com/the-open-journalism-project</a> to publicly discuss what works and doesn’t.</p><p>If you’d like to be part of the conversation, or talk, drop me a note at <a href="/service/mailto:pippin@pippinlee.com?Subject=Open%20Journalism">pippin@pippinlee.com</a> or ping us on Twitter <a href="https://twitter.com/openjo/">@OpenJo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47fb10fa83de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-open-journalism-project/shipping-a-fellow-47fb10fa83de">Shipping a Fellow</a> was originally published in <a href="https://medium.com/the-open-journalism-project">The Open Journalism Project</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Thoughts from making tools for literature]]></title>
            <link>https://medium.com/codex-hack/will-the-future-of-writing-be-more-like-software-49f2a6900ca9?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/49f2a6900ca9</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[publishing]]></category>
            <category><![CDATA[writing]]></category>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Fri, 10 Jul 2015 02:18:02 GMT</pubDate>
            <atom:updated>2015-07-10T18:50:06.518Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5moklcdrPhG6WmN-KC3sLw.png" /></figure><h3>How Literature and Code Overlap</h3><p>When we think about languages and how they evolve, it’s usually at the needs of the people that use them. If we want to share a bit of information, whether that be an emotion, or description — we can create a new collection of letters to form a new word. As long as another person has a common understanding of this combination of letters, we can now use our new word in <a href="http://number27.org/wb-language">sharing information from one mind to another</a>. Languages can be pretty neat this way, and can get even more descriptive if we talk about emojis.</p><p>Software, on the other hand, has overhead and accessibility issues. Sure, it also allows you to invent new meanings, and even create new languages — but currently it only evolves when you tell the machine to do something specific, and with correct details. This makes it tough for most to get involved and interested in seeing how the web, and computers will shape publishing going forward.</p><p><a href="https://twitter.com/maxbittker">Max Bittker</a> and I were lucky enough to spend the last weekend in June on the West Coast working along side folks who also care about a number of these issues at <a href="http://codexhackathon.com/">CODEX Hackathon</a>. CODEX billed itself as an event to play with the future of publishing, but was really just a good excuse for smart designers, developers, and publishers to come together to build things that asked an important question of the direction of reading.</p><p>So over that very frenetic weekend, we built <a href="http://forkpad.me/">forkpad.me</a> — a service that allows the remixing of stories on the popular writing site <a href="https://www.wattpad.com/home">Wattpad</a> — to help their community become better writers, and keep the web weird. Along the way we realized that the process of writing could take inspiration from the open source software development process. This idea lead us to asking these three questions, which helped shape the tool we wanted to build:</p><ul><li><em>What happens when writers are able to remix and reuse each others work?</em></li><li><em>What happens when a story is never “finished”?</em></li><li><em>What happens when a story can have multiple authors?</em></li></ul><p>Intrigued by this, Max and I asked ourselves how we could make a tool that could bring key aspects of Open Source Software (OSS) development to the writing and publishing process.</p><h3>Informality Will Rule Publishing</h3><p>One of the things Max and I love about the web is its ability to share information easily. Thanks to all the underlying technical protocols, we’re now able to build tools to reach many. When we showed up to <a href="http://codeforamerica.org/">Code for America</a>’s San Francisco office Saturday morning, we knew the community on Wattpad was something we could have a lot of fun with.</p><p>We were given <a href="http://www.wattpad.com/codex/">early access to the API</a> for Wattpad, a massive serial writing community <a href="https://www.wattpad.com/about">with over 40 million users</a> who collectively spend more than 900 million years on the Wattpad site each month (85% of it on mobile).</p><p>In the time you’ll spend reading this article, Wattpad will receive enough werewolf fan fiction to last the rest of your life, in addition to other fan fiction of all niches and all personalities. I personally recommend checking out the entire <a href="http://www.wattpad.com/story/30644165-50-shades-of-drake-drake-fanfic-series">50 Shades of Drake</a> volume, fan fiction about the rapper, if you’re curious.</p><p>As we explored the number of story categories and forums within Wattpad we found prolific communities sharing the wildest parts of their minds through stories. By far the biggest success within fan fiction (and informal publishing) is 50 Shades of Grey, which <a href="https://en.wikipedia.org/wiki/Fifty_Shades_of_Grey#Background">started as Twilight fan fiction</a>.</p><p>So we began thinking about the web and its impact on publishing. Traditionally, the final product would be a book in the local store, but what <em>is</em> the “final product” when the process to publishing is just a ≤button/&gt; on a page?</p><h3>The End of a Story is No Longer Static</h3><p>Traditionally a book would go through a publisher’s Book Product™ machine — but the web now allows the final product to be something different. The final product for an online book is no longer a function of traditional publishing’s Book Product™ formula, but something much less refined. A novella can be quickly written and published, and then evolve with reader interest to spawn a whole series of stories.</p><p>As we began to work with Wattpad’s API, we took a closer look at the niches that existed within the story comments and forums on Wattpad. We saw authors discussing ideas, and adding thoughts in hopes of pushing each other to create new narratives. This public process means that the end product isn’t static <em>thing</em> — and because the web allows a story to be edited and rewritten, every story has the ability to be dynamic and changing. Stories may end, but the process for many books is always evolving — which is especially true on Wattpad.</p><h3>CODEX Hackathon on Twitter</h3><p>Ambient lighting. Gives @pippinlee a nice glow. He worked on Forkpad with Max. pic.twitter.com/ODK8B9ntdo</p><h3>Can Remixing Help the Writing Process?</h3><p>From our quick study of Wattpad, Max and I realized that this community could likely use a tool to help create a better feedback loop for authors working through story ideas. The idea of the “remix loop”, where creators spin off each others work can be incredible powerful tools within the writing process. Forkpad allows writers to take a part (“chapter”) from a Wattpad story and adapt it to their own liking.</p><p>When developing Forkpad, we found the concept of being able to change an ending or adapt characters to follow our own ideas a fun and important tool in the writing process. The ability to “fork” a Wattpad story means that others can post a link to their Forkpad version back on Wattpad to better share their thoughts on a story, as well as build off of each others’ work.</p><p>This idea of “forking” code is now common in software development. When computers were giant room-sized machines, the people who operated them (usually academics) would use tape as a medium to “write” their programs on. They would then feed these lengths of tape into a reader which would interpret their programs and run them on the computer. The programs were usually relatively small because computers could only process small bits of information. The idea of open source can be traced back to these early computer users, who stored their tapes in a drawer so that anyone using the computer could use their completed code. This saved colleagues from having to write a similar program from scratch, because they could share their completed tape programs.</p><p>Despite a period of highly commercialized and proprietary software during the 1980s and 1990s (a shift catalyzed, in part, <a href="https://en.wikipedia.org/wiki/Open_Letter_to_Hobbyists">by Microsoft</a>), developers’ natural desire to share their code has roared back to life in the last twenty years (embodied by <a href="https://en.wikipedia.org/wiki/Linus_Torvalds">Linus Torvalds</a> and his software versioning system <a href="https://en.wikipedia.org/wiki/Git_(software)">Git</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YFy69Cx6aidKv8BrMit7VA.png" /><figcaption>Early computers would run programs that were created by punching holes in tape. Photo: wikimedia</figcaption></figure><p>Building off each other’s work is common in the open source world of software, where a team may spend a year solving a specific problem, and then they’ll share the code with other developers so that others don’t have to waste their time solving a problem that’s already been solved. Starting with an empty text box is a barrier to writing — so we wanted Forkpad to help get writers writing more. The process of consuming other’s work, and building off of it is one way that we can learn to be better writers, and Forkpad aims to make that easier.</p><p>As Saturday ended and Sunday began, we’d already built the main functionality of the Forkpad app — so we spent the last few hours before demo time getting our app up and running with a database so that writers could get a link that they could share. The ability to easily share their remixes means writers will hopefully be encouraged to build off each others work.</p><p>The process for using and publishing with Forkpad is much less formal than a traditional publishing process, but we think giving writers the ability to <em>get writing </em>will always trump a process that start with big end goals in mind. Writing is a process, so hopefully Forkpad can ask how that process can evolve for writers with the help of the machine. Much like the atmosphere at CODEX, it’s not a zero-sum game between the machine and publisher, but finding situations that allow both to be part of the creative process.</p><p>We’re not sure what writers will make, and we definitely don’t know what the end products will be, but we think it will be something fun.</p><p><a href="http://forkpad.me/">Forkpad.me</a> is live. Send us some Drake fanfic?</p><p>@<a href="https://twitter.com/maxbittker">maxbittker</a> and @<a href="https://twitter.com/pippinlee">pippinlee</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49f2a6900ca9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex-hack/will-the-future-of-writing-be-more-like-software-49f2a6900ca9">Thoughts from making tools for literature</a> was originally published in <a href="https://medium.com/codex-hack">CODEX HACK</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Open Journalism Project: Better Student Journalism]]></title>
            <link>https://medium.com/the-open-journalism-project/the-open-journalism-project-better-student-journalism-fb39f4f701bb?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb39f4f701bb</guid>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[medium]]></category>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Tue, 17 Mar 2015 16:27:40 GMT</pubDate>
            <atom:updated>2015-08-25T18:29:29.925Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oBWUXtszDsiv_-Qq2bFLTQ.png" /></figure><h2>Open Journalism Project:</h2><h4><em>Better Student Journalism</em></h4><p>We pushed out the first version of the <a href="http://openjournalismproject.com/">Open Journalism site</a> in January. Our goal is for the site to be a place to teach students what they should know about journalism on the web. It should be fun too.</p><p>Topics like <a href="http://openjournalismproject.com/Mapping/">mapping</a>, <a href="http://openjournalismproject.com/Security/">security</a>, command line tools, and <a href="http://openjournalismproject.com/Open-source/">open source</a> are all concepts that should be made more accessible, and should be easily understood at a basic level by all journalists. We’re focusing on students because we know student journalism well, and we believe that teaching maturing journalists about the web will provide them with an important lens to view the world with. This is how we got to where we are now.</p><h3>Circa 2011</h3><p>In late 2011 I sat in the design room of our university’s student newsroom with some of the other editors: Kate Hudson, Brent Rose, and Nicholas Maronese. I was working as the photo editor then—something I loved doing. I was very happy travelling and photographing people while listening to their stories.</p><p>Photography was my lucky way of experiencing the many types of people my generation seemed to avoid, as well as many the public spends too much time discussing. One of my habits as a photographer was scouring sites like Flickr to see how others could frame the world in ways I hadn’t previously considered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*AzYWbe4cZkMMEUbfRjysLQ.png" /><figcaption>topleftpixel.com</figcaption></figure><p>I started discovering beautiful things the <a href="http://wvs.topleftpixel.com/13/02/06/timelapse-strips-homewood.htm">web could do with images</a>: things not possible with print. Just as every generation revolts against walking in the previous generations shoes, I found myself questioning the expectations that I came up against as a photo editor. In our newsroom the expectations were built from an outdated information world. We were expected to fill old shoes.</p><p>So we sat in our student newsroom—not very happy with what we were doing. Our weekly newspaper had remained essentially unchanged for 40+ years. Each editorial position had the same requirement every year. The <em>big</em> change happened in the 80s when the paper started using colour. We’d also stumbled into having a website, but it was updated just once a week with the release of the newspaper.</p><p>Information had changed form, but the student newsroom hadn’t, and it was becoming harder to romanticize the dusty newsprint smell coming from the shoes we were handed down from previous generations of editors. It was, we were told, all part of “becoming a journalist.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*d0Hp6KlzyIcGHcL6to1sYQ.png" /></figure><h3>We don’t know what we don’t know</h3><p>We spent much of the rest of the school year asking “what should we be doing in the newsroom?”, which mainly led us to ask “how do we use the web to tell stories?” It was a straightforward question that led to many more questions about the web: something we knew little about. Out in the real world, traditional journalists were struggling to keep their jobs in a dying print world. They wore the same design of shoes that we were supposed to fill. Being pushed to repeat old, failing strategies and blocked from trying something new scared us.</p><p>We had questions, so we started doing some research. We talked with student newsrooms in Canada and the United States, and filled too many Google Doc files with notes. Looking at the notes now, they scream of fear. We annotated our notes with naive solutions, often involving scrambled and immature odysseys into the future of online journalism.</p><p>There was a lot we didn’t know. We didn’t know <strong>how to build a mobile app</strong>. We didn’t know <strong>if we should build a mobile app</strong>. We didn’t know <strong>how to run a server</strong>. We didn’t know <strong>where to go to find a server</strong>. We didn’t know <strong>how the web worked</strong>. We didn’t know <strong>how people used the web to read news</strong>. We didn’t know <strong>what news should be on the web</strong>. If news is just information, what does that even look like?</p><p>We asked these questions to many students at other papers to get a consensus of what had worked and what hadn’t. They reported similar questions and fears about the web but followed with “print advertising is keeping us afloat so we can’t abandon it”.</p><p>In other words, we knew that we should be building a newer pair of shoes, but we didn’t know what the function of the shoes should be.</p><h3>Common problems in student newsrooms (2011)</h3><p>Our questioning of other student journalists in 15 student newsrooms brought up a few repeating issues.</p><ul><li>Lack of mentorship</li><li>A news process that lacked consideration of the web</li><li>No editor/position specific to the web</li><li>Little exposure to many of the cool projects being put together by professional newsrooms</li><li>Lack of diverse skills within the newsroom. Writers made up 95% of the personnel. Students with other skills were not sought because journalism was seen as “a career with words.” The other 5% were designers, designing words on computers, for print.</li><li>Not enough discussion between the business side and web efforts</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_9KYIFrk_PqWFgptsMDeww.png" /><figcaption>From our 2011 research</figcaption></figure><h3>Common problems in student newsrooms (2013)</h3><p>Two years later, we went back and looked at what had changed. We talked to a dozen more newsrooms and weren’t surprised by our findings.</p><ul><li>Still no mentorship or link to professional newsrooms building stories for the web</li><li>Very little control of website and technology</li><li>The lack of exposure that student journalists have to interactive storytelling. While some newsrooms are in touch with what’s happening with the web and journalism, there still exists a huge gap between the student newsroom and its professional counterpart</li><li>No time in the current news development cycle for student newsrooms to experiment with the web</li><li>Lack of skill diversity (specifically coding, interaction design, and statistics)</li><li>Overly restricted access to student website technology. Changes are primarily visual rather than functional.</li><li>Significantly reduced print production of many papers</li><li>Computers aren’t set up for experimenting with software and code, and often locked down</li></ul><p>Newsrooms have traditionally been covered in copies of The New York Times or Globe and Mail. Instead newsrooms should try spend at 20 minutes each week going over the coolest/weirdest online storytelling in an effort to expose each other to what is possible. “<a href="http://nytlabs.com/">Hey, what has the New York Times R&amp;D lab been up to this week?</a>”</p><p>Instead of having computers that are locked down, try setting aside a few office computers that allow students to play and “break”, or encourage editors to buy their own Macbooks so they’re always able to practice with code and new tools on their own.</p><p>From all this we realized that changing a student newsroom is difficult. It takes patience. It requires that the business and editorial departments of the student newsroom be on the same (web)page. The shoes of the future must be different from the shoes we were given.</p><p>We need to rethink how long the new shoe design will be valid. It’s more important that we focus on the process behind making footwear than on actually creating a specific shoe. We shouldn’t be building a shoe to last 40 years. Our footwear design process will allow us to change and adapt as technology evolves. The media landscape will change, so having a newsroom that can change with it will be critical.</p><p><strong>We are building a shoe machine, not a shoe.</strong></p><h3>A train or light at the end of the tunnel: are student newsrooms changing for the better?</h3><p>In our 2013 research we found that almost 50% of student newsrooms had created roles specifically for the web. <strong>This sounds great, but is still problematic in its current state.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*Vh2MpQjqjPkzYJaaWExoVg.png" /><figcaption><strong>We designed many of these slides to help explain to ourselves what we were doing</strong></figcaption></figure><p>When a newsroom decides to create a position for the web, it’s often with the intent of having content flow steadily from writers onto the web. This is a big improvement from just uploading stories to the web whenever there is a print issue. <em>However…</em></p><ol><li><strong>The handoff</strong><br>Problems arise because web editors are given roles that absolve the rest of the editors from thinking about the web. All editors should be involved in the process of story development for the web. While it’s a good idea to have one specific editor manage the website, contributors and editors should all play with and learn about the web. Instead of “can you make a computer do XYZ for me?”, we should be saying “can you show me how to make a computer do XYZ?”</li><li><strong>Not just social media<br></strong>A web editor could do much more than simply being in charge of the social media accounts for the student paper. Their responsibility could include teaching all other editors to be listening to what’s happening online. The web editor can take advantage of live information to change how the student newsroom reports news in real time.</li><li><strong>Web (interactive) editor<br></strong>The goal of having a web editor should be for someone to build and tell stories that take full advantage of the web as their medium. Too often the web’s interactivity is not considered when developing the story. The web then ends up as a resting place for print words.</li></ol><p>Editors at newsrooms are still figuring out how to convince writers of the benefit to having their content online. There’s still a stronger draw to writers seeing their name in print than on the web. Showing writers that their stories can be told in new ways to larger audiences is a convincing argument that the web is a starting point for telling a story, not its graveyard.</p><p>When everyone in the newsroom approaches their website with the intention of using it to explore the web as a medium, they all start to ask “what is possible?” and “what can be done?” You can’t expect students to think in terms of the web if it’s treated as a place for print words to hang out on a web page.</p><p>We’re OK with this problem, if we see newsrooms continue to take small steps towards having all their editors involved in the stories for the web.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Ln_DmC95Xpz6LzgywkcFQ.png" /><figcaption>The current Open Journalism site was a few years in the making. This was an original launch page we use in 2012</figcaption></figure><h3>What we know</h3><ul><li><strong>New process</strong><br>Our rough research has told us newsrooms need to be reorganized. This includes every part of the newsroom’s workflow: from where a story and its information comes from, to thinking of every word, pixel, and interaction the reader will have with your stories. If I was a photo editor that wanted to re-think my process with digital tools in mind, I’d start by asking “how are photo assignments processed and sent out?”, “how do we receive images?”, “what formats do images need to be exported in?”, “what type of screens will the images be viewed on?”, and “how are the designers getting these images?” Making a student newsroom digital isn’t about producing “digital manifestos”, it’s about being curious enough that you’ll want to to continue experimenting with your process until you’ve found one that fits your newsroom’s needs.</li><li><strong>More (remote) mentorship</strong><br>Lack of mentorship is still a big problem. <a href="http://www.google.com/get/journalismfellowship/">Google’s fellowship program</a> is great. The fact that it only caters to United States students isn’t. There are only a handful of internships in Canada where students interested in journalism can get experience writing code and building interactive stories. We’re OK with this for now, as we expect internships and mentorship over the next 5 years between professional newsrooms and student newsrooms will only increase. It’s worth noting that some of that mentorship will likely be done remotely.</li><li><strong>Changing a newsroom culture</strong><br>Skill diversity needs to change. We encourage every student newsroom we talk to, to start building a partnership with their school’s Computer Science department. It will take some work, but you’ll find there are many CS undergrads that love playing with web technologies, and using data to tell stories. Changing who is in the newsroom should be one of the first steps newsrooms take to changing how they tell stories. The same goes with getting designers who understand the wonderful interactive elements of the web and students who love statistics and exploring data. Getting students who are amazing at design, data, code, words, and images into one room is one of the coolest experience I’ve had. Everyone benefits from a more diverse newsroom.</li></ul><h3>What we don’t know</h3><ul><li><strong>Sharing curiosity for the web</strong><br>We don’t know how to best teach students about the web. It’s not efficient for us to teach coding classes. We do go into newsrooms and get them running their first code exercises, but if someone wants to learn to program, we can only provide the initial push and curiosity. We will be trying out “labs” with a few schools next school year to hopefully get a better idea of how to teach students about the web.</li><li><strong>Business</strong><br>We don’t know how to convince the business side of student papers that they should invest in the web. At the very least we’re able to explain that having students graduate with their current skill set is painful in the current job market.</li><li><strong>The future</strong><br>We don’t know what journalism or the web will be like in 10 years, but we can start encouraging students to keep an open mind about the skills they’ll need. We’re less interested in preparing students for the current newsroom climate, than we are in teaching students to have the ability to learn new tools quickly as they come and go.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zz5haO6iz7Hlj0z2IUHulg.png" /><figcaption>Another slide from 2012 website</figcaption></figure><h3>What we’re trying to share with others</h3><ul><li><strong>A concise guide to building stories for the web</strong><br>There are too many options to get started. We hope to provide an opinionated guide that follows both our experiences, research, and observations from trying to teach our peers.</li></ul><p>Student newsrooms don’t have investors to please. Student newsrooms can change their website every week if they want to try a new design or interaction. As long as students start treating the web as a different medium, and start building stories around that idea, then we’ll know we’re moving forward.</p><h3>A note to professional news orgs</h3><p>We’re also asking professional newsrooms to be more open about their process of developing stories for the web. You play a big part in this. This means writing about it, and sharing code. We need to start building a bridge between student journalism and professional newsrooms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*bXaR_NBJdoHpRc8lUWSsow.png" /><figcaption>2012</figcaption></figure><h3>This is a start</h3><p>We going to continue slowly growing the content on <a href="http://pippinlee.github.io/open-journalism-project/">Open Journalism</a>. We still consider this the beta version, but expect to polish it, and beef up the content for a real launch at the beginning of the summer.</p><p>We expect to have more original tutorials as well as the beginnings of what a curriculum may look like that a student newsroom can adopt to start guiding their transition to become a web first newsroom. We’re also going to be working with the <a href="http://queensjournal.ca/">Queen’s Journal</a> and <a href="http://ubyssey.ca/">The Ubyssey</a> next school year to better understand how to make the student newsroom a place for experimenting with telling stories on the web. If this sound like a good idea in your newsroom, we’re still looking to add 1 more school.</p><p>We’re trying out some new shoes. And while they’re not self-lacing, and smell a bit different, we feel lacing up a new pair of kicks can change a lot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*lulfisQxgSQ209vPHMAifg.png" /></figure><p><strong>Let’s talk. Let’s listen.</strong></p><p><strong>We’re still in the early stages of what this project will look like, so if you want to help or have thoughts, let’s talk.</strong></p><p><a href="/service/mailto:pippinblee@gmail.com"><strong>pippin@pippinlee.com</strong></a></p><p><em>This isn’t supposed to be a </em><strong><em>manifesto™©</em></strong><em> we just think it’s pretty cool to share what we’ve learned so far, and hope you’ll do the same. We’re all in this together.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb39f4f701bb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-open-journalism-project/the-open-journalism-project-better-student-journalism-fb39f4f701bb">The Open Journalism Project: Better Student Journalism</a> was originally published in <a href="https://medium.com/the-open-journalism-project">The Open Journalism Project</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Flying Above Toronto]]></title>
            <link>https://medium.com/@pippinlee/flying-above-toronto-2880ac1eb972?source=rss-28ff78fed88e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2880ac1eb972</guid>
            <dc:creator><![CDATA[Pippin Lee]]></dc:creator>
            <pubDate>Thu, 25 Sep 2014 00:08:49 GMT</pubDate>
            <atom:updated>2014-09-26T02:37:56.482Z</atom:updated>
            <content:encoded><![CDATA[<p>Thank you, Gravity.</p><p>Convention has it that we applaud at the end of a flight—but then shouldn’t we give standing ovations during takeoff, if only as a nod to the mathematicians and physicists who toyed and perfected the art of the flight.</p><p>Sure, controlled falling (landing) is cool, but pushing metal planes into the sky seems remarkable in comparison.</p><p>A few days ago I was fortunate enough to see my hometown from above. I set off with good friends Nick (the pilot), and Yuni on a clear Tuesday afternoon to see Toronto, and southern Ontario.</p><p>Gravity is an incredible constraint—but we’ve learned to use its constraints to build machines that can create pressure differentials. And what can we do with pressure differentials? We can create lift! Lift ultimately allow us to reach thousands of feet in the sky.</p><p>The following images will hopefully give you a good visual census of Toronto.</p><p>Thanks, Gravity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dWpsoCxFR58Aot1FzIymQg.jpeg" /><figcaption>Nick goes through his pre-flight checklist</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3cr-eLrbnbscnqPX-2oFeA.jpeg" /></figure><p>Travelling down the runway, my ears and body feel the roar of our 1600-pound Cessna 172 as it hits 60MPH (55 kts). As our wheels leave the ground, I’m struck by the feats of engineering that allow us to simply jump into the sky in a aluminium box. It is the aviation giants whose shoulders we stand on which seem to give us a control switch to gravity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mXvVISFpJfvrJ9uACR__kA.jpeg" /><figcaption>Toronto waterfront, from Lake Ontario</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qsdffgDNAs9EnKhRXQ9xsg.jpeg" /><figcaption>Looking West. Mississauga and Oakville in the distance</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pFoxuFqHHo3K9OgeVfFYKA.jpeg" /><figcaption>Yuni’s heart rate peaking</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QkuD3C-95ByI5JP1cptehA.jpeg" /><figcaption>The Bloor-Danforth viaduct bridge, and Don Valley River</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3_rgbtA0nSA60xmg9OSvQ.jpeg" /><figcaption>This building was the tallest in the World for 34 years</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PvhFi-hcTJavaHaud2YPDg.jpeg" /><figcaption>Looking East along the Lakeshore</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fdQxscWabTJxm9yE9mDEWw.jpeg" /><figcaption>Nick and I, pleased with Gravity</figcaption></figure><p>Next time during takeoff, consider raising your Styrofoam cup, if only to give a slight nod to the aviation giants of the past, and wishing you could tell them how bad airline coffee is.</p><p>Images were taken with a Nikon D700 + 24–70 lens.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2880ac1eb972" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>