<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Panda on Medium]]></title>
        <description><![CDATA[Stories by Panda on Medium]]></description>
        <link>https://medium.com/@panda061325?source=rss-22e3e0f5070c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9lCwitr_uESqztO6BhzxAA.jpeg</url>
            <title>Stories by Panda on Medium</title>
            <link>https://medium.com/@panda061325?source=rss-22e3e0f5070c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:19:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@panda061325/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Planning to Travel Around the World, with Python?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/planning-to-travel-around-the-world-with-python-42fac1d21a6e?source=rss-22e3e0f5070c------2"><img src="https://cdn-images-1.medium.com/max/1920/1*scYnG2z-YgGLoRIg8rojrw.jpeg" width="1920"></a></p><p class="medium-feed-snippet">S1: Drawing a Map with Basemap</p><p class="medium-feed-link"><a href="https://medium.com/data-science/planning-to-travel-around-the-world-with-python-42fac1d21a6e?source=rss-22e3e0f5070c------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/planning-to-travel-around-the-world-with-python-42fac1d21a6e?source=rss-22e3e0f5070c------2</link>
            <guid isPermaLink="false">https://medium.com/p/42fac1d21a6e</guid>
            <category><![CDATA[basemap]]></category>
            <category><![CDATA[matplotlib]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[networkx]]></category>
            <dc:creator><![CDATA[Panda]]></dc:creator>
            <pubDate>Sun, 27 Oct 2019 03:12:39 GMT</pubDate>
            <atom:updated>2019-10-27T04:18:25.810Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Reverse Mentoring Sharing — Open Source]]></title>
            <link>https://medium.com/data-science/reverse-mentoring-sharing-open-source-868cc19da97a?source=rss-22e3e0f5070c------2</link>
            <guid isPermaLink="false">https://medium.com/p/868cc19da97a</guid>
            <category><![CDATA[knowledge-sharing]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Panda]]></dc:creator>
            <pubDate>Mon, 15 Apr 2019 00:21:19 GMT</pubDate>
            <atom:updated>2019-04-15T02:49:07.288Z</atom:updated>
            <content:encoded><![CDATA[<h3>Reverse Mentoring Sharing — Open Source</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ND7KaK413kFB8D-kPUZpPQ.jpeg" /><figcaption>Source: genesis_3g on Pixabay</figcaption></figure><p>Recently I participated in the reverse mentoring program to tutor our executives, who have great experience in their domain knowledge but have limited understanding toward new trends in technologies. We focus on four fields which they have strongest interests: <strong>Open Source</strong>, <strong>Artificial Intelligence</strong>, <strong>Coding &amp; Prototyping</strong> and <strong>Digital Trends</strong>. Mentees and I, as a mentor, feel that we have learned a lot from each other and it worths to share with more people, and that is why I am writing the series.</p><p>In this series, we have four articles for each topic:</p><ul><li>Reverse Mentoring Sharing — Open Source [<a href="https://towardsdatascience.com/reverse-mentoring-sharing-open-source-868cc19da97a"><strong>Article</strong></a>] [<a href="https://www.slideshare.net/ssuserf1b9cf/open-source-140737960"><strong>Slide</strong></a>]</li><li>Reverse Mentoring Sharing — Artificial Intelligence (to be continued…</li><li>Reverse Mentoring Sharing — Coding &amp; Prototyping (to be continued…</li><li>Reverse Mentoring Sharing — Digital Trends (to be continued…</li></ul><p>In each topic, I would first introduce <em>what is it</em> to help mentees have basic understanding toward the topic. Once they learned the topic, I would explain <em>why the topic is important</em> for people nowadays. As executives, they care more about <em>how our company can benefit from the technology</em>, and it would be included in the third part.</p><p>In each article, I will share my PowerPoint and what I mentioned on each slide. Besides, I will conclude with my feeling with each topics after I shared with executives and their feedbacks in the end. <strong>You can go the the last part to understand opinions from executives and me once you have already understood the topic</strong>. I will exclude anything sensitive in our business and hopefully everyone, no matter you are familiar with those topics or not, would be able to learn something from the articles. <strong>If you have any new ideas, please leave your comments and discuss with us.</strong> I am really appreciate the responses and it is the best thing to learn with people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*miarooczUvw7ArsksY84Cw.jpeg" /></figure><p>There would be three parts in the instruction: what is open source? Why is open source important? How our company can benefit from open source? In the first part, it would start with the world without the idea of open source and thus people can understand the importance of open source. After it, I will give the definition of open source and some famous open source projects. In the second parts, I will justify why we need to care about open source by demonstrating how energetic and active for people in open source communities. Last, I will pinpoint how a company can benefit from open source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fTBUwJNyMQd9cI6s4Oh8mg.jpeg" /></figure><p>Before open source, we tend to use intellectual property to protect people who come up with new ideas. We believe that by protecting property right of inventors and giving them excess rewards, we can incentive more creations, which is beneficial to our society.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HfwuODGU_8f023gwI0aAeQ.jpeg" /></figure><p>However, there are some downsides for the system of intellectual property. First, users would no longer know the products thoroughly, as developers and companies need to hide their recipe and keep on earning the excess rewards. Besides, users, as the people who have the real experience in the products and know what should be improved, cannot do anything as they do not have the rights. In the end, the world does not become innovative and cooperative but self-centered and filled with endless lawsuits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ML05EXD99nayW_A_hDbnQ.jpeg" /></figure><p>To make people be able to work together instead of fighting against each other, the idea of open source become popular. It allows others to access and distribute of an end products, including but not limited to software or program, freely. By doing so, we solve the adversity from intellectual intelligence. One thing worths to be mentioned is that open source is actually happening in many fields. For example, <a href="https://en.wikipedia.org/wiki/Open-source_cola">Open-source cola</a> is a published and sharable recipe for soft drinks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWRO5B1ixwzKmmIqjYNioQ.jpeg" /></figure><p>Not any projects can be defined as open source on their own. From Open Source Initiative, there are <a href="https://opensource.org/osd-annotated">10 principles </a>an open source project needed to follow. On the fifth principle “No Discrimination Against Persons or Groups” indicates products only free for non-commercial purpose could not be defined as open source. It makes open source software very different from traditional “free software” and more adaptable for businesses. Also, open source is not only an idea but legal licenses. Three of the most popular open source licenses are: MIT license, GPLv2 license and Apache License.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tQVSRBnC1lmOq45x6bmJ2Q.jpeg" /></figure><p>Open source is closely related to the success of technologies and we can see it all around us. From operation system like Linux and Android, web browser Firefox, analytical tools Python and R, database like Hadoop to algorithms, hidden behind many applications, like TensorFlow and OpenCV, we really cannot image lives without them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NYely8isK7iJvu7l2EnbVQ.jpeg" /></figure><p>As we mentioned, open source is critical for our current technologies, and in this part we will talk about why it could be so powerful. A successful product means it has great amount of users and providers, so we can start with users’ side to understand why companies and people want to use open source.</p><p>First and the most obvious reason, open source is cost advantage. However, other from the reason, open source can prevent companies from annoying troubles like lawsuits and compatible issues. With large and active communities, open source can usually apply the latest technologies and have easy to read instructions and examples from many websites and user communities. If you still face a very customized problems, there are actually a lot of paid support waiting for you with experienced consultants. Thus, using open source is a cheap and simple solution for companies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jKbJuRBBaqxye1NM5cH6FA.jpeg" /></figure><p>It might be easy to think why companies to use open source but hard to tell why they want to provide it. In fact, companies are willing to open their projects to the public, as Google hosting 2000+ projects an example. The reason behind it is that opening their source codes could also be beneficial to companies. For example, it is the best advertisement for them, not only for their products but also for talents. Also, open source can attract right people to join their companies and spend less time on training them what they are going to do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IneV3YyUjCo7ltz40GIyxA.jpeg" /></figure><p>Another crucial participants in open source are volunteering developers. What makes companies to use and to open their codes to the public is the groups of talents behind open source communities. The reasons for programmers and developers to spend hours on contributing to open source are a lot, and I just give five points as instances. With those incentives, we can believe that there would be more and more people continuing to make a contribution passionately to open source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*54Ex1q140L4aZnkW5JrovQ.jpeg" /></figure><p>In my opinion, open source is more than legal documents but the culture of sharing. As a non-technology company, what we can learn from open source, or the rapid growth in technology world, is <strong>the influence from openness and cooperation.</strong> We tend to make everything we learned as a secret and prevent others from knowing it. However, to cooperate with others efficiently, we should not be afraid to share our knowledge with others. We need to believe in the power of openness and make open as a normal, close as an exception. For a big company, it is not necessary to open to the public, but to other departments. If you agree that we want the same fast progress as tech companies, we need to embrace the mindset of openness.</p><h3>Feedback</h3><p>As a technology evangelist, I am a big fan of open source. From my perspective, without open source, we could not see the convenience from new technologies, and the idea of Big Data and AI would no longer be possible. Our progress would be slow and less efficient, and I cannot create anything interesting with affordable cost (I am a heavy user of Python and R.) Also, I am used to the phenomenon that thousands of requests and issues are happening on GitHub everyday. I am optimistic that everything could be solved by openness.</p><p>However for executives, even though they are impressed with the idea of open source, they saw more challenges than brightness. They pointed out that, for example, the <a href="https://www.zdnet.com/article/how-to-steal-a-tesla-model-s-in-seconds/">defect of Tesla’s key</a> could be linked to its open source culture. Also, as a matured business with reliable brand reputation, our customers would not tolerate mistakes in the same way as those tech companies. At the same time, we need to be more careful about what we choose to disclose since it could impact on our partners, such as suppliers, as well. That could also include sharing information within company.</p><p>Everyone understands the value of collaborating and sharing, and that is the core value behind open source, contradicting to the idea of intellectual property. Nonetheless, as people in different positions, they would have various perspective to applying the concept of open source. Probably not everyone can replicate what those tech companies do, but we should always keep in mind the capability of sharing. We can never know what influence you can make by telling others what you know, so just try not to keep it secret in order to make a better world.</p><h3>Reference</h3><ul><li><a href="https://en.wikipedia.org/wiki/Open-source_model"><strong><em>Open-source model [Wikipedia]</em></strong></a></li><li><a href="https://en.wikipedia.org/wiki/Open-source_software"><strong><em>Open-source software [Wikipedia]</em></strong></a></li><li><a href="https://rubygarage.org/blog/why-open-source-is-good-for-business"><strong><em>Why Open Source Is Good For Business [RubyGarage]</em></strong></a></li><li><a href="https://readwrite.com/2015/01/28/open-source-code-5-reasons-to-open-up/"><strong><em>5 Reasons Your Company Should Open Source More Code [Matt Asay]</em></strong></a></li><li><a href="https://en.wikipedia.org/wiki/The_Open_Source_Definition"><strong><em>The Open Source Definition [Wikipedia]</em></strong></a></li><li><a href="https://en.wikipedia.org/wiki/Apache_License"><strong><em>Apache License [Wikipedia]</em></strong></a></li><li><a href="https://www.makeuseof.com/tag/people-contribute-open-source-projects/"><strong><em>Why Do People Contribute to Open Source Projects [Joel Lee]</em></strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=868cc19da97a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/reverse-mentoring-sharing-open-source-868cc19da97a">Reverse Mentoring Sharing — Open Source</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Step by Step to Visualize Music Genres with Spotify API]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/step-by-step-to-visualize-music-genres-with-spotify-api-ce6c273fb827?source=rss-22e3e0f5070c------2"><img src="https://cdn-images-1.medium.com/max/1280/1*zOshpZng8plvNt3pPv6KIA.png" width="1280"></a></p><p class="medium-feed-snippet">You are what you listen to. Can we tell, or see, the difference of music genres with data?</p><p class="medium-feed-link"><a href="https://medium.com/data-science/step-by-step-to-visualize-music-genres-with-spotify-api-ce6c273fb827?source=rss-22e3e0f5070c------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/step-by-step-to-visualize-music-genres-with-spotify-api-ce6c273fb827?source=rss-22e3e0f5070c------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce6c273fb827</guid>
            <category><![CDATA[music]]></category>
            <category><![CDATA[spotify]]></category>
            <category><![CDATA[shiny]]></category>
            <category><![CDATA[r]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Panda]]></dc:creator>
            <pubDate>Mon, 18 Feb 2019 08:44:25 GMT</pubDate>
            <atom:updated>2019-02-18T08:44:25.804Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Analysis of Stock Market Cycles with fbprophet package in Python]]></title>
            <link>https://medium.com/data-science/analysis-of-stock-market-cycles-with-fbprophet-package-in-python-7c36db32ecd0?source=rss-22e3e0f5070c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c36db32ecd0</guid>
            <category><![CDATA[fbprophet]]></category>
            <category><![CDATA[stock-market]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Panda]]></dc:creator>
            <pubDate>Fri, 12 Oct 2018 14:53:03 GMT</pubDate>
            <atom:updated>2018-10-12T14:53:03.222Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>IMPORTANT: THIS IS NOT INVESTMENT ADVICE.</strong></p><p><a href="https://en.wikipedia.org/wiki/Stock_market_cycles"><em>Stock market cycles are the long-term price patterns of stock markets and are often associated with general business cycles. They are key to technical analysis where the approach to investing is based on cycles or repeating price patterns.</em></a> If we have better understanding toward the cycles of stock market, we can always buy with relative low-price and sell at relative high price in each cycle, and we’ll always have positive return. What a wonderful world! Of course, there is no superior strategy in stock market which can make money forever, but fbprophet package in Python or R can help us look deeper into the hidden cycles in stock market. In this analysis, we can take a look at how fbprophet can assist us to make an investment decision, and all codes are available from <a href="https://barrypan.github.io/Fbprophet_ANA/Final_Analysis.html">here</a>.</p><h3>Introduction to fbprophet</h3><p>Fbprophet is an open source released by Facebook in order to <a href="https://peerj.com/preprints/3190.pdf">provide some useful guidance for producing forecast at scale.</a> By default, it would divide a time series into trend and seasonality, which might contain yearly, weekly and daily. However, analysts can define their own seasonality. To get better understanding about the package, the <a href="https://facebook.github.io/prophet/docs/quick_start.html">document</a> from Prophet is really helpful.</p><p>One of the feature for package is its simplicity and flexibility. Since cycles in stock market we want to figure out are not limited to yearly, weekly or daily, we should define our own cycles and find out which can fit the data better. Besides, we should not use weekly seasonality since there is no trading on weekend. We can also define our ‘self_define_cycle’ by add_seasonality function. All the settings can be done with only two lines of code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/806e2cceaa9a374d95400b9178e68425/href">https://medium.com/media/806e2cceaa9a374d95400b9178e68425/href</a></iframe><h3>Stock Market Cycles Analysis — taking Costco as an example</h3><p>We can use close prices for Costco from 2015/10/1 to 2018/10/1 as an example to have better understanding about what we are doing. With pandas_datareader we can access stock price easily. The documentation is <a href="https://pandas-datareader.readthedocs.io/en/latest/remote_data.html">here</a>. In the Figure 1 we can see there is a strong trend of growing price from 2015. However, there are still a lot of up-and-down, or cycles, during the journey, and these are what we want to make money.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*tjSOilp_FadNhtHRKo_Awg.png" /><figcaption><strong>Figure 1: Costco stock price from 2015/10/01 to 2018/10/01</strong></figcaption></figure><p>For prediction models, one way to evaluate them is out-sample mean squared error. We can use 2015/10/1 to 2018/3/31 for training and keep the last 6 months for testing and calculating out-sample mean squared error. Within each cycle, we can optimized our return by buying at the lowest price and selling at the highest. To make the process easier, we use self-define function cycle_analysis. The output is a list containing projected return per cycle and out-sample mean squared error. Inputs for the function require:</p><ul><li><strong>data:</strong> Pandas dataframe with time index</li><li><strong>split_date:</strong> the date to split training and testing data</li><li><strong>cycle:</strong> periods (days) per cycle</li><li><strong>mode:</strong> additive or multiplicative for seasonality <em>(optional, default additive)</em></li><li><strong>forecast_plot:</strong> whether to print the forecast plot or not <em>(optional, default False)</em></li><li><strong>print_ind:</strong> whether to print the projected return per cycle and out-sample mean squared error or not <em>(optional, default False)</em></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c7f94e26574b7836d85c414e8356520b/href">https://medium.com/media/c7f94e26574b7836d85c414e8356520b/href</a></iframe><p>In the Figure 2 and Figure 3, we applied the function with two different lengths of cycle, 30 and 300 respectively, on Costco stock price and took 2018/4/1 as the split date for training and testing. As we can see, if we pick a length that is too short (e.g 30 days,) the return within one cycle is small and we need to make transaction frequently(Figure 2;) in contrast, if we pick a length which is too long (e.g 300 days,) it would distort our prediction (Figure 3.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*kwJDzv2Qq4Cb7SLQZImNBA.png" /><figcaption><strong>Figure 2: Prediction with cycle equal to 30 days</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*7fOHQo50k9A4VCGsLUP4nQ.png" /><figcaption><strong>Figure 3: Prediction with cycle equal to 300 days</strong></figcaption></figure><p>We can apply a loop over our cycle_analysis function to calculate projected return and out-sample mean squared error for different length of cycle, and we displayed the outcome in Figure 4. As we can see, the longer the length, both projected return per cycle and out-sample mean square error would increase. In consideration of the cost of transaction, the projected return within per cycle should be greater than $10. Under this constrain, we can choose the cycle that leads to minimum out-sample mean squared error, and it is 252 days. The projected return per cycle is $17.12 and out-sample mean squared error is 15.936. Both are pretty good!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/959/1*SZXpjdQEa8fmdB8rqshZmw.png" /><figcaption><strong>Figure 4: Projected Return and Out-Sample Mean Squared Error for different length of cycle</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*Uo7qtJrGdKnOreszOOXa5w.png" /><figcaption><strong>Figure 5: Prediction with cycle equal to 252 days</strong></figcaption></figure><p>To further illustrate the investment strategy, we can see the buying and selling dates between 2015/10/1 and 2018/10/1. The Return_Dates function could return all buy and sell dates as output, with input:</p><ul><li><strong>forecast:</strong> fbprophet forecast object</li><li><strong>stock_data:</strong> Pandas dataframe with time index</li><li><strong>cycle:</strong> length of cycle</li><li><strong>cycle_name:</strong> name for the cycle column in forecast object</li><li><strong>time_name:</strong> name for the time column in forecast object</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e2c4211317a5dcd7183dcd979db3bdc/href">https://medium.com/media/4e2c4211317a5dcd7183dcd979db3bdc/href</a></iframe><p>During 2015/10/1 and 2018/10/1, we would buy and sell Costco four times. In summary, we would spend $604.56 on buying and get $744.78 in return when we sold them on those specific dates. For a simplified return rate (without considering reinvestment, time value and so on,) it is 23.2% for 3 years. Probably not very attractive, but at least it is positive return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8sr4BOKP7aJzNNHo9jktAQ.png" /></figure><h3>Applying to more stocks</h3><p>Definitely, this method could be applied to as many stocks as possible. We listed average buy price, average sell price, length of cycle, out-sample mean squared error, number of buying, number of selling and projected return within each cycle for Costco, Apple, Microsoft, Home Depot and Nike.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2cbfc14bfb462a73fcfa0cd2b59476bd/href">https://medium.com/media/2cbfc14bfb462a73fcfa0cd2b59476bd/href</a></iframe><p>For Microsoft and Nike we cannot find any cycle matched to our requirement of more than $10 return per cycle. For Costco, Apple and Home Depot, we can find a cycle around 250 days and make a good prediction and decent return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TLXgQSIXzSwJ1j5NesN9uQ.png" /></figure><h3>Summary</h3><p>With the help of Python and fbprophet package, we can have better understanding toward the stock market. Taking Costco as an example, we can find a cycle with 252 days so that it can generate enough return and have a good fit of data. According to the cycle we found, we can have about 23% return for 3 years. Perhaps this investment strategy cannot satisfy what you want, but you can always set your own approaches based on your knowledge and experience. Powerful fbprophet package can make your analysis on stock market much deeper and easier.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c36db32ecd0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/analysis-of-stock-market-cycles-with-fbprophet-package-in-python-7c36db32ecd0">Analysis of Stock Market Cycles with fbprophet package in Python</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stock Clustering with Time Series Clustering in R]]></title>
            <link>https://medium.com/@panda061325/stock-clustering-with-time-series-clustering-in-r-63fe1fabe1b6?source=rss-22e3e0f5070c------2</link>
            <guid isPermaLink="false">https://medium.com/p/63fe1fabe1b6</guid>
            <category><![CDATA[timeseries]]></category>
            <category><![CDATA[investment]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[stock-market]]></category>
            <category><![CDATA[r]]></category>
            <dc:creator><![CDATA[Panda]]></dc:creator>
            <pubDate>Fri, 10 Aug 2018 02:43:02 GMT</pubDate>
            <atom:updated>2018-08-10T02:43:02.712Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>IMPORTANT: THIS IS NOT INVESTMENT ADVICE.</strong></p><p>As a newbie in stock market, the amount of choices available always prevents us from moving forward. It would be much easier if there is a tool can classify different stocks based on their historical stock price and then we can determine my investment strategy. For this purpose, time series clustering with dtwclust package in R is perfect. It can compare different stock prices and group them together, with few lines of R code.</p><h3>Acquiring Data</h3><p>For a conservative investor, we choose to invest in S&amp;P 500 companies. It is pretty easy to access the a list of S&amp;P 500 companies list and symbols from the internet. After we have the list of companies and symbols, we can lookup historical stock prices from yahoo finance with quantmod package in R. Sample code to access Apple’s historical stock price from 7/1/2015 is below:</p><pre>library(quantmod)<br>stock = getSymbols(&quot;AAPL&quot;,src=&#39;yahoo&#39;,from = &#39;2015-07-01&#39;, auto.assign = F)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dTxAYvYZHn4euJDVuiEDXA.png" /></figure><h3>Time Series Clustering</h3><p>In this analysis, we use stock price between 7/1/2015 and 8/3/2018, 780 opening days . Besides, to be convenient, we take close price to represent the price for each day.</p><p>As stock prices for different companies vary a lot, we need to standardize them before comparison. Here, we can calculate the z score with the corresponding sample mean and deviation for each company in our dataset.</p><p>Without doing and grouping, we can plot the time series of standardized stock prices for the companies in Figure 1. As we can see, it is hard to detect which companies worth our money. However, generally there is a rising trend for S&amp;P 500 companies in the recent three years. The coefficient for the regression line is <strong><em>0.002</em></strong>, indicating that the standardized stock price would increase 0.002 per opening day, on average. That is, if you are rich enough to buy all the stocks in S&amp;P 500 and put them for three years, you should have some positive returns on your investment, without considering any inflation or interest rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*2q3jnTxcvsi_8XKdpfnAtA.png" /><figcaption><strong>Figure 1: standardized price time series</strong></figcaption></figure><p>However, for most of us, at least for me, cannot execute the previous investment strategy. Therefore, we need to do some grouping based on historical stock price performance. There are many options for dtwclust package in R, and we strongly recommend the two articles: <a href="https://cran.r-project.org/web/packages/dtwclust/vignettes/dtwclust.pdf">Comparing Time-Series Clustering Algorithms in R — Using the dtwclust Package</a> and <a href="https://github.com/asardaes/dtwclust">Time Series Clustering Along with Optimizations for the Dynamic Time Warping (DTW) Distance</a>, for more information.</p><p>In this analysis, we first try partitional clustering with Dynamic Time Warping distance (DTW) and Euclidean distance, for 5, 10 and 20 clusters. The sample code for DTW distance and Eulidean distance of 5 clusters is following:</p><pre># DTW distance</pre><pre># The input of normalized_price is a list of normalized time-series stock price</pre><pre>dtw_cluster = tsclust(normalized_price, type=&quot;partitional&quot;,k=5,<br>                      distance=&quot;dtw_basic&quot;,centroid = &quot;pam&quot;,seed=1234,trace=T,<br>                      args = tsclust_args(dist = list(window.size = 5)))</pre><pre># Euclidean distance</pre><pre>eu_cluster = tsclust(normalized_price, type=&quot;partitional&quot;,k=5,<br>                     distance=&quot;Euclidean&quot;,centroid = &quot;pam&quot;,seed=1234,trace=T)</pre><p>After clustering, we can draw the time series plot for each group to evaluate the groups in Figure 2 and Figure 3. As we can see, instead of aggregating too many companies together and losing the significant trends or dividing into a lot of groups and few differentiation between each group, 10 clusters might be a good choice. In the following analysis, we will focus on the 10 clusters with DTW distance, with window size equal to 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*nexkZVVhYTSTWBSBTfkQgA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*_QGx-60yCSa7vW2GMmxETQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*g-PE8PT516jB8bndzfGzlw.png" /><figcaption><strong>Figure 2: clustering with DTW distance for 5, 10 and 20 clusters</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*QgpKRbNpmRzf1GDK3DCYrg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*Kd_Zcf16lpH577GpP1cqcg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*Aw_tfojAHIBSYILyEu8qlA.png" /><figcaption><strong>Figure3: clustering with Euclidean distance for 5, 10 and 20 clusters</strong></figcaption></figure><h3>Cluster Analysis</h3><p>We can build up a better investment strategy by putting more money in stocks in booming groups and being careful to stocks in slumping clusters. In Figure 4 we can observe the major trend in each group. Obviously, group 1 and 5 have stronger growing trend for standardized stock price; in contrast, group 2, 6 and 10 have somehow decreasing trend.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*yqELj9FtkszTBEz5E8rEXw.png" /><figcaption><strong>Figure 4: 10 clusters with DTW distance</strong></figcaption></figure><p>To make it clearer, we can calculate the average gross spread, as part of investment return, for each group. In Figure 5 we can see that on average, we can earn about $80 per stock if we invest in group 1, but lose $30 per stock in group 6 in recent three years. In terms of return rate, if we consider only the investment returns from the price differences, the return rate is about 97% if we put our money in group 1 as an investment portfolio for three years. Oppositely, there is around -30% return rate for the portfolio of group 6 stocks. For now, our investment strategy seems to be better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*OLF5wt3PnDt7H6Wluvtssg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*kmKz_loIfKYhWtx6Jx6yFQ.png" /><figcaption><strong>Figure 5: gross spread and return rate for each group</strong></figcaption></figure><p>However, besides considering only return, we also need to think of risk. In Figure 6, we have smoothed density estimates plots for each group, based on the differences of standardized stock prices on 7/1/2015 and 8/3/2018. It is amazing that the standardized gross spread for group 1 and group 5 are concentrated above 0, indicating that they are not only providing better return but also less risky. If I were able to go back to 2015, I would put all of my money into stocks in group 1 and group 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*coUi19hlKksQq97EegDBEQ.png" /><figcaption><strong>Figure 6: standardized gross spread distribution for each group</strong></figcaption></figure><p>Sadly, we cannot travel back the time, so what we can do is to put more effort to understand the composite of each group. By doing so, at least we can learn something from the past. Besides, if we believe that the market does not change a lot (unfortunately, it is hardly possible), we can still put our money in group 1 or group 5. It shows the percentage of different industrial sectors in each group in Figure 7. <em>Information Technology</em> is the major sector in the “historically lucrative” group 1, similar to what we realized in the recent years. Besides, <em>Financials</em> and <em>Industrials</em> might also are good choices as they take some shares in group 1 and group 5 too. On the contrary, we should be more careful for <em>Health Care </em>industry as it is one of the major sector in declining group 6.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*r-58MpHp8eH4IRDBXQ6_JQ.png" /><figcaption><strong>Figure 7: Percentage of different industrial sectors in each group</strong></figcaption></figure><p>Other from looking at industry sectors, we can also search for the groups of our target companies. If they are in group 1 or group 5, we might have greater confidence as they had good performance in the past. If the companies are in group 6 or group 7, hopefully they are at end of the night now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QYhGuP-U5mFWG7ZU_Q6hhA.png" /></figure><h3>Summary</h3><p>In this analysis, we apply R with dtwclust package to classify S&amp;P 500 stocks into different groups, based on their historical stock price. The discrepancy between groups is large, as the best group can provide 97% return on investment in recently three years, but -30% for the worst one. Also, the distributions of return, indicating the risk of investment, are very different from groups to groups. By doing the analysis, we can come up with our own strategy much easier as we can point out which companies had more desirable performance in the past. Also, we can have better understanding about the overall performance in different industries by the analysis.</p><p>However, as all text books mentioned, the good performance in the past <strong>DOES NOT</strong> guarantee a better one in the future. All of our analysis is based on historical data and therefore the outcome cannot exactly tell what you should do now. After all, if I know an investment strategy with 97% return, I will keep it in secret!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63fe1fabe1b6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>