<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nick James on Medium]]></title>
        <description><![CDATA[Stories by Nick James on Medium]]></description>
        <link>https://medium.com/@njprivatemail?source=rss-31f0f5d507b5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*M72V_LqwA_jFeqfv</url>
            <title>Stories by Nick James on Medium</title>
            <link>https://medium.com/@njprivatemail?source=rss-31f0f5d507b5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:26:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@njprivatemail/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Spatio-temporal trends in the propagation and capacity of hydrogen projects]]></title>
            <link>https://njprivatemail.medium.com/spatio-temporal-trends-in-the-propagation-and-capacity-of-hydrogen-projects-8643f6c067ee?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/8643f6c067ee</guid>
            <category><![CDATA[hydrogen]]></category>
            <category><![CDATA[climate-change]]></category>
            <category><![CDATA[environment]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Thu, 14 Oct 2021 08:02:39 GMT</pubDate>
            <atom:updated>2021-10-14T08:12:46.330Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Overview:</strong></p><p><em>This white paper studies trends in the propagation of hydrogen plants and their respective capacity over the past two decades. We introduce a number of mathematical frameworks to reveal new insights into the evolution of hydrogen energy production across a number of geographic regions. We use data from the International Energy Agency.</em></p><p><strong>Background:</strong></p><p>Hydrogen has great potential as an alternative fuel source, and may play a role in the world’s coordinated attempt to reach net-zero emissions during this century. Hydrogen could be used for numerous purposes, including industry, transport, heating and energy production. Indeed by mass, hydrogen contains more than twice the energy potential as natural gas. However, hydrogen in its pure form does not exist naturally on earth. It must be synthesised via a variety of different procedures.</p><p>The production of hydrogen is classified by colours according to its mode of preparation. “Green hydrogen’’ refers to production techniques that do not generate any greenhouse gas (GHG) emissions. This is of course the ideal method of production to forward the goal of an alternative energy source that does not contribute further to global warming. Typically, green hydrogen plants use renewable sources of energy (such as solar) to extract hydrogen via electrolysis of water. This is clearly a desirable process, however the energy production capacity of green plants has been quite limited. On the other hand, black, brown and grey hydrogen refer to production techniques that use black coal, brown coal and natural gas respectively, generating harmful gases including carbon dioxide and monoxide. Blue hydrogen is defined as the generation using natural gas, followed by carbon capture and storage (CSS). This is not a truly zero-emissions process, as not all generated GHGs can be captured.</p><p>There is significant variability in a country’s level of sophistication with regards to alternative and clean energy. Many European countries such as Germany have been able to reduce carbon dioxide emissions over the past 50 years, largely driven by their willingness to drive the adoption of new technologies, often with significant initial cost. Less willing to wear the immediate economic consequences, many developing countries as well as the United States and Australia have been more hesitant to commit to green and alternative energy sources. Instead, they continue to utilise energy sources that have already reached economies of scale.</p><p>This paper investigates trends in the roll out and prevalence of hydrogen plants by both technology and location. We study the changing propagation and capacity of hydrogen plants over time, with particular interest in the increasing potential of green hydrogen plants. We also investigate differences in these roll outs on a geographic basis. Our main finding is promising: an exponential increase in the capacity of green plants over time, and a dramatic closing between the capacity of green and non-green plants.</p><p><strong>Data:</strong></p><p>Our data comes from the International Energy Agency, and consists of plants built in (or projected to be built in) 2000–2028, a period of T=29 years. Each plant is classified according to one of five underlying technologies: four different types of water electrolysis, which are all green plants, and ``fossil’’, indicating the use of fossil fuels. By aggregating the types of electrolysis, the plants are essentially classified as green or non-green (``fossil’’). For each plant, the location is also recorded, either specified by the country of location or continent. As such, all our analysis proceeds on a continent-by-continent basis, in which we record and analyse only the continent of location. We divide the plants into continental groups as follows: North America, South America, Europe, Oceania, East Asia (China and Japan) and Other Asia (mostly consisting of Indian plants).</p><p><strong>Linear Regression Analysis:</strong></p><p>In Figure 1, we display all the plants with a known energy capacity in our dataset. Displaying the logarithm of the capacity against the year of construction, we see an approximate linear trend between the log of capacity and the year. This suggests an exponential growth in capacity vs time. Earlier on, we see that fossil (ie non-green) plants are several orders of magnitude greater in capacity than green plants, but this difference reduces dramatically over time. To further explore these findings we implement two linear regressions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*VnYbTCq-cFiuqN4XtNPoVw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*e08C5F5M9SI_X8hxflOtjQ.png" /></figure><p>We encode six continents as Europe, North America, South America, East Asia (Japan and China), Other Asia (predominantly India), Oceania, with Europe as the default categorical variable. We also consider five technologies: ALK, PEM, SOEC, Unknown PtX and Fossil, with ALK as our default variable. For model 1, the adjusted R² is 0.17, while for model 2, adjusted R² is 0.65. This strongly suggests a better fit where capacity is predicted and understood to increase exponentially over time, confirming the qualitative observations in Figure 1.</p><p>We include additional details on model 2 in Table 2. Relative to Europe as a baseline, South America has a significantly greater capacity p=0.042, while PEM and SOEC have less capacity than ALK. This could be of potential interest to those interested in owning and operating green hydrogen plants. In particular, one could consider the profitability implications when contrasting the varying costs of electrolysis utilising PEM, SOEC and ALK, and a candidate plant’s respective output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*zHNLjNJx4bZh35RYkvzJfw.png" /></figure><p><strong>Geographic Variance:</strong></p><p>In this section, we study the geographic propagation of hydrogen plants around the world. We use same n=6 continent groups as previous sections. We wish to investigate the changing geographic spread of hydrogen plants and their energy production capacity with time. For that purpose, we convert these time series into rolling distributions.</p><p>In Figure 2a, we plot the time-varying geographic variance of grouped 5-year distributions. That is, this captures the geographic spread of new plants throughout a rolling 5-year period. As elsewhere in the manuscript, we are consistently interested in the analysis of green vs non-green plants; in this case, due to the comparatively low number of fossil plants, the plot is near identical if we consider all new plants or just all green plants.</p><p>Both figures 2a and 2b exhibit the same trajectory between 2004–2024, where geodesic variance increases until 2008, decreases until 2016, and then increases until 2024. Both figures start with an initially low geodesic variance in 2005, which is likely explained by Europe’s dominance in hydrogen plant propagation. During the initial increase, we see several plants appear throughout Asia, which leads to an increase in the variance. Between 2010–2015, the geodesic variance declines, which is likely due to East Asia’s levelling off in hydrogen plant propagation. In 2015–2016, East Asia experiences a significant boost in hydrogen plant commencement. This is largely due to China, whose first plant appears in 2017. Since then, China has accounted for numerous other hydrogen plants. We see a similar pattern for our two collections, all hydrogen plants and green hydrogen plants exclusively. This is predominantly due to the green hydrogen plants accounting for the vast majority of all hydrogen plants. The sharp rise in geodesic variance beyond 2016, may indicate the increased awareness of clean energy and decarbonisation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*i6IUN1JInvELhdClpZ0ovw.png" /></figure><p><strong>Linear estimates:</strong></p><p>To elucidate the changes in plants by continent, we plot the cumulative number of plants and fit linear trends to them in Figure 3. We highlight the results of Europe, North America, East Asia and other Asia in Figures 3a, 3b, 3c and 3d respectively. Europe, North America and Other Asia all display a relatively consistent linear trend throughout the entire period of analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*BtusZeev0S7vGkOWaRjheg.png" /></figure><p>However, East Asia’s cumulative number of plants is not modelled well by a linear fit. The cumulative number of plants is perhaps best modelled by a hyperbolic function, given the relatively constant level between 2000–2015, followed by rapid growth between 2015–2020, and another constant period during the early-mid 2020’s. These patterns, which demonstrate the consistency of hydrogen plant propagation over time, may indicate the evolution of each continent’s interest in decarbonisation and hydrogen production</p><p><strong>Distance Correlation:</strong></p><p>Next, we explore the distance correlation between North America and Europe in Figure 4. Distance correlation is not to be confused with the better known, more widely used Pearson correlation. Distance correlation captures linear and nonlinear associations between two random variables, while Pearson correlation can only detect linear relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*15qg80LrwoU6RgEIWUrFUQ.png" /></figure><p>Early on in our analysis window, the distance correlation is low — which is predominantly due to the sparsity of data. Beyond that, our time-varying distance correlation exhibits a local minimum in 2013. During this time, Europe and North America display profound differences in their concavity behaviours in new plants. During this period, Europe’s cumulative plants display a concave up shape, while North American cumulative plants display a concave down shape. Beyond this point, both North America and Europe exhibit relatively consistent linear increases, which is reflected in the high distance correlation between these two regions’ cumulative plants.</p><p><strong>Proportion of fossil to green plants with time:</strong></p><p>In Figures 5a and 5b, we plot the proportion of fossil plants and capacity, respectively, with time. We see that initially, there are no fossil plants, but as soon as the first appears in each continent, it dominates close to 100% of that continent’s hydrogen energy capacity. However, this dips to more reasonable proportions as we approach the end of the period of analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*cUY37oBH_apRkRBABqbOVg.png" /></figure><p>In Figure 5c, we display the time-varying matrix norm over our period. This total inconsistency is zero until 2011 due to either the complete lack of fossil plants (hence no inconsistency between the production vs number of fossil vs green plants) and then only Europe having fossil plants (so the adjacency matrices have a block diagonal structure):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/184/1*uXyOwxzQwOC2V77SO7eZMw.png" /></figure><p>Then in 2011, we see North America generate a fossil project which takes up 99% capacity as fossil and 20% of plants are fossil. This is first introduction of inconsistency between the continents in terms of their relationships between their proportions of fossil and green number of plants vs variance. Subsequently, when other continents start to roll out fossil-related projects, they account for most capacity and relatively small # plants, promoting further inconsistency in # fossil projects and their capacity. In 2016, both Other Asia and East Asia establish their first fossil plants.</p><p><em>Please note that these findings are highly dependent on the dataset we have used. Future studies on more up-to-date, and possibly richer datasets, could reveal further insights related to hydrogen plant propagation and their capacity. As always, please feel free to email me at nickj@arowanaco.com if you have any questions related to the white paper.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8643f6c067ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spatio-temporal trends in carbon dioxide emissions: 1970–2019]]></title>
            <link>https://njprivatemail.medium.com/spatio-temporal-trends-in-carbon-dioxide-emissions-1970-2019-86a2ac173d19?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/86a2ac173d19</guid>
            <category><![CDATA[climate-change]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Thu, 07 Oct 2021 01:17:04 GMT</pubDate>
            <atom:updated>2021-10-07T01:22:18.293Z</atom:updated>
            <content:encoded><![CDATA[<p>In this white paper, we introduce new frameworks to study spatio-temporal patterns among 50 countries in carbon dioxide emissions, demographic trends and economic patterns over the past 50 years. Our analysis is broken up into 3 sections. First, we do a decade-by-decade study of carbon dioxide emissions trajectories. There, we demonstrate notable changes in cluster structures in each time period. This highlights shifts between country emissions profiles over time. Next, we introduce a new method to classify countries into one of three characteristic emissions classes: one segment, two segments, or three segments. Here, we highlight that most countries are best represented by a piecewise linear model with one change point. This suggests that most countries experienced two periods of characteristic emissions during our analysis window. Finally, we use GDP, population and carbon dioxide data, and apply dimensionality reduction and clustering to group countries based on similarity in their real and carbon economies. This technique is a new way of viewing similarity between countries capturing data over 50 years, capturing economic and emissions data.</p><p><strong>Decade-by-decade emissions trajectories:</strong></p><p>In this section we investigate country emissions trends on a decade-by-decade basis, exploring the evolutionary structure of country behaviours. Both the number of clusters and cluster constituents are dynamic, with both varying as we proceed forward in time.</p><p>First we take the period from 1970–1979, where we observe a predominant cluster, consisting of 3 sub-clusters, and a small collection of outlier countries (primarily displayed as separate sub-clusters within the predominant cluster). The dendrogram corresponding to this period is shown in Figure 1a. All sub-clusters within the predominant cluster produce increasing trends of carbon dioxide emissions over the decade. The first sub-cluster consists of countries such as Russia and the Ukraine. These countries are characterised by huge growth, and accelerating emissions trends. The second sub-cluster, consisting of Italy, Canada and Argentina exhibit moderate growth in carbon dioxide emissions. The final cluster which includes countries such as Spain and Brazil also displays relatively steady growth behaviours. Countries such as Vietnam and Kuwait exhibit declining carbon dioxide emissions, which is anomalous with respect to the rest of the collection.</p><p>Next we turn to 1980–1989, shown in Figure 1b. This period produces a dendrogram consisting of 3 distinct clusters. The first cluster contains countries such as India, China etc. which display continued growth throughout the period. The second cluster consists of a variety of countries, but displays pronounced similarity between Eastern European countries such as Kazakhstan, Russia and Ukraine. These countries all experienced huge growth in emissions, peaking in around 1990. The final cluster consists of France, Belgium and Nigeria. These countries display erratic emissions behaviours, with all trajectories displaying limited trend and substantial volatility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0NniBfBPDhHQFw6M2oJxYQ.png" /></figure><p>In the 1990–1999 period shown in Figure 1c, countries form one primary cluster, with a small collection of outlier countries in a separate, significantly smaller cluster. The primary cluster consists of countries which displayed consistent growth over the prior decade (to varying extents). The small (outlier) cluster consisting of countries such as Ukraine, Russia and Kazakhstan, all experienced precipitous drops in their emissions at the beginning of the period — and this behaviour continued throughout the remainder of the time period. The latent phenomenon the cluster formation captures here is the fall of the Iron Curtain countries.</p><p>The 2000–2009 period produces one primary trajectory cluster (consisting of two similarly-sized sub-clusters), and a collection of outlier countries. The hierarchical clustering results are displayed in Figure 1d. The clear bifurcation in the large cluster is indicative of contrasting trends in emissions behaviours between various countries. The first sub-cluster consists of countries such as the Netherlands, Italy, Germany, USA, Canada, Japan and Belgium. Most countries within this sub-cluster are more developed, and have taken a stronger stance in introducing policies to reduce emissions. Such countries exhibit either flat or declining emissions trajectories throughout the decade. The second sub-cluster consisting of countries such as India, Iran and Turkey, produce sustained growth in emissions during this period. The significantly smaller second cluster, consists of countries who produce emissions profiles more like that of the second sub-cluster. This cluster consists of Oman, China, Vietnam and Qatar.</p><p>Finally we turn to the most recent decade of analysis, 2010–2019, which is shown in Figure 1e. This period produces 3 characteristic classes of emissions trajectories. The first cluster consists of countries with lower HDI (human development index) levels: Vietnam, Bangladesh, the Philippines, Iraq and Pakistan. HDI is a summary metric representing average achievement in various areas of human life. These countries produce emissions trajectories that increase significantly throughout the period. The second cluster consists of Venezuela, Ukraine, the UK, Italy and UAE. These countries mostly produce declining trajectories, which may represent greater collective focus on reducing carbon dioxide emissions at the national level. The final cluster consists of two primary sub-clusters. The first contains countries such as Japan, the Netherlands and Belgium. These countries are primarily characterised by erratic emissions output, with a declining trend overall. The second sub-cluster consists of countries such as China, Thailand and Chile. Although these countries produce a positive trend over the entire decade, the rate of increase has slowed, and in some cases, overall emissions have begun to trend downward.</p><p><strong>Characteristic emissions classes:</strong></p><p>In this section we introduce a new method to classify countries based on their emissions trends over time. Having noticed that many countries exhibit a piecewise linear trend in their CO2 emissions, we introduce a new framework to determine the most appropriate model for each country. <br>We assume that each country’s emission behaviours over time is best represented by one of the three following models:</p><ul><li>Model 0 (M_0): No change points (one linear component).</li><li>Model 1 (M_1): 1 change point (two piecewise linear components).</li><li>Model 2 (M_2): 2 change points (three piecewise linear components).</li></ul><p>In our algorithmic procedure, we optimise with respect to the number and placement of (up to two) change points, such that our function’s average R² is maximised. We compute average R², taking an arithmetic average of all segment’s R². Given our preference for a simpler model in the case where a more complex one accounts for a similar level (or marginally more) explanatory variance, we introduce a slight penalty for model complexity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lTzZEUqQGPiwAUx9XCwxwg.png" /></figure><p>The most frequently occurring model is M_1, followed by M_2 and then M_0. That is, in the vast majority of cases, carbon dioxide emissions are best modelled with the existence of either one or two change points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EUnNot5JWtbj2vsELUBtwQ.png" /></figure><p>In Figure 2, we display two representative countries for each model. One can see that in some cases, such as Algeria in figure 2a and Germany 2b, there is one persistent trend over the past 50 years. For Algeria, there is a consistently positive trend in emissions, while in Germany, emissions are best modelled by a linear decline over the past 50 years. In the case of Egypt (figure 2c and Morocco 2d, both countries produce a more strongly positive linear slope beyond the late 1980s and early 1990s respectively. Thus, these phenomena are best modelled with one change point. In the case of the Netherlands and India, seen in Figures 2e and 2f respectively, both emissions patterns require two change points to model the dynamics. For the Netherlands, there are obvious discontinuities and abrupt changes in total emissions, seen in the early 1980s and early 2000s respectively. While in India, the early 1980s and mid 2000s display obvious increases in the slope of carbon dioxide emissions.</p><p><strong>Identifying similarity in real and carbon economies:</strong></p><p>In this section we introduce a new method to identify countries who share similar economic, demographic and emissions histories. First, we gather GDP, population and CO2 emissions data over the past 50 years, and generate distance matrices between all countries for each respective measurement. We then generate an aggregate distance matrix, where each metric is scaled by a constant, such that we normalise for the scale of each metric in our aggregate distance. We then apply multi-dimensional scaling to the distance matrix, projecting the matrix into a lower dimensional data space, and apply K-means clustering to the resulting projection. The number of clusters, <em>K</em>, is determined by optimising the silhouette score. For the sake of interpretability, we accompany these K-means clustering results with a hierarchical clustering dendrogram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ufDPQsOaynhxU8TUsVrFTg.png" /></figure><p>When we apply this analysis to all 50 countries, the USA exhibits highly anomalous behaviour. In fact, we must sequentially remove the USA, China, India, Russia and Japan to identify the general structures without these outlier countries. In the proceeding analysis, we remove these 5 countries, and study the structural patterns among the remaining countries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nuQlRNdBH2zxugoOzYDxSA.png" /></figure><p>First we explore the similarity between countries with respect to each metric individually. Figure 3 displays the three distance matrices for GDP, population and C02 emissions. In figure 3a, one can see that Germany, France and the United Kingdom are the most anomalous countries with respect to GDP based on their significant economic output over the past 50 years. In figure 3b, we see distance between country populations, where Indonesia, Brazil and Bangladesh exhibit the greatest dissimilarity to the rest of the collection. In figure 3c, Germany exhibits the greatest dissimilarity to the remainder of the collection of countries.</p><p>For the sake of exposition, we present the hierarchical clustering results of our distance matrix capturing similarity in countries’ real and carbon economies. Figure 4 highlights the existence of one prominent cluster, a small sub-cluster and an outlier. The primary cluster consists of three sub-clusters. The first of which contains countries such as: Chile, Iraq, Vietnam, Romania, Bangladesh, etc. Most countries in this cluster are characterised by developing economies, growth in population, and significant growth in carbon dioxide emissions over time. The second sub-cluster consists of countries such as the Netherlands, Argentina, Nigeria, Taiwan, etc. These countries mostly displayed consistent growth in GDP and population, and lower levels of carbon dioxide emissions over time. The final sub-cluster consists of Spain, South Korea, Indonesia, Mexico, Canada and Turkey. Most of these countries displayed moderate growth in GDP and population, and growth in carbon dioxide emissions, exhibiting reasonable variability over time. In particular, many of these countries experience a flattening in emissions over the final 5–10 years of the analysis window. The second cluster includes the UK, France, Italy and Brazil. These countries are characterised by increasing emissions trajectories until the later parts of the analysis window, where a flattening or decline in emissions occurs. Germany is identified as an outlier, which is due to its steady decline in emissions over time and the strong GDP growth over the entire analysis window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_TzgdW4KXmtnWH1_x354uQ.png" /></figure><p>This analysis may probe further interesting studies. First, one could explore the key industries and sectors of the economy that drive emissions. This may vary significantly between countries, or perhaps, select industries drive the majority of emissions. Next, one could explicitly study the correlation between population growth and emissions, and identify countries that have done the best job in controlling emissions levels relative to their population growth. Figure 5 displays the emissions profile of 5 major countries: China, USA, India, Russia and Japan. Basic visual inspection confirms the pronounced similarity between China and India’s emissions profiles, while the USA and Japan share broadly similar emissions histories. Russia’s sharp decline in emissions in the early 1990s, corresponds to the fall of the Iron Curtain. One could speculate that these emissions are highly related to GDP — and studying this relationship explicitly could provide salient insights.</p><p><strong>Conclusion:</strong></p><p>We apply recently introduced and new methods in spatio-temporal data analysis to identify structural similarity and evolutionary patterns in emissions behaviours over time. First, we demonstrate pronounced heterogeneity in cluster number, size and constituency among emissions trajectories on a decade-by-decade basis. This section highlights the dynamic nature of this problem and the need for constant monitoring of country behaviours. In the second section of this white paper, we demonstrate that most countries are well modelled by a piecewise linear data generating process. The consistency of this finding is surprising, and would be interesting to monitor moving forward. Should countries more actively police their emissions, one could expect to see further propagation of change points in the near term. Finally, we introduce a framework to group countries based on their real and carbon economies. Our methodology captures GDP, population and emissions data over the past 50 years, and includes dimensionality reduction and clustering. This framework could be applied to other problems, or one could generalise some of the questions explored in this paper using different economic and demographic metrics.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86a2ac173d19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quantifying non-fungible token electricity costs via simulation]]></title>
            <link>https://njprivatemail.medium.com/quantifying-non-fungible-token-electricity-costs-via-simulation-a8a3d7bb7a6?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8a3d7bb7a6</guid>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Mon, 13 Sep 2021 06:00:34 GMT</pubDate>
            <atom:updated>2021-09-13T06:04:33.595Z</atom:updated>
            <content:encoded><![CDATA[<p>There is unquestionable interest surrounding the production and sale of non-fungible tokens (NFTs). An NFT is a digital asset that represents ownership in real-world objects (these can vary from art, to music, etc.) A recent sale of the artwork <em>“Everydays: The first 5000 days”</em> for $69m made news headlines. NFTs are produced on the Ethereum blockchain, and among other criticisms, they are notorious for their significant energy consumption. In fact, the production of one NFT (inclusive of minting, bidding, selling, etc.) is purported to require ~369 kWh of electricity. However, the landscape is set to change imminently. Ethereum 1.0 is in the process of transitioning to Ethereum 2.0 (Eth2) in Q1/Q2 2022, and in doing so, will move from a Proof-of-work (POW) to a Proof-of-stake (POS) consensus system. In doing so, energy requirements are estimated to drop somewhere in the order of 100x to 10,000x (although we believe that the former is indeed more likely). Accordingly, NFTs are likely to reduce their environmental impact quite significantly. But how great will this reduction be? Given the uncertainty in many of our model parameters, this is a difficult metric to quantify, and one should account for uncertainty in parameter values, as well as the interplay of such parameters. In this note we seek to:</p><ul><li>Provide an overview of the variability in electricity costs across the United States.</li><li>Discuss some of the expected changed in the transition from Ethereum 1.0 to Ethereum 2.0.</li><li>Explore some of the key differences in POW/POS consensus protocols.</li><li>Introduce a simulation-based framework to estimate the distribution of potential NFT cost paths over the next year.</li></ul><p><strong>Retail electricity costs in the United States:</strong></p><p>As at April 2021, the average price a residential customer in the USA would pay for electricity is ~ 13.31 cents/kWh. However, there is significant variability in electricity costs over time, and perhaps more importantly, there is pronounced variability in the cost of electricity between states. The state with the lowest electricity rate is Louisiana, paying an average of 9.53 cents/kWh. Below, we display the spatial distribution of electricity costs across the United States.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0_l_EoIurS1x2RSJBecHwg.png" /><figcaption>Cost of retail electricity prices across the United States</figcaption></figure><p><strong>Changes in the move from Ethereum 1.0 to Ethereum 2.0:</strong></p><p>Ethereum 2.0 refers to a collection of upgrades which are designed to enhance the scalability, security and sustainability of the Ethereum platform. The vision includes:</p><ul><li><strong>Improved scalability: </strong>Managing 1000s transactions per second, and making applications run fast, quickly, with an intuitive feel</li><li><strong>Security: </strong>Building protocol protection against various forms of attack</li><li><strong>Sustainability: </strong>Reducing the use of power and energy in the platform</li></ul><p><strong>Proof of stake (POS) vs Proof of work (POW):</strong></p><p>The table below outlines the key differences between the POS and POW consensus protocol methodologies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RfZ58zAcTA5EfLj4zbUQUQ.png" /></figure><p><strong>Uncertain variables surrounding NFT energy consumption:</strong></p><p>When considering the cost of NFT production over the proceeding year, one notices that it is a function of several parameters.</p><ol><li><strong>Time of the switch from Ethereum 1.0 to Ethereum 2.0.</strong></li><li><strong>Magnitude of reduction in energy usage when switch to Ethereum 2.0 occurs.</strong></li><li><strong>Cost of electricity.</strong></li><li><strong>Energy consumption required to produce an NFT.</strong></li></ol><p><strong>Estimating NFT cost via simulation:</strong></p><p>To explore the most probable cost of NFT production over the proceeding year, we design a simulation framework to study the distribution of NFT cost paths. To account for the uncertainty in aforementioned variables 1–4, we treat key parameters as random variables and run 25,000 simulation to explore the distribution of possible scenarios. Given the uncertainty in our parameters, we make the following assumptions regarding model parameters:</p><ul><li>The metric of interest is a total cost path for producing 1 NFT per day for 1 calendar year.</li><li>We study the calendar year from 1/9/2021–31/8/2022.</li><li>The cost of electricity is treated as a Gaussian random variable and is assumed to be centred at 3.5 cents/kWh (our Arowana estimate of a wholesale electricity rate).</li><li>Average kWh per NFT is treated as a Gaussian random variable, and is assumed to be centred at 369 kWh/NFT.</li><li>The magnitude of the reduction in average kWh per NFT post Ethereum 2.0 is treated as a log-uniform random variable with a lower bound of 100x and an upper bound of 1,000x.</li><li>The number of days until Ethereum 2.0 transition is treated as a Gaussian random variable, and assumed to be centred 210 days away (end of Q1 2022).</li><li>The cost of producing 1 NFT = # kWh required for NFT production x Cost of electricity.</li></ul><p>One can see the most probable parameter values for 1) # days until Ethereum 2.0 activation and 2) the magnitude of the reduction in electricity production after the transition below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-feFPUvkLkxdVT_KzH2imA.png" /><figcaption>25,000 samples from prior distributions to demonstrate probability densities.</figcaption></figure><p>The simulation procedure integrates over a range of possible scenarios, taking account of their interactions, and producing paths of NFT costs. The sharp reduction in costs (with varying magnitudes) most prominently reflects the timing and impact of the switch to Ethereum 2.0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*1LvFt_xR6gkCnRgo67mjrA.png" /><figcaption>25,000 sample cost paths for NFT production model</figcaption></figure><p>Our simulation procedure suggests that, for one concerned with the electricity cost of producing 1 NFT per day for the proceeding year, it will likely <strong>cost ~ $2,715. </strong>One should note that this result is highly dependent on the underlying model assumptions, and should one have strong prior beliefs regarding parameter values the simulation can easily be adjusted to account for this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*hYD-09kyJhldnpmFZIKOTg.png" /><figcaption>Total cost distribution for daily NFT production for 1 year.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8a3d7bb7a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evolutionary correlations and collective dynamics in the cryptocurrency market]]></title>
            <link>https://njprivatemail.medium.com/evolutionary-correlations-and-collective-dynamics-in-the-cryptocurrency-market-32fa51d1f282?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/32fa51d1f282</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[financial-markets]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Mon, 26 Jul 2021 03:42:37 GMT</pubDate>
            <atom:updated>2021-07-26T03:52:42.014Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Overview:</strong></p><p>In this note I introduce new methods to study behaviours among the 52 largest cryptocurrencies between 01–01–2019 and 30–06–2021. I explore evolutionary correlation behaviours and apply a recently proposed turning point algorithm to identify regimes in market correlation. Next, I inspect the relationship between collective dynamics and the cryptocurrency market size — revealing a strong negative relationship between the size of the market and the strength of collective dynamics. This finding has meaningful implications for the future of the cryptocurrency market, regardless of the asset class’ ultimate importance in investor portfolios. My findings indicate that if the cryptocurrency market grows, behaviours will become more heterogeneous, and if the market shrinks, behaviours will become more homogenous.</p><p>Over the last several years, the cryptocurrency market has attracted substantial interest from both institutional and retail investors. The market has experienced significant growth in total assets, accompanied by commensurate levels of volatility. Since the COVID-19 and BitMEX market crash in early 2020, cryptocurrency prices had a strong rally, followed by a subsequent decline in prices across the market. Given the differing views on the asset class’ long-term viability, I wanted to study evolutionary behaviour in cryptocurrency correlation structure and collective dynamics, the relationship between total market size and the strength of collective behaviours, the nature of time-varying anomalies, and the persistence of regimes in market volatility.</p><p><strong>Market behaviours and correlation regimes:</strong></p><p>Like many asset classes in financial markets, cryptocurrency returns are characterised by distributions that exhibit significant tail risk. Given their relative infancy and polarising views of the asset’s long-term viability by financial market participants, this makes their price behaviour more susceptible to extreme volatility and erratic behaviours. Figure 1 displays the general volatility in log returns and Figure 2 highlights the negative skew in the returns distribution throughout our analysis window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*p4TKD3pHDOj1eeZl2xhvcw.png" /></figure><p>To explore the temporal evolution of collective strength in correlation behaviours, I examine changes in the correlation structure. To do so, compute the time-varying L¹ norm of the correlation matrices at all points in time. To identify regimes of correlation behaviour, a Savitzky-Golay filter is applied to smooth out the minimum-adjusted series of L¹ norms, yielding a smoothed function. A turning point algorithm is then applied to the smooth function, to generate a set of turning points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*o3tuU7yQPj-nM7tVRngSxQ.png" /></figure><p>Figure 2 demonstrates that there are 6 turning points in correlation behaviours. Of particular note, are the turning points identified at 08–05–2020 and 13–08–2020. These dates reflect the COVID-19 market crash, and the subsequent recovery in the cryptocurrency market. Clearly, correlations are most strongly positive during the market crash, and most dissimilar during the subsequent growth in the cryptocurrency market. The turning point on 30–06–2020 reflects a strong increase in correlation behaviours during the latter part of our analysis window. This is most likely indicative of the aggressive sell-off in cryptocurrency assets from (approximately) mid-April through till present day. Again, a striking pattern is revealed where the general growth of the cryptocurrency market is inversely related to correlations. The identification of such peaks and troughs in correlations may provide investment managers with a decision support tool as to whether they are in a market environment where security selection is of greater or less importance. In market environments where correlations are strongly positive, it may be more difficult for investment managers to produce return streams that exhibit lower market beta.</p><p>To further inspect this finding, we partition our analysis window into 5 periods and construct an empirical correlation matrix within each of these windows. We plot the density of these correlation matrix entries in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*7dmAunT5A5VqLie-Sp2aPA.png" /></figure><p>Figure 3 displays kernel density estimates of the elements in our 5 correlation matrices. The findings are generally consistent with Figure 2. With the exception of the COVID-19 market crash, most market behaviours are shown to be highly similar. One finding of slight surprise is the cryptocurrency bear market partition, which exhibits correlations more like the Pre-COVID, Post-COVID and bull market windows. It is quite possible that with a longer analysis window and a slightly different date range, this distribution would produce a significantly higher average correlation score.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*p4SqUAf7kgDSiQkMYGC_Mw.png" /></figure><p><strong>Collective dynamics, fragmentation and market size:</strong></p><p>Next, I study the evolutionary dynamics and collective dynamics of our cryptocurrency collection. To do so, I apply principal components analysis (PCA) to the time-varying correlation matrices, studying the change in cryptocurrency dynamics over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*ecv85LMMZzPMspD7nq80Og.png" /></figure><p>Figure 4 shows the explanatory variance accounted for by this first eigenvalue, during our window of analysis. There are two findings of primary interest. First, and consistent with earlier experiments, there is a spike in this normalized eigenvalue during early 2020. This reflects the highly correlated behaviours of cryptocurrencies and indiscriminate selling during the COVID-19 pandemic. Subsequently, the explanatory variance exhibited by the first eigenvalue declines until May 2021. This (most evident) decline corresponds to the cryptocurrency bull market, where total cryptocurrency assets grew by several orders of magnitude. One obvious drawback from this technique, is that the correlation measured at any point in time is representative of the underlying dynamics for the prior 90 days (our window length). That being said, there is an interesting trend toward the end of our analysis window which warrants further investigation. There is a significant rise in the explanatory variance, corresponding to higher correlations in the 90-day rolling correlation matrix. This increase occurred contemporaneously with the aggressive sell-off in cryptocurrency assets, suggesting that there may be a relationship between the size of the market and the strength of the underlying collective dynamics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*6VSlXRmBHslr0PDjxPuJLw.png" /></figure><p>To investigate, we explore the relationship between the size of the cryptocurrency market and the collective strength of the market. We take a multivariate time series of cryptocurrency market sizes, and sum over them at every point in time (giving us a proxy for total market size). Due to the significant volatility exhibited by the market, we compute a rolling average of this total, using the same 90 day window. This relationship is shown in Figure 5, where an apparently negative relationship is revealed. To quantify this observation, we compute the correlation between the rolling size of the cryptocurrency market, and the collective strength of the market. The correlation between these two series is equal to -0.558, suggesting that as the cryptocurrency market grows in size the strength of collective dynamics decline.</p><p>This is a noteworthy finding with important implications for the future of the cryptocurrency market, especially given the divided views on the future of the market’s viability. Consider the (contrived) case where one of two scenarios exist; the former (bull case) where cryptocurrencies prices recover and it becomes a systemically important asset class, and the latter (bear case), where cryptocurrency prices continue to decline and they lose the interest of institutional investors. Our findings indicate that in the case of the former, behaviours are likely to become increasingly fragmented and heterogeneous, and there will be opportunities for highly skilful security selection to generate portfolio alpha. In the latter case, where prices decline and the size of the market decreases, behaviours are likely to become more homogenous and there will be less opportunities for alpha generation through security selection, as correlations will be strongly positive.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32fa51d1f282" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A case study and use cases for drone technologies]]></title>
            <link>https://njprivatemail.medium.com/a-case-study-and-use-cases-for-drone-technologies-3d44e84a8c1a?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d44e84a8c1a</guid>
            <category><![CDATA[drones]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Wed, 26 May 2021 00:21:45 GMT</pubDate>
            <atom:updated>2021-05-26T00:28:20.299Z</atom:updated>
            <content:encoded><![CDATA[<p>Given the growing interest in drone technologies, we wanted to examine their applicability and present our findings in a Sydney-based case study. We have partnered with Aerologix <a href="https://www.aerologix.com/">https://www.aerologix.com/</a>, who have created a platform for drone aerial vision. To highlight their platform’s functionality, we have captured and analyzed traffic footage taken at the intersection of Francis Street and Bondi Road (in Bondi Junction, Sydney, NSW, Australia). In this note, we outline some of the insights generated by our analysis and suggest other possible application areas for such technology.</p><p><strong>Experiment details</strong><br>Our experiment captured traffic information at the intersection of Francis Street and Bondi Road in Bondi Junction, Sydney, NSW, Australia. Drone pilots captured footage for one hour per day between 15/3/2021 and 28/3/2021 (15/3/21, 16/3/21, 17/3/21, 19/3/21, 22/3/21, 23/3/21, 24/3/21, 25/3/21, 26/3/21, 27/3/21, 28/3/21). Footage was captured at 9:00am on all dates, for the sake of consistency in our analysis. We study the consistency and topology of traffic patterns over our observation window. We highlight several insights below.</p><p><em>Consistency in vehicle behaviours</em><br>First, we study the similarity in the trajectories of various types of vehicles over our analysis window. In particular, we highlight the types of vehicles which present the most heterogeneous behaviours in the 11 hours of footage. We separate our traffic footage into a collection of vehicle types: buses, cars, heavy vehicles, medium vehicles and motorcycles. On each date, we sum the total number of vehicles observed within each interval, at our defined sampling rate. We then study the similarity in each vehicle type’s normalised trajectory over time. That is, for each vehicle type we generate a distance matrix based on the difference in its trajectory behaviour on various dates. Such analysis highlights which vehicles display the greatest variability in their trajectories on different dates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*1eYUJ7RMIn9qYIboBz1P2A.png" /></figure><p>Several insights can be made based on our analysis (the accompanying images are in Figure 1): <br>1. Buses and heavy vehicles are the two vehicle types that display the greatest variability in their trajectories. The latent phenomenon this may have identified is less activity from heavy vehicles and buses on the weekend.<br>2. Cars, medium-vehicles and motorcycle exhibit significantly less variability in their normalised trajectory behaviours.<br>3. Car trajectories are shown to be most different on 15/3/21 and 23/3/21. These dates correspond to a Saturday and Sunday, and reflect differences in the community’s driving patterns when contrasting weekdays and weekends.</p><p>Although these findings seems trivial, merely confirming commonsense, in many contexts we will have a less intuitive understanding of the problem phenomenology and such insights could prove to be valuable.</p><p><em>Topology of traffic composition changes</em><br>Next, we explore the patterns of vehicle composition collectively over our analysis window. We compute the norm of the difference in trajectory matrices between all possible dates, measuring all vehicle types’ traffic patterns, examining the topology of the behaviour manifold. Our analysis, seen in Figure 2, demonstrates that although the most heterogeneous behaviours are exhibited on the weekend — there is a general consistency in traffic patterns, regardless of the day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/1*G7WjO1v2UKB6K5lIBqK9AQ.png" /></figure><p><strong>Applications of drone technology</strong><br>Our case study above provides a small example of where drone technologies and aerial footage may be deployed to yield valuable insights regarding traffic patterns. If this were applied over a longer time period, or at a specific location, the insights could indeed be more powerful. In particular, one could study the periodicity of such behaviours and identify long-term trends in community traffic patterns. That being said, there is a wide variety of potential areas where drone technologies can be applied at present:<br><strong>1. Community behaviours:</strong> One potential application area of great relevance is monitoring human behaviours. For instance, one could track community movements in regions where there COVID-19 lockdowns are in place. This could help governments and enforcement agencies make data-driven decisions in real-time during the pandemic.</p><p><strong>2. Asset due diligence:</strong> One could use footage to study assets or areas of interest. Factories, ports and commodity storage facilities are all good candidates for such analysis.</p><p><strong>3. Environmental Modelling: </strong>Drone footage such as this could be used during environmental crises, such as the Australian bushfires. More generally, drones can capture weather-related information.</p><p><strong>4. Property development:</strong> One could study the patterns and movement among communities and traffic around potential areas of development.</p><p><strong>5. Leisure:</strong> Many people seem to be interested in filming themselves, or their surroundings, in all sorts of contexts.</p><p><strong>Contact:</strong><br>For anyone interested in learning more about Aerologix drone technologies, please don’t hesitate to contact the Aerologix team, Tom Caska (<em>tom@aerologix.com</em>) and Rakesh Routhu (<em>rakesh@aerologix.com</em>). For those interested in the data science methodology, please feel free to email me at <em>nickj@arowanaco.com</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d44e84a8c1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Changing dynamics of market crises: a review of country, sector and equity behaviours over the…]]></title>
            <link>https://medium.com/data-science/changing-dynamics-of-market-crises-a-review-of-country-sector-and-equity-behaviours-over-the-1933e107fdc6?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/1933e107fdc6</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[financial-markets]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[equity]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Wed, 21 Apr 2021 23:14:48 GMT</pubDate>
            <atom:updated>2021-04-29T01:42:02.260Z</atom:updated>
            <content:encoded><![CDATA[<h3>Changing Dynamics of Market Crises: A Review of Country, Sector, and Equity Behaviours Over the Past Two Decades</h3><h4>A mathematical study of evolutionary financial market behaviours</h4><p>In this paper, we study the behaviours and similarity profile of country financial indices, sector financial indices and equities over the past 20 years. First, we introduce a new methodology to determine the similarity of 20 countries and 11 sectors with respect to their time series’ structural breaks. We introduce a new metric that can quantify distance between any two candidate sets, where a probability distribution determines the uncertainty surrounding various elements’ locations. Next, we turn to US equities, where we administer three separate experiments. First, we analyse the evolution of equity crisis correlations over time, highlighting the increase in equity similarity in recent years, and in times of market panic. Next, we investigate the collective similarity of equity trajectories over time and demonstrate greater distance between trajectories during times of crisis. Finally, we explore the dynamics of our equity market collection, via a time-varying principal component analysis. We highlight current areas of difficulty in spreading investments among assets exhibiting homogenous behaviours, and suggest approaches that may improve investor diversification.</p><p><strong>Motivation of this study:</strong></p><p>In March-June 2020, global markets exhibited extreme behaviours due to the COVID-19 crisis. Investors struggled in the midst of declining asset prices and volatility, as many `traditional’ diversification strategies did not appear to be effective. In fact, intra-asset correlations seemed to be particularly high — even relative to previous market crises. This triggered our curiosity, and we thought it warranted a thorough investigation.</p><p>Despite the wealth of data available to financial market participants, there is a tendency for many investors to focus on the recent past. This is sensible. Market dynamics are continuously changing, and although history may rhyme, it certainly doesn’t repeat itself and financial market behaviours in 2020 are most unlikely to be identical to those in 2000; as many aspects of financial markets have altered substantially over the past two decades. This paper is written with a different purpose. We wish to study how country, sector and equity behaviours have evolved over the past 20 years — especially during times of crisis. We introduce new methodologies to examine changes in erratic behaviours, trajectories and time-varying dynamics.</p><p>We break our analysis into two parts. First, we study the similarity in structural breaks among countries and sectors. This analysis could be useful for any asset allocator or investor looking to diversity their investments with respect to country or sector exposure. Second, we study S&amp;P 500 equities, and explore the evolution of correlations, trajectories and time-varying dynamics. Insights from this work could be used by any global equity investor interested in the dynamic phenomenology exhibited by equities over the past 20 years.</p><p><strong>Data:</strong></p><p>Our country and sector data spans 4/1/2002–8/10/2020 and 4/1/2000–8/10/2020 respectively, and is measured on a daily basis. The countries we analyse include: Australia, Brazil, Canada, China, France, Germany, India, Indonesia, Italy, Japan, Korea, the Netherlands, Russia, Saudi Arabia, Spain, Switzerland, Turkey, United Kingdom and the United States. The sectors we analyse include: Communications, Consumer Discretionary, Consumer Staples, Energy, Financials, Healthcare, Industrials, Information Technology, Materials, Real Estate and Utilities. Our equity data spans 3/1/2000–8/10/2020. We study all S&amp;P 500 equities in existence from the date of inception until the end of our analysis window. This leaves us with 347 equities. All of our data is sourced from Bloomberg.</p><p><strong>Erratic behaviour modelling:</strong></p><p>In this section we study the similarity in the erratic behaviours of 20 country indices and 11 financial market sectors. To determine similarity, we apply a change point detection method to each time series and generate sets of change points. Importantly, we use a methodology that detects change points based on changes in the power spectrum. This alleviates issues that many change point detection methodologies face, when confronted with dependent data. Prior work has demonstrated that the methodology applied in this paper appropriately detects change points in piecewise autoregressive processes, highlighting the algorithm’s ability to partition dependent data. For those interested in the methodologies applied, please see one of the following papers:</p><p><a href="https://www.tandfonline.com/doi/abs/10.1080/01621459.2012.716340">https://www.tandfonline.com/doi/abs/10.1080/01621459.2012.716340</a></p><p><a href="https://arxiv.org/pdf/2003.02367.pdf">https://arxiv.org/pdf/2003.02367.pdf</a></p><p>As dependence is often the case in financial time series, we use the aforementioned technique. This methodology is implemented in a Bayesian framework, and captures uncertainty around candidate change points. We adjust a previously introduced semi-metric to account for uncertainty surrounding candidate change points, when measuring distance between sets.</p><p><em>Country similarity:</em></p><p>The erratic behaviour cluster structure of the country financial indices suggests that there is one predominant cluster of countries, and several other diffuse clusters. The primary cluster consists of USA, Korea, Turkey, UK, France, Italy, Switzerland, Netherlands, Canada, Australia, Germany and Saudi Arabia. The collection of outlier countries includes: Russia, Brazil, Spain and China. These cluster memberships and associated change points may be due to substantial similarity in the geographic location and political association of such countries, leading to similar market behaviours during events such as Brexit. Notably, Spain is determined to be markedly less similar than its European counterparts. The most anomalous country within the collection is China. This is unsurprising, as many of the idiosyncratic restrictions related to their local equity market may provide different dynamics to those exhibited by other financial markets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/533/1*Vhpx81Z6QxOFybYrcIKMFQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*fF5zH4Ww8R4K2nwBdbQ6IA.png" /></figure><p><em>Sector similarity:</em></p><p>The <em>sector erratic behaviour dendrogram</em> consists of 2 clearly separated clusters. The primary cluster consists of all stocks except materials and consumer staples. Interestingly, the utilities, communications and IT sectors exhibit no structural breaks in the period between 2012–2019. By contrast, several change points are detected in the materials sector time series during this time window. Change points are also detected between 2012–2019 in the sector time series within the predominant cluster (financials, consumer discretionary, etc.). Figure 4 displays the log returns and detected change points for the communications, utilities and materials sectors. One can see the stark difference in the propagation of structural breaks in the period between 2012–2019.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*vSaPPyTMO5MRmAkgTRD30w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*RaxK2W5QsT2Uovaqh4iYJQ.png" /></figure><p><em>Comparison:</em></p><p>To compare the degree of similarity between countries and sectors, we compute the matrix norms of our erratic behaviour matrices, and normalise them by the number of elements in the matrix (as there is a different numbers of countries/sectors in our previous experiments). The country erratic distance matrix norm is higher than the sector erratic distance matrix norm. The smaller value in the sector norm suggests that the erratic behaviour profile is much more similar among sectors than it is among countries. This could suggest that diversification benefits are more substantial to investors diversifying with respect to countries (as opposed to sectors).</p><p><strong>Equity analysis:</strong></p><p>In this section, we study 347 US equities over ~ 20 years. We break our analysis into three separate sections. First, we examine the behaviour of correlation coefficients during various discrete (and distinct) time periods. Next, we study the collective similarity in rolling equity trajectories. Finally, we explore the time-varying eigenspectrum — as a means of highlighting periods in time where the `market effect’ is most pronounced (indicated by the size of the leading eigenvalue).</p><p><strong>Correlations:</strong></p><p>In this section, we partition our analysis window into 5 periods: GFC, Peak GFC, Interim (the period between the GFC and COVID), COVID, and Peak COVID. The date windows for each time partition are as follows:</p><ul><li>GFC: 1/1/2007–31/5/2010</li><li>GFC Peak: 2/9/2008–1/6/2009</li><li>Interim: 1/6/2010–28/2/2020</li><li>COVID: 2/3/2020–31/8/2020</li><li>COVID Peak: 2/3/2020–29/5/2020.</li></ul><p>Within each period, we compute the correlation matrix for all equities in our collection, and plot the distribution of correlation coefficients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*aeixQ3T0BU2-VoTRKp8J1A.png" /></figure><p>Figure 5 presents two findings. First, there is clearly an increase in the correlation of equities during times of crisis. This is reflected in both the GFC and COVID-19 market crashes, but this spike in correlation is even more pronounced at the peak of these crashes. That is, the Peak GFC and Peak COVID distributions are centred further to the right than the GFC and COVID distributions respectively. Second, this increase in correlations has become more pronounced in recent times. Figure 5 suggests that equity behaviour has become more similar during times of crisis. The implications of this finding are alarming, and suggest that traditional Markowitz-inspired portfolio diversification frameworks have to be altered, at least within the context of pure equity portfolios.</p><p><strong>Rolling trajectory analysis:</strong></p><p>In this section we study the rolling similarity in equity trajectories. To do so, we normalise all trajectories by their L_1 norm, and compute distances between all candidate trajectories on a 45 day rolling basis. Figure 6 demonstrates that periods of crisis provides the most heterogeneity in trajectories. In particular, the dot-com bubble, GFC and COVID-19 market crisis all display spikes in the rolling norm computation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*tmdJj0BZ5ZBoGoM6xkw9tg.png" /></figure><p>Although there is an increase in trajectory distance matrix norms during periods of crisis, the direction of trajectories are still largely uniform. During crises, the overwhelming direction of equities is downward. The uniformity is confirmed in our correlation analysis, where correlation coefficients are more strongly positive. This is interesting.</p><p><strong>Time-varying PCA:</strong></p><p>Finally, we implement a time-varying principal components analysis, where we plot the percentage of explanatory variance exhibited by each eigenvector. Given that the first eigenvector represents the collective strength of the market, when there is a spike in the first element of the eigenspectrum it is indicative of an increase in the strength of collective market behaviours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*CuB-hg6oj05bSOmn6gFpwQ.png" /></figure><p>Figure 7 shows spikes in the first element of the eigenspectrum during the dot-com bubble, GFC and the COVID-19 market crisis. This confirms our earlier findings in the study of correlations, indicating that market dynamics are markedly more homogenous during times of crisis.</p><p><strong>Discussion and limitations:</strong></p><p>Our erratic behaviour modelling highlights sectors and countries that behave most similarly and dissimilarly. Of the 20 countries analysed, there is one predominant cluster that consists primarily of European countries. The sector erratic behaviour analysis finds 1 primary cluster among the 11 sectors analysed. Materials and consumer staples are found to be most dissimilar to other sectors in terms of their erratic behaviours. Our (normalised) matrix norm analysis suggests that it may be more difficult to diversify across sectors than countries. This is highlighted by the larger value for the country erratic distance matrix norm. Our correlation analysis provides two insights. First, equity correlation has increased over time. Second, equity correlation spikes during market crises. Our trajectory analysis and rolling PCA indicates that although trajectories show the most heterogeneity during crises, collective market dynamics are strongest during these periods. Such findings could be of interest to global equities and international investors, concerned with understanding market dynamics and diversifying their investment portfolio.</p><p><em>I have kept mathematical discussion/notation to a minimum for the sake of accessibility. For those interested in the mathematical details, please feel free to email me at nickj@arowanaco.com</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1933e107fdc6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/changing-dynamics-of-market-crises-a-review-of-country-sector-and-equity-behaviours-over-the-1933e107fdc6">Changing dynamics of market crises: a review of country, sector and equity behaviours over the…</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pandemic impacts and the efficiency of the real and financial economies]]></title>
            <link>https://njprivatemail.medium.com/pandemic-impacts-and-the-efficiency-of-the-real-and-financial-economies-a09c6a160914?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/a09c6a160914</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Tue, 23 Feb 2021 20:39:31 GMT</pubDate>
            <atom:updated>2021-02-23T20:45:55.196Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Overview:</strong></p><p>In this paper, we undertake a mathematical investigation of 20 countries to answer three questions in relation to the evolution of the COVID-19 pandemic, the real economy and the financial economy. We study three multivariate time series measuring country’s daily mobility data, daily COVID-19 cases, and financial index performance. First, we analyse which countries behave similarly with respect to each of these features independently. Second, all countries’ trajectories are studied in conjunction to determine if any two candidate features exhibit more pronounced similarity. We demonstrate that mobility data and financial indices exhibit the most similarity in their trajectories. Finally, we study whether there is a difference in the time taken for COVID-19 cases data to be reflected in financial index performance and economic activity. A new optimisation method is introduced which demonstrates that mobility data tends to lag financial market performance, highlighting greater efficiency among financial market participants than governments in considering and acting upon new COVID-19 data. This finding suggests that studying mobility data as a leading indicator for financial market performance during the pandemic was, at least in the short-term, misguided.</p><p><strong>Motivation for the study:</strong></p><p>During the pandemic there was a surge of interest in alternative data among the investment management community. One of these data sources was Apple mobility data. The basic hypothesis here, was that mobility data may be a leading economic indicator and provide insights into the probable movement of financial indices. The initial motivation for this study, was seeking to determine whether such mobility data provided any utility to investment managers in their estimating future performance of respective financial indices. The countries included in this study are: Argentina, Australia, Brazil, Canada, Denmark, France, Germany, India, Italy, Japan, Mexico, the Netherlands, New Zealand, Norway, Russia, Singapore, Spain, Switzerland, United Kingdom and United States. For each country, we analyse three time series (COVID-19 new cases, financial index performance, mobility data). Our analysis uses data starting 1/1/2020 and finishing 30/12/2020. The data sources are Bloomberg, Apple Mobility (<a href="https://covid19.apple.com/mobility">https://covid19.apple.com/mobility</a>) and Our World in Data (<a href="https://github.com/owid/covid-19-data/tree/master/public/data">https://github.com/owid/covid-19-data/tree/master/public/data</a>).</p><p><strong>Cluster behaviours:</strong><br>In this section we study the independent behaviours of COVID-19 new cases, mobility activity and financial index performance among our 20 countries of interest. In each section, we generate a distance matrix between the normalised trajectories of each country’s respective feature. We apply hierarchical clustering to each distance matrix to study the groupings and candidate similarity among countries. A summary of clustering results is below.</p><p><em>COVID-19 cases:</em><br>The COVID-19 cases trajectory dendrogram is comprised of 2 clusters, a large predominant cluster and a small cluster of outlier countries (New Zealand, Australia, and Singapore). The larger cluster is characterised by new case trajectories exhibiting multiple waves of new cases, with many countries experiencing growth in cases toward the end of 2020. The common theme among the countries in the anomalous cluster, (consisting of New Zealand, Australia and Singapore) is their low total number of cases throughout 2020, and their suppressed growth in cases toward the end of 2020.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*U-B1htLijLginEiCBNNEQg.png" /></figure><p><em>Mobility activity:</em><br>The mobility activity trajectory dendrogram is comprised of 2 more equally-sized clusters. The first cluster includes: Mexico, Brazil, Australia, Argentina, Singapore, India, New Zealand and Japan. Mobility trajectories within this cluster are consistently increasing (or, in select instances, flat) among these countries. These trajectories are representative of countries that experienced waves of COVID-19 cases earlier in 2020, imposed restrictions to reduce case numbers, and gradually reopened their economies. The second cluster consists of the United Kingdom, Spain, the United States, Canada, the Netherlands, Italy, Switzerland, Germany, France, Denmark, Russia and Norway. All mobility trajectories for countries within this cluster possess declining trajectories in their mobility. This is almost certainly due to government restrictions in respective countries, which were in response to significant growth in COVID-19 cases and deaths. Interestingly, the two clusters exhibit geographical affinity. The first cluster consists of countries in South America and Oceania, while the second cluster comprises European and North American countries. This highlights the similarity in the timing of lockdowns among many countries in proximal geographies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*qPwRgYjxRg1_XMjVUVVifA.png" /></figure><p><em>Financial index performance:</em><br>The financial trajectories dendrogram consists of 2 clusters, a primary cluster and a smaller anomalous cluster. The anomalous cluster consists of the United Kingdom, Mexico, Spain and Singapore. The outlier cluster is comprised of countries that share similar financial index trajectories. Specifically, there is a large drop in March, a subsequent drop in November and a slight to moderate recovery during the latter half of November and December. Interestingly, all of these countries’ financial indices finish 2020 at a level lower than their pre-COVID position. The dominant cluster which consists of the remaining countries, exhibits far less uniform trajectories. However, many countries in the dominant cluster experienced more substantial recoveries than the anomalous cluster. For instance, the United States and New Zealand (which possess highly similar trajectories) both post consistent gains after their precipitous drops in March.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*6cpQ2gtZD1QuxnKCtVUCwg.png" /></figure><p><em>Trajectory associations and interplay</em><br>The preceding analysis highlights several noteworthy points regarding the consistency in country cluster behaviours in reference to the above features.</p><ul><li>There is no consistency in cluster structure among COVID-19 cases, mobility activity and financial index performance. The size of the clusters is highly variable. However, in all dendrograms there are 2 clusters.</li><li>There is no consistency in cluster composition. In both the COVID-19 cases and financial index performance dendrograms there is a small, anomalous cluster. There is limited intersection between the countries in each anomalous cluster, highlighting a lack of consistency in the way that countries naturally group — when clusters are formed with respect to different metrics’ trajectories.</li><li>There is clearly no relationship between countries’ financial index performance and their mobility or COVID-19 trajectories. The most obvious example of this is Singapore. Hierarchical clustering highlights that Singapore managed COVID-19 cases anomalously well, while its financial index performance was anomalously poor. By contrast, the United States leads the world in COVID-19 cases and deaths and exhibited very strong financial index performance. This is seen in Figure 4.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F2Bt_7OhK-KEbkPIvlREfg.png" /></figure><p><strong>Trajectory similarity:</strong><br>Next, we seek to determine collective similarity among country attributes that we have studied. To do so, we generate a 60 x 60 matrix measuring the normalised inner product between all attributes’ trajectories, and apply hierarchical clustering to the resulting distance matrix. The results can be seen in Figure 5. There is obvious structure in this dendrogram. Financial and mobility activity trajectories are highly similar, while COVID-19 case trajectories are significantly less similar. This makes sense. Both the mobility and financial index trajectories exhibit similar shapes; there is generally a sharp drop around March, followed by a subsequent recovery in activity and gains respectively. COVID-19 cases on the other hand exhibit dissimilar trajectories. First, COVID-19 case trajectories between countries are much more dissimilar — influenced by many factors such as the government’s willingness to enforce restrictions, the restriction of travel and human movement, etc. Second, COVID-19 case trajectories are prone to `wave’ behaviour, where there may be multiple peaks and troughs throughout the evolution of a country’s 2020 case numbers. Interestingly, there is no tendency for trajectories to be grouped based on country. For example, the Japan COVID-19 cases, financial index performance and mobility trajectories do not cluster similarly. Given the collective phenomenology, one would naturally wish to understand whether COVID-19 cases impact financial indices or mobility activity first — and whether this varies among countries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*_qtstXpXZVD4Z7v6673ejA.png" /></figure><p><strong>Offset analysis:</strong></p><p>Given the similarity in the behaviour of mobility activity and financial index performance, we now compare the efficiency of governments and financial market participants in their use of COVID-19 data for decision-making. To do so, we introduce a new inner product-based optimisation framework, where the objective function is to estimate an offset between any two trajectories such that their inner product is maximised. Effectively, this aligns the major peaks and troughs in any two candidate trajectories. In this model, we assume that either mobility activity or financial index performance may be the laggard. The offset results displayed in Table 1 reveal several findings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*JkLx-nb3AiCfwp9eLyzzlg.png" /></figure><ol><li>In general, mobility trajectories lag financial index trajectories. In fact, only 3/20 countries possess trajectories where their mobility activity reflects COVID-19 case information more quickly than the respective financial index.</li><li>There is significant variability in the offset among countries. The range of offsets is 16 days. Spain and Switzerland have offsets of only 2 days, while Japan has an 18 day lag between its financial index and mobility activity trajectories.</li><li>The 18 day mobility lag in Japan suggests that COVID-19 information was captured far more efficiently by financial markets than the Japanese government. This is made clear in Figure 6.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gxhpKPAEo1PDGI3Qtb8gnQ.png" /></figure><p><strong>Discussion and limitations:</strong><br>All studies have their limitations, and this one is no different. Out analysis could be improved in several ways. First, one could study more granular financial and mobility data. It is possible that analysing data that is aggregated at the country and financial index level respectively, we may be ignoring subtleties in the data and may fail to capture important relationships. That being said, there are several interesting findings in this work. First, we demonstrate that there is very little consistency in cluster behaviours or constituents when studying the trajectories of COVID-19 cases, mobility activity and financial index performance. This was most clearly evidenced by Singapore, who managed their COVID-19 case numbers particularly well and had particularly weak performance in its financial index. By contrast, the United States experienced huge COVID-19 case (and death) numbers, while the country index produced strong performance after the market crash in March. Next, we demonstrate that financial index and mobility trajectories are most similar, with no tendency for trajectories to be grouped around their respective countries. Having established this similarity, our new optimisation method demonstrates that mobility trajectories generally lag financial index trajectories. This suggests that throughout the pandemic, financial market participants incorporated COVID-19 case numbers into their trading decisions faster than governments were able to act on this information by way of instituting restrictions.</p><p><em>I have tried to keep mathematical notation to a minimum for the sake of readability. If you have any questions related to the above article, please feel free to contact me at nickj@arowanaco.com</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a09c6a160914" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evolving cryptocurrency and equity market dynamics]]></title>
            <link>https://njprivatemail.medium.com/evolving-cryptocurrency-and-equity-market-dynamics-e30ba7398cf8?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/e30ba7398cf8</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[financial-markets]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Mon, 11 Jan 2021 01:40:57 GMT</pubDate>
            <atom:updated>2021-09-06T14:18:59.394Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding the dynamics of cryptocurrencies in relation to traditional asset classes</h4><p>This note introduces mathematical methods to study the evolutionary dynamics of cryptocurrencies. I wanted to understand whether the behaviours of cryptocurrencies and equities have become more similar in recent years, and in particular, whether this similarity changes during market crises. In this short note, I study the time-evolving dynamics and correlation structure of both collections.</p><p>For the sake of readability, I have tried to limit mathematical discussion, and instead I have focused on a few key ideas and results. For those interested in the full exposition of the methodologies presented, the full academic paper has been published in the journal Physica A: <a href="https://www.sciencedirect.com/science/article/pii/S0378437121001035">https://www.sciencedirect.com/science/article/pii/S0378437121001035</a></p><p><strong>Data:</strong></p><p>In the proceeding analysis, I study cryptocurrency and equity multivariate time series between 03–12–2018 to 08–12–2020. I analyse the 45 largest cryptocurrencies by market capitalisation (excluding those with anomalous price behaviours such as USDT) and 72 global equities whose market capitalisations are greater than US$100 billion. In select sub-sections, I partition the analysis window into 3 periods:</p><p>Pre-COVID: 03–12–2018 to 28–02–2020,</p><p>Peak COVID: 02–03–2020 to 29–05–2020, and</p><p>Post-COVID 01–06–2020 to 08–12–2020.</p><p>Data is sourced from <a href="https://coinmarketcap.com/">https://coinmarketcap.com/</a> and Bloomberg.</p><p><strong>Collective market dynamics:</strong></p><p>First, I study the time-varying eigenspectrum for both collections, to reveal changes in the dynamics of both asset classes. There is a wealth of literature studying correlation dynamics in financial markets, particularly during market crises. However, I am unaware of two asset classes’ dynamics being compared explicitly, and their difference quantified, during various market regimes (bull market, bear market, etc.). I introduce a measure to do this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*Vk6-ROnsMkNQUsitiWw7ig.png" /></figure><p>Generally speaking, the first eigenvector of the correlation matrix can be thought of as the impact of “the market” on any collection of securities. A larger first eigenvalue (the scaling associated with the corresponding eigenvector) is indicative of a stronger market force. I examine how this impact changes over time in both of our collections. The figures above display the % explanatory variance exhibited by the first 10 eigenvalues at every point in time for our cryptocurrency and equity collections. I use a rolling window of 60 days to compute the time-varying correlation matrix, and respective eigenspectrum. There are several noteworthy insights below.</p><ol><li>Both figures (1a and 1b) display a similar shape, where the majority of variance is provided by the first several eigenvectors. Beyond that, the remaining portion of total variance falls away quickly over the entire window of analysis. There is much greater consistency in the size of the first eigenvector exhibited by the cryptocurrency collection.</li><li>The explanatory variance provided by the first cryptocurrency eigenvector is consistently higher than the corresponding equity market eigenvector. This demonstrates that the collective force of the market is much more pronounced in cryptocurrencies than that of equities. Anecdotally, this is consistent with many market commentators who suggest that cryptocurrencies’ behaviour is primarily driven through price movements while equities are often driven through earnings announcements, company news, etc.</li><li>There is significant variability in the explanatory variance provided by the first eigenvector of the equity collection. This highlights that within equity markets, there is more variability related to the market’s impact on collective behaviour than that of cryptocurrencies. The time-varying explanatory variance of the first principal component is displayed in Figure 2 (below). One can see that the level of explanatory variance attributed to the first eigenvector is generally ~ 15-20% higher for cryptocurrencies, except during the peak of the COVID-19 market crisis, where these measures are somewhat comparable. This finding highlights the increase in the impact of the market on both collections during market crises.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*RMfzatCnMLcV_4uxtniadA.png" /></figure><p><strong>Dynamics similarity in varying market regimes:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w3mhPldy6Io9v5UZzxjraQ.png" /></figure><p>Based on the partition above, I split the analysis window into three periods, and compare the dynamics and correlation behaviours within each period. To quantify the difference in the dynamics, I introduce a measure which I refer to as a ‘Dynamics deviation’ score. It is computed by summing over the absolute element-wise difference in the normalised 2-dimensional eigenspectrum (which varies over time and ordered eigenvalues). The normalised eigenspectrum captures the % variance explained by each eigenvalue, and is ordered from largest to smallest. The measure is then normalised for a) the number of securities in each collection, and b) the length of the time period. The computation is shown below (for more details, see the main paper):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*11zbjLKoGspScPO5w9iBIg.png" /></figure><p>The dynamics deviations scores in each period are highlighted in the table below. The markedly smaller deviation during the Peak COVID period, highlights much more similarity in the behaviours of cryptocurrencies and equities during market crises. Given that the first eigenvector accounts for such a significant proportion of total variance, it is clear that during times of crisis, the collective weight of market selling is the most important determinant of security variance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*YQmPd9mTKxAjT1CFJjztLQ.png" /></figure><p><strong>Correlation structure in varying market regimes:</strong></p><p>Finally, I explore the behaviour of correlations during our three different analysis windows to determine collective similarity in varying market regimes.</p><p>Figure 4 (below) displays kernel density estimates of the off-diagonal correlation matrix elements within each period. Understanding how the distribution of correlation coefficients vary in different markets is of great interest to those concerned with portfolio dynamics, and mitigating risk in volatile market conditions. There are three interesting insights from this analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aFBU1qdOjKShW-rRZk2Usg.png" /></figure><ol><li>Cryptocurrencies display higher correlation levels than that of equities. During Pre-COVID and Post-COVID market conditions, the modal correlation value was ~ 0.5 for cryptocurrencies, and ~0.1–0.3 for equities.</li><li>This strong correlation among cryptocurrencies causes difficulties within traditional portfolio optimisation frameworks. Let us consider a portfolio optimisation routine which captures portfolio risk in computing the objective function (eg. maximising a portfolio’s Sharpe Ratio). The objective function in this routine builds upon the traditional Mean-Variance framework, and aims to spread capital across investments such that there is an optimal trade-off between portfolio risk and return. With such high levels of correlation among the collection, those investing exclusively in cryptocurrencies may find it hard to diversify systematic cryptocurrency risk away. By contrast, equities have a lower level of correlation across the collection, and do in fact exhibit some negative correlation coefficients. One would think that diversifying across an exclusive collection of equities to reduce systematic risk is an easier task than that of cryptocurrencies.</li><li>Finally, cryptocurrencies and equities behave most similarly during market crises. The modal correlation coefficient for the cryptocurrency collection is ~ 0.8, while the modal correlation coefficient for the equity collection is ~ 0.7. This highlights the phenomenon of highly correlated drawdowns across various asset classes during market crises — indiscriminate selling from investors is the primary driver of such dynamics.</li></ol><p><strong>Summary:</strong></p><p>Cryptocurrencies are shown to exhibit stronger collective behaviours and correlations than that of equities, and this is amplified during non-crisis periods. This is demonstrated through the time-varying eigenspectra and correlation studies. Those investing exclusively in cryptocurrencies should consider including other asset classes in their portfolio. In the event of another cryptocurrency market correction such as the one seen in early 2018, asset classes other than cryptocurrencies should provide better diversification benefits.</p><p><em>Please feel free to contact me at nickj@arowanaco.com if there are any questions related to the above.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e30ba7398cf8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[COVID-19 second surge phenomena and associated equity market impacts]]></title>
            <link>https://njprivatemail.medium.com/covid-19-second-surge-phenomena-and-associated-equity-market-impacts-11dee0bc6ec?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/11dee0bc6ec</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[stock-market]]></category>
            <category><![CDATA[covid19]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Sun, 09 Aug 2020 21:35:13 GMT</pubDate>
            <atom:updated>2020-08-09T21:41:59.463Z</atom:updated>
            <content:encoded><![CDATA[<p>Countries continue to battle with COVID-19. Although various countries are at different stages of their struggle against the virus, there is significant similarity in the behaviour of various countries’ local equity indices. This article studies the evolution of COVID-19 and the associated equity market impacts. There are three separate components to our analysis. First, we study the changes in COVID-19 cases and deaths across countries over time with a dynamic clustering algorithm. Second, we determine distances between countries’ COVID-19 trajectories based on their critical points. We use these distances to group countries, and highlight groups of countries displaying similar phenomenology in the spread of COVID-19. Finally, we study the relationship between countries’ COVID-19 situation and their respective equity market performance. This highlights the changing dynamics of equity market index behaviours over time, and a gradual depression in investor sensitivity regarding COVID-19 news.</p><p><strong>Dynamics of COVID-19 evolution</strong><br>Countries around the world have battled the COVID-19 pandemic since its inception. The evolution of COVID-19 cases and deaths over time has been complex, with different countries experiencing variable severity in the growth of cases and deaths at various points in time. Anecdotally, we know that China was the first country to be severely impacted by COVID-19, followed by the likes of Italy and Spain, and more recently, Brazil, India and the United States. We implement a dynamic clustering algorithm, applied to cumulative case and death counts, to dynamically group countries into clusters over time. This serves several purposes:<br>1. We can study various countries’ change in cluster membership over time to assess their success in managing the spread of COVID-19. <br>2. Second, we can study the evolution of the number of clusters over time. This provides some insight into the dynamics of the virus and how varied its effect is on the world at any point in time.<br>3. Analysing these results in conjunction provides some understanding of the time-varying relationship between COVID-19 cases and deaths.</p><p><strong>Cluster membership over time</strong><br>Figure 1 displays time-varying cluster memberships for various countries based on their total case and death counts. The case clusters range from 2–17 over time while death clusters range from 1–17. A darker colour represents membership in a more badly affected cluster. Figure 1a confirms the natural history of the virus’ spread. First, China was the only country to be grouped into cluster 1. However, as the virus began to spread internationally and case numbers grew exponentially, countries’ cluster memberships changed rapidly. Italy and Spain were the first two countries to move into cluster 1 (in March), followed by the United States and the United Kingdom (in April). Due to a myriad of factors involved in governments’ management of the virus, there has been a wide range of outcomes among these countries’ case and death totals. Most notably, China’s excellent job in containing the virus moved them back to cluster 3 by the end of our analysis window, while countries such as Brazil, India and Mexico moved into the worst-affected cluster. Figure 1b displays countries’ cluster membership with respect to total deaths over time. As at 26/7/2020, China, Italy and Spain had moved out of the worst-affected death cluster while countries such as the United States, Brazil, India and Russia had moved into the worst-affected death cluster. Another notable takeaway is the contrasting position of the United Kingdom with respect to cases and death clusters. The UK’s belonging to a lower-ranking case cluster than death cluster is indicative of better management of total cases than total deaths. This finding is supported by the UK’s anomalously high death rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*atsF2nqCTXjgkmYU9x25gg.png" /><figcaption>Figure 1: (a) Daily cases evolution and (b) daily deaths evolution. Cluster evolution over time. Darker colours represent membership in a more badly affected cluster. Figure 1(a) studies the evolution of cases over time, (b) studies the evolution of deaths over time.</figcaption></figure><p><em>Number of clusters over time and relationship between cases and deaths</em><br>Prior work has demonstrated remarkably similar trajectories in the evolution of the number of clusters when studying cases and deaths across countries and over time [1] up to a suitable offset. Figure 1 displays the number of case and death clusters, adjusting for the offset. The general shape of these two trajectories is highly similar. In particular, the trajectories share a sharp rise in cluster numbers until a peak of ~ 16/17 is reached, followed by a gradual decline to ~ 10 and ~ 13 case and death clusters respectively. Both trajectories appear to stabilise around mid-June. The fact that there are more death clusters than case clusters may be indicative of more varied performance across countries in their ability to stop the progression from COVID-19 cases to deaths.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*vwfxvseXvLDCUI6FZbk10A.png" /><figcaption>Figure 2: Cases and deaths evolution adjusting for estimated offset. Both trajectories have sharp increases in their number of clusters, followed by a steady decline, finally reaching respective steady-states.</figcaption></figure><p><strong>Second surge analysis</strong><br>One of the defining attributes of COVID-19 is its high risk of contagion. This has resulted in many countries’ inability to stop the growth in new case numbers. A common theme among countries who have managed to contain the virus is the implementation of harsh restrictions, these include; the closure of restaurants and bars, stay at home orders, curfews etc. Among countries who had managed to reduce new case growth, there is now a notable dichotomy between a) countries who have managed to keep case numbers down and b) countries who appear to be exhibiting a <em>second-surge</em>.</p><p>To better understand the similarity among various countries’ growth trajectory in new case numbers, we determine the distance between various countries’ peaks and troughs and apply hierarchical clustering to the resulting distance matrix (for details regarding our distance computation between sets of critical points please see [2]).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3sBYKZpNf4NeKOFVw1kvHw.png" /><figcaption>Figure 3: Critical point distance dendrogram</figcaption></figure><p>The dendrogram in figure 3 determines the existence of three clusters of countries; two smaller collections and one predominant cluster consisting of two subclusters. The first cluster consists of countries such as Brazil, South Africa and India. These are countries still in their first surge of COVID-19. These countries have never experienced a local maximum, indicating that the number of new cases is on the decline. These countries are not experiencing a second surge, as they are still firmly located within their first surge. The second cluster consists of countries such as the Netherlands, Germany and Italy. There are countries that experienced spikes in their new case growth around April, and have since experienced significant declines in the growth of new cases. These countries are not experiencing a second surge. The final predominant cluster has two dominant subclusters exhibiting strong self-similarity. The first includes countries such as Egypt, Pakistan and Russia. These are countries that experienced their first surge, and now have a declining trend in new case growth at the end of our analysis window. Finally, the other notable subcluster within the dendrogram consists of countries such as Israel, Switzerland and the United States. These are countries who experienced a peak in new cases, a subsequent decline, and are again experiencing growth in new case numbers. These countries are well into their second surge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YAL-3SECSLcZGSp7Fpl0rA.png" /><figcaption>Figure 4: 4 groups of characteristic time series. Cluster 1 examples (a) Brazil, (b) South Africa(c) India. Cluster 2 examples (d) Netherlands (e) Germany (f) Italy. Cluster 3 examples (g)Egypt (h) Pakistan (i) Russia. Cluster 4 examples (j) Israel (k) Switzerland (l) USA.</figcaption></figure><p><strong>Equity index analysis</strong><br> 1. Equity market performance is varied across equity indices. Results can be seen in Table 1 with the figure showing local equity market performance in figure 5. The best-performing equity index (measured by returns from 1/1/2020–17/7/2020) is China, and the worst-performing equity index is Spain. The median performance across these equity indices is -9.7 %.</p><p>2. In terms of peak-to-trough drawdown, the three worst-performing local equity indices are those of; Argentina (-50.1 %), Russia (-49.5 %) and Brazil (-46.8 %). The best performing equity indices are those of China (-27.3 %) and Switzerland (-27.5 %). This finding is of particular interest, especially when compared with the results of the cluster memberships in figure 1(a) and figure 1(b). Brazil and Russia’s entry into lower-ranking cluster case and death clusters occurs in April and May respectively. Meanwhile, the trough in their equity market performance occurs on the 23/3/2020 and 20/3/2020 respectively. This finding indicates, that even during these countries’ first surge in COVID-19 there was very little sensitivity between local equity market performance and the domestic COVID-19 situation (in terms of cases or deaths).</p><p>3. Earlier this year we published a paper examining the relationship between the real and financial economy during times of market crisis. We found that during market crises the drop in the real and financial economies was more or less coincident. This is likely due to investor psychology during stressful market situations, where herding behaviours and irrational selling are exacerbated. Table 1 and figure 5 highlight the striking similarity in the trajectories between all equity indices. In particular, the simultaneous sharp decline characterised by high volatility and low autocorrelation, and the subsequent recovery characterised by lower volatility and high autocorrelation. This finding suggests that during times of crisis, for investors seeking short-medium term alpha-generating opportunities via tactical asset allocation, there is limited upside in betting against a strong macroeconomic thematic. A better approach could be the identification of depressed equity market indices and/or sectors, and (such as Argentina, Brazil and Russia) thoughtful addition of exposure to distressed assets across the portfolio. <br> 4. Unsurprisingly, equity markets that experienced the largest peak-to-trough drawdown also exhibited the highest volatility in their returns. Argentina is of particular note, with a drawdown in excess of -50 %, proceeded by a strong recovery. Argentina’s annualised volatility was 63 % as at 17/7/2020.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ce_KodcGzWsiTlicEFh05Q.png" /><figcaption>Table 1. Returns, volatility, peak-to-trough drawdown, date of minimum equity index value and 2020 returns for 17 countries’ local equity market</figcaption></figure><p><em>Sensitivity to second surge</em><br>Finally, we study the sensitivity of local equity market indices to potential second surge effects in local countries. Figure 6 displays the rolling 30-day volatility for our 17 equity indices under inspection. Again, we see similarly-shaped trajectories with respect to equity index volatility behaviours. Specifically, there is a spike in rolling volatility in March 2020, which persists until the end of April. During the height of the crisis, the difference in annualised volatility between the most (Brazil) and least (China) volatility equity indices is 77 %. Beyond that point, volatility levels stabilise across all equity markets. At the end of our analysis window, the range of volatility levels (Switzerland and Argentina) is 29 %. Looking at figure 4(j), figure 4(k) and figure 4(l), one can see that second surge phenomena in these countries commenced in late May and early June. It is reasonable to assume that if local equity market indices were sensitive to second surge phenomena, then there would be a change in the volatility behaviour of equity indices. However, when looking at figure 6, one can see the clear stability in equity index volatility from late May until July 17. In fact, the only country with a meaningful change in its volatility over this period is China, which did not experience growth in new COVID-19 cases during the period. One should also note that China’s volatility is driven by strongly positive recent equity market performance, rather than precipitous declines (as is often the case during times of crisis).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aYQ9fxSi5Hfw_ygARDET_g.png" /><figcaption>Figure 5: Equity market performance (1/1/2020–17/7/2020)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*00q61LLMe-7qSeoiXm7Igw.png" /><figcaption>Figure 6: Rolling volatility</figcaption></figure><p><strong>Conclusion and takeaways</strong><br>In this white paper, we propose two methods to understand the evolution of COVID-19 across countries and over time and use these insights to determine the impact on equity markets. Our dynamic clustering implementation confirms the natural history of the virus. Different countries are at various stages of their battle in containing the growth of cases, and the progression from cases to deaths. Hierarchical clustering on the distance between new case trajectory turning points allows us to make identify clusters of countries, where each cluster is characterised by a different stage of its battle with COVID-19. We identify four characteristic groups; (1) countries still in their first surge, (2) countries over their first surge and decline in cases with limited case growth, (3) countries over their first surge with declining case numbers and (4) countries over their first surge, experienced declining cases, and now experiencing their second-surge. We apply these insights to study equity market dynamics during COVID-19. We discover that there is no relationship between local equity market performance and local COVID-19 case or death numbers, and show that equity markets are exhibiting very little sensitivity to global second surge effects. These findings highlight the transient nature of investor psychology within current market dynamics. With such market dynamics in existence, it is likely that near-medium term alpha-generating opportunities will come from the marriage of finding depressed equity prices, and taking positions in line with the overall macroeconomic thematic.</p><p><em>We have tried to avoid overwhelming readers with an excessive mathematical background and detailed notation. Instead, we have tried to explain our analysis in relatively simple terms. For readers interested in the specific techniques applied, please feel free to contact me at nickj@arowanaco.com</em></p><p>References:</p><p>[1] Nick James and Max Menzies. Cluster-based dual evolution for multivariate time series: Analyzing COVID-19.Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(6):061108, June 2020.</p><p>[2] Nick James, Max Menzies, Lamiae Azizi, and Jennifer Chan. Novel semi-metrics for multivariate change point analysis and anomaly detection. Physica D: Nonlinear Phenomena, 412:132636, November 2020.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11dee0bc6ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Asymmetry and financial sector clusters during equity market crises: making inference during…]]></title>
            <link>https://njprivatemail.medium.com/asymmetry-and-financial-sector-clusters-during-equity-market-crises-making-inference-during-aad3acfde733?source=rss-31f0f5d507b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/aad3acfde733</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[financial-markets]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nick James]]></dc:creator>
            <pubDate>Tue, 23 Jun 2020 23:07:02 GMT</pubDate>
            <atom:updated>2020-06-24T02:51:53.830Z</atom:updated>
            <content:encoded><![CDATA[<h3>Asymmetry and financial sector clusters during equity market crises: making inference during COVID-19</h3><p>Forecasting financial market behaviours and understanding financial market dynamics based on empirical patterns in data is a difficult task. Properties of financial time series such as non-stationarity, heteroskedasticity, and low signal-to-noise regimes cause problems in the naive application of statistical learning approaches. The chaotic nature of financial markets and a more recent shift in investor interest from active to passive investment management has changed market dynamics. In this short note, we seek to identify sectors that behaved similarly and dissimilarly during prior market crises by measuring the distance between various sectors’ structural breaks. These algorithmic techniques are coupled with prior knowledge of financial market phenomenology to make inference on the interplay and relationship between real and financial economies during periods of stable market dynamics and market crises. Finally, we analyse the dynamics of the S&amp;P 500 constituents before and during the COVID-19 crises. Our analysis indicates that an increase in the magnitude of total equity market returns may present more opportunities to value-oriented investment managers who are able to exploit larger deviations in the magnitude of equity returns.</p><p><strong>Market crises and the interplay of real and financial economies</strong></p><p>Many financial market observers are aware, anecdotally, of a phenomenon where the recovery of financial markets may precede the recovery of the real economy. We seek to investigate the relationship between the real and financial economies during the Dot-com bubble and the Global Financial Crisis (GFC). In particular, we address the following questions:</p><ol><li>Is the starting point of the crash coincident between the real and financial economies?</li><li>How long does the crash last for both the real and financial economies? Is there a distinct difference between the real and financial economies, and does this difference persist during various crises?</li><li>Is there a meaningful distinction in the trajectory of the recovery for both the real and financial economies?</li></ol><p>We seek to answer some more general questions regarding financial market dynamics during crises. In different types of market dynamics, which sectors behave similarly? In particular, we focus on analysing the similarity between various sectors’ structural breaks. We then address two questions. First, what is the general degree of similarity across equity markets during various crises? Is there persistence in the types of sectors that behave similarly during different market crises? We follow methods proposed in James et al. (2019) to do so. Finally, we look at the evolution of equity returns during COVID-19 to try and better understand where market opportunities may present themselves in the near future.</p><p><strong>Clustering financial market sectors in various market regimes</strong></p><p>We seek to determine the similarity in the erratic behaviour within a collection of time series. We follow the notation of James et al. (2019) and follow their procedure to determine the distance between two time series’ structural breaks. Structural breaks, or change points, are indices in time where the underlying statistical properties of a time series change. Securities are re-based to 100 as at 1/1/1999 to provide some indication of the performance of such indices and asset classes prior to any respective crisis.</p><p><strong>Dot-com bubble: 10/3/2000–7/2/2003</strong><br>The dot-com bubble was initially caused by excessive speculation in technology companies. Although impacting the valuation of technology companies most significantly, the effects resonated across all equity markets. As a proxy for the underlying economic component driving the crash, we use the NASDAQ Index as a proxy. There are, of course, some issues in using the NASDAQ as a measure of performance for the real economy. First, being an equity index, it is impacted by passive money flows. It is not in fact representative of the real economy. Second, like all indices, there is a risk of index survivorship bias — only the strongest companies survive. Retrospective analysis can, therefore, be biased. Finally, as NASDAQ constituents will receive passive money flows from investors seeking exposure to a broad market portfolio, some share price recoveries may be more reflective of the general economy’s health rather than the technological economy, specifically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Rp1cQWrTBxx1PZ4fRiSGw.png" /><figcaption>S&amp;P 500 Index vs NASDAQ</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*pP5TrKowjHKXSESJf5cj-Q.png" /><figcaption>The NASDAQ vs S&amp;P 500 Index performance after the Dot-com bubble. The drop in the S&amp;P 500 index is less severe, and the recovery is more immediate. Indices do not start from the same level to provide an indication of prior performance before the analysis window.</figcaption></figure><p>The hierarchical clustering results in the dendrogram below provide several interesting insights.</p><ol><li>The hierarchical clustering groups the IT sector by itself. This is indicative of the nature of the Dot-com bubble, where technology companies were sold off most aggressively.</li><li>R/E, industrials, materials and financials stocks were all determined to cluster together. These sectors all experienced share price declines among their constituent equities, however, they were impacted much less severely than technology companies.</li><li>The behaviour of consumer staples and consumer discretionary stocks was most anomalous relative to the rest of the market. These two sectors were algorithmically determined to lie within the same cluster. This is an interesting insight. During this period, consumer staples stocks significantly outperformed consumer discretionary stocks — however, the similarity of their structural breaks indicates that the sectors still behave similarly with regards to erratic behaviour.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fmSjp-tpMD1KPw03TWFUZA.png" /><figcaption>Hierarchical clustering results for bull market run.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v5sRCnr-d1WzcgTYSIhkTg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YCpnDDgdsHYbp6Oo6lqu7g.png" /><figcaption>Hierarchical clustering for the GFC (left) and Dot-com bubble (right).</figcaption></figure><p>The general degree of similarity across equity sectors is not high. The significantly darker colouring in the hierarchical clustering output, as seen in the Dot-com dendrogram, indicates that there was more variability in the behaviour of structural breaks across equity market sectors during the Dot-com bubble. The marked difference in the general degree of similarity across equity market sectors between the Dot-com bubble and other crises (such as the GFC in the next section) sparks several ideas. First, the nature of the real economy’s crisis has great importance in how structural breaks manifest across equity sectors. The Dot-com bubble’s crisis more exclusively focused on technology companies. The GFC had more widespread consequences, impacting a variety of equity market sectors. This is confirmed by our analysis of the distance between sector structural breaks.</p><p><strong>Global financial crisis: 1/1/2008–9/3/2009</strong></p><p>The global financial crisis was triggered by a breakdown in the subprime mortgage market in the USA. Arguably the most profound catalyst for the crisis was the high default rate in the USA subprime home mortgage sector. Some factors argued to be most significant in causing the crash include: low-interest rates for a prolonged period, securitisation of mortgage-backed securities and limited regulation regarding predatory lending. Almost immediately proceeding the breakdown in the real economy, global financial markets entered a prolonged recession.</p><p>To study the relationship between the real and financial economy during the GFC, we use the percentage of US household debt outstanding as a proxy for the health of the US lending market and thus the real economy. We use the S&amp;P 500 index as a proxy for the performance of the financial economy. The table below and the figure below display some interesting findings regarding the GFC recovery.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*6D3X9ca_0y2ikSMM3YBa9Q.png" /><figcaption>S&amp;P 500 Index vs USA % debt outstanding</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*JadLKr0CSyusppu4KsHqzA.png" /><figcaption>USA % household debt outstanding vs S&amp;P 500 Index. Indices do not start from the same level to provide an indication of prior performance before the analysis window.</figcaption></figure><p>1. The start of the decline in both the real and financial economies is almost coincident. That is, the declining state of the US mortgage market was priced into the market efficiently.</p><p>2. The difference in the point in time of the local minima for the real and financial economies is very different and highly significant. The low point of the financial economy was experienced in early March 2009, while the low point of the real economy occurred in October 2012 — more than three and a half years later.</p><p>3. 5 and 10 years beyond the starting point of the GFC, the S&amp;P 500 index was 1.8% and 72.2% higher than the inception point. In contrast, the percentage of household debt outstanding was significantly lower 5 years on and still ~ 5% lower 10 years on.</p><p>Perhaps of even greater interest than the distinctly varying recovery dynamics between the real and financial economies during the GFC, is the study of sectors that behave similarly. The GFC dendrogram displayed above displays the results of hierarchical clustering applied to the distance matrix between various sectors’ structural breaks between the dates of the GFC. There are four noteworthy takeaways from such analysis.</p><p>1. The healthcare and communications sectors were least similar to the rest of the market in terms of their structural breaks. This finding seems more or less intuitive, especially for the healthcare sector.</p><p>2. The distance in the structural breaks of real estate and financial stocks is remarkably similar. This result confirms the actuality of the GFC, where equities with exposure to the financial and real estate economies performed poorly.</p><p>3. Consumer staples, consumer discretionary, industrials, materials and energy sectors all behaved similarly in terms of their structural breaks. Again, this result is indicative of the linear dependence exhibited across many equity sectors during this period.</p><p>The general degree of similarity across the market during the GFC is extremely high. This is evidenced in the significantly brighter colouring of the GFC hierarchical clustering. This indicates that structural breaks were significantly more similar during the GFC across all market sectors. This is an interesting insight for two reasons. First, it provides some insights into the systematic and indiscriminate selling across the market during the GFC. This is a surprising result. Since the GFC, many asset allocators have thoughtfully reallocated capital toward financial products with higher beta attributes. Specifically, passive and factor-based products. Seeing such striking similarity in the structural breaks across various equity sectors should make us particularly cautious of current market dynamics, where there appears to be significantly greater concentration risk in financial products that have a high correlation with the general health of the market. Second, the general similarity in the hierarchical clustering output may indicate that there is a high degree of linear dependence across equity sectors (up to a small deformation). This may indicate that general sentiment across equity markets was more universally negative across equity sectors.</p><p><strong>Crisis similarities</strong></p><p>Our analysis of the relationship between the real and financial economies during the Dot-com bubble and GFC is not without its limitations. However, we observe some similarities in the interplay between the real and financial economies during both crises — suggesting that there is perhaps a more general theme in this interplay during periods of market crisis. Both the Dot-com bubble and GFC experience a drop in the real and financial economies which is more or less coincident. Both the S&amp;P 500 and the NASDAQ experienced a sharp drop in their prices, albeit the trajectory of the fall in the NASDAQ was more aggressive than that of the S&amp;P 500. Also like the GFC, the recovery of the real economy significantly lags that of the financial economy as evidenced in the Dot-com bubble table and recovery chart. In fact, 10 years after the beginning of the crisis, the S&amp;P 500 had reached a level ~ 12.7 % higher than before the Dot-com bubble while a comparable measure of the NASDAQ ~ 54.4%.</p><p><strong>Bull market: 4/10/2011–5/10/2018</strong></p><p>Immediately proceeding the GFC global equity markets entered an unprecedented bull run. Having analysed the general degree of similarity across equity sectors and clusters of similarity during market crises, it is of great interest to see the similarities and contrasts in this behaviour during a period of buoyancy in the equity market.</p><p>1. IT, Real estate and consumer discretionary stocks are all identified as anomalous with respect to their structural breaks.<br>2. Utilities, communications and consumer staples are identified to have similar patterns in their structural breaks.<br>3. Industrials, energy and materials are also identified as a distinct cluster where the behaviour of structural breaks is similar.<br><br>The general degree of similarity across the market is reasonably low with regard to sector structural breaks. This may be due to the relatively consistent behaviour across equity markets during this period of analysis. In particular, the relatively low volatility and a higher degree of autocorrelation exhibited across equity market sectors may have led to structural breaks being driven more so by unsystematic phenomena rather than systematic market forces. The bull market hierarchical clustering indicates that the clear groupings in the equity sector clustering are beginning to dissipate. One possible explanation for this is the advent of factor-style investing over the past 10–15 years. This may have led to less similar behaviour between assets, particularly with regard to asset shocks. Another plausible explanation is that due to the nature of equity market gains and losses. Bullish equity markets are generally characterised by stable equity trajectories exhibiting minimal volatility. As a result during such market dynamics, structural breaks would be more prone to inherent dissimilarity.</p><p><strong>COVID-19 financial market analysis: VIX for value</strong></p><p>Having assessed market dynamics and behaviour during prior crises, we seek to determine the early effects of COVID-19 and the potential impacts on active management. To do so, we propose using a new measure to determine the overall variability among stock returns based on computing matrix norms over time. At each point in time, we compute a distance matrix to monitor the total variability in S&amp;P 500 constituents’ returns. We compute the Frobenius norm at each point in time. Monitoring the change in this market norm over time highlights periods where there is most variability in stock returns over time. One could interpret this measure as an indicator of potential alpha generation opportunities for value managers, or a ‘VIX for value’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*TanrjgQk5F5dfjdBITncqg.png" /><figcaption>Daily norms of S&amp;P 500 constituent returns. A spike in this measure during COVID-19 highlights more short-term deviation in equity market returns and potentially more opportunities for value investors.</figcaption></figure><p>The figure above has one primary takeaway. As we track the matrix norms over time, one would notice a spike in March and April of 2020. This is indicative of an increase in the magnitude of absolute returns, measured across all S&amp;P 500 stocks. As this norm increases, it indicates that there is increased variability in stock performance — both to the upside and to the downside. This is a striking contrast to the characteristics of the equity market we have observed over the past 10 years. This could be an early indicator of an opportunity set favouring opportunistic, value-style investors rather than quantitative and factor-driven investment approaches.</p><p><strong>Summary:</strong></p><p>Although each crisis presents differences in the similarity profile of various equity sectors, there are some common themes. First, the real economy and the financial economy appear to have coincident drops. However, the financial economy generally leads the real economy in their respective recoveries — returning to higher values significantly more quickly than the real economy. Having analysed the interplay of the real and financial economies and specific equity sectors during prior equity market crises, we analyse the impacts of COVID-19 on the S&amp;P 500 index. Our time-varying norm computation is one way of measuring the total variability within an equity market system. The sharp peak in our norm measure during February-April 2020 indicates that COVID-19 may present opportunities to generate alpha for value-style investors. That is, increased variability among stock returns and increased absolute magnitude among equity index constituents is likely to reward highly competent stock pickers, and possibly be even more damaging to returns for those unable to pick winners.</p><p><em>We have tried to avoid overwhelming readers with an excessive mathematical background and detailed notation. Instead, we have tried to explain our analysis in relatively simple terms. For readers interested in the specific techniques applied, please feel free to contact me at nickj@arowanaco.com</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aad3acfde733" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>