<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Richard L on Medium]]></title>
        <description><![CDATA[Stories by Richard L on Medium]]></description>
        <link>https://medium.com/@richardhwlin?source=rss-19f76801d2b9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*KmFD9alJCO4sO2wDxPquNg.gif</url>
            <title>Stories by Richard L on Medium</title>
            <link>https://medium.com/@richardhwlin?source=rss-19f76801d2b9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 14:19:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@richardhwlin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What I asked AI about — the Debt Ceiling]]></title>
            <link>https://medium.com/@richardhwlin/what-i-asked-ai-about-the-debt-ceiling-cfab8c8a5d4?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/cfab8c8a5d4</guid>
            <category><![CDATA[us-economy]]></category>
            <category><![CDATA[money]]></category>
            <category><![CDATA[prompt-investing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[debt-ceiling-deadline]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Thu, 25 May 2023 23:31:00 GMT</pubDate>
            <atom:updated>2023-05-25T23:31:00.537Z</atom:updated>
            <content:encoded><![CDATA[<h3>What I asked AI about — the Debt Ceiling</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_zmmUGJnlMBI8YeT" /><figcaption>Photo by <a href="https://unsplash.com/@jsshotz?utm_source=medium&amp;utm_medium=referral">Jorge Salvador</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This will be a brief series of discussions highlighting key subjects in the global capital market, with a particular focus on the current topic of the imminent US debt ceiling. Fortunately, I am now assisted by my AI companion, <a href="https://bard.google.com/">Google Bard</a>.</p><h3>Prompt 1: Debt ceiling update and its impact on the capital market: What makes it unique this time?</h3><p><strong>Bard says: </strong>The debt ceiling is the maximum amount of money that the United States government is allowed to borrow. When the debt ceiling is reached, the government cannot borrow any more money to pay its bills. This can lead to a government shutdown, which is when the government is unable to provide essential services to the public.</p><p><strong>The debt ceiling has been raised 80 times since 1960. </strong>If Congress does not raise the debt ceiling, the government will be forced to default on its debt. This would be a major financial crisis, and it would have a significant impact on the capital markets. When the government defaults on its debt, it will lose the confidence of investors. This will lead to a decline in the value of stocks and bonds, and it will make it more expensive for businesses to borrow money. This will have a negative impact on the economy, and it will lead to a decline in economic growth.</p><p>This time, the debt ceiling is so special because the United States is facing a number of economic challenges. <strong>The economy is still recovering from the COVID-19 pandemic, and inflation is at a 40-year high. If the government defaults on its debt, it will make it even more difficult for the economy to recover.</strong></p><blockquote><em>The editor’s note: Not entirely accurate, the US economy has moved beyond the impact of the pandemic, and signs of inflation fading have become apparent.</em></blockquote><p><strong>In addition, the debt ceiling is being used as a political bargaining chip.</strong> Republicans in Congress are demanding concessions from Democrats in exchange for raising the debt ceiling. This is creating uncertainty in the markets, and it is making it difficult for businesses to plan for the future.</p><blockquote><em>The editor’s note: The opposition party must consistently take this action to safeguard taxpayers’ funds! However, a more crucial concern is whether the US Treasury bonds will face default in the event of the government’s failure to increase the debt ceiling.</em></blockquote><h3>Prompt 2: Will <em>the US Treasury bonds will face default in the event of the government’s failure to increase the debt ceiling?</em></h3><p><strong>Bard Says:</strong> Yes, if the US government fails to increase the debt ceiling, it will be unable to pay its debts, including interest payments on Treasury bonds. This would be a default on the US debt, and it would have a significant impact on the economy and the financial markets.</p><p>If Congress does not raise the debt ceiling, the government will be unable to borrow any more money to pay its bills. This would force the government to make difficult choices about which bills to pay and which to default on. <strong>The government would likely default on interest payments on Treasury bonds, which would be a default on the US debt.</strong></p><p>A default on the US debt would have a significant impact on the economy and the financial markets. Investors would lose confidence in the US government’s ability to repay its debts, and this would lead to higher interest rates. Higher interest rates would make it more expensive for businesses to borrow money, and this would slow down economic growth. In addition, a default on the US debt would likely lead to a decline in the value of the dollar, which would make imports more expensive and would further slow down economic growth.</p><blockquote>The editor’s note: More or less, it is reasonably accurate, but it lacks specific details. It would be beneficial to obtain numerical information regarding the current status of the US government levels.</blockquote><h3>Prompt 3: What is the total value of US treasury debts affected by the debt ceiling crisis? What percentage does it represent in relation to the total US treasury debts? Furthermore, what is the projected overall loss that investors would face in the event of a US debt default?</h3><p><strong>Bard Says: </strong>The total value of US treasury debts affected by the debt ceiling crisis is unknown, as the amount of debt that the US government will be unable to pay if the debt ceiling is not raised is not known. However, the US government currently owes over $30 trillion in debt, and the debt ceiling is currently set at $28.9 trillion. <strong>This means that the US government could be unable to pay over $1 trillion in debt if the debt ceiling is not raised.</strong></p><p>The percentage of total US treasury debts that could be affected by the debt ceiling crisis is also unknown. However, the US government currently owes over $30 trillion in debt, and the debt ceiling is currently set at $28.9 trillion. <strong>This means that the debt ceiling could represent over 30% of the total US treasury debt.</strong></p><blockquote>The editor’s note: I’m not sure how Bard get this 30%.</blockquote><p>The projected overall loss that investors would face in the event of a US debt default is also unknown. However, a default on the US debt would be a major financial crisis, and it would have a significant impact on the economy and the financial markets. Investors would lose confidence in the US government’s ability to repay its debts, and this would lead to higher interest rates. Higher interest rates would make it more expensive for businesses to borrow money, and this would slow down economic growth. In addition, a default on the US debt would likely lead to a decline in the value of the dollar, which would make imports more expensive and would further slow down economic growth.</p><blockquote>The editor’s note: Blah, blah, blah. In general, generative AI still lacks proficiency in data analysis, the data scientists’ jobs are secured! :) See if I can tackle this down.</blockquote><h3>Prompt 4: Present a tabular representation displaying the values of government debts, corporate debts, household debts and GDP. Generate the table for the top 20 countries with the highest GDP.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Luu2WVuPXysjVSPUcGceBg.png" /><figcaption>Source: Google Brad; unit in trillions of U.S. dollars.</figcaption></figure><blockquote>The editor’s note: Brad excels at summarizing data in table format, and it even provides a download link that allows me to open an online Google Doc for additional editing.</blockquote><p>Upon initial observation, the level of government debt in the United States appears high, but not as exaggerated as portrayed by the public. When examining the government-to-GDP ratio, the USA closely resembles China and Italy, with Japan’s level surpassing all others by a significant margin. Despite the fact that a nation’s default can raise the benchmark rate and impact the financing of both public and private sectors, the total debt-to-GDP level in the United States is actually quite reasonable. It is considerably lower than that of most developed countries, indicating that the US economy is not excessively burdened by debt, contrary to the claims of many mainstream media outlets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*dV8kaRWPbmCQD0CxigvD1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*nUrL3o9ktwfWWwWlPgAiyA.png" /><figcaption>Source: Google Brad; unit in trillions of U.S. dollars.</figcaption></figure><p>Great, although I’m still curious about the potential losses in the event of an immediate US default in June. Let’s begin by speculating on the impact it would have on the US fixed income market:</p><h3>Prompt 5: Compare the 5-year Credit Default Spreads and yields of USA, CHINA, and ITALY. Present the most recent values as well as the values from the beginning of the year in a table.</h3><p><strong>Bard says:</strong> Here is a table comparing the 5-year credit default spread and yield for the United States, China, and Italy as of the latest date and as of the beginning of the year:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WRC3N47v9Pzy02eJkRRe-g.png" /><figcaption>Source: Google Bard</figcaption></figure><p>The credit default spread is the difference between the yield on a corporate bond and the yield on a government bond of similar maturity. A higher credit default spread indicates that investors are more concerned about the risk of default by the issuer of the bond. A lower credit default spread indicates that investors are less concerned about the risk of default.</p><p>As you can see, the United States has the lowest credit default spread and yield, while Italy has the highest. This is because the United States is perceived as being the least risky of the three countries.</p><blockquote>The editor’s note: Thus, it appears that the US default will result in a <strong>100 basis</strong> point (bps) or 1% rise in the overall US credit market. This implies the potential downgrade of UST from AAA to AA. <strong>Assuming a total debt size of 58.6 trillion and an average duration of 7 years, the estimated loss in the US credit market exceeds 4 trillion. In the global credit market, considering the G20 as a representation, the loss could reach nearly $20 trillion!</strong></blockquote><p>For a livelier estimation of this figure, here’s what Brad adds:</p><p><strong>According to the International Monetary Fund (IMF), the world governments have spent a total of $16.9 trillion on fighting the coronavirus pandemic as of March 8, 2023. This includes direct spending on healthcare, economic support, and other measures to mitigate the impact of the pandemic.</strong></p><p>Impressive! The potential cost of a US default in a single day could surpass the collective spending of the entire human society over the past three years in combating the virus!</p><p>On a much more exaggerated level, we could estimate the heavy stock market sell-off in the event of a US default. Here are some renowned investment gurus who have made predictions:</p><h3>Prompt 6: Check if any investment guru has predicted a stock market decline in the event of a US default.</h3><p><strong>Bard says:</strong> Yes, there are a number of investment gurus who have forecasted a stock market drop if the US defaults on its debt. Here are a few examples:</p><ul><li>Michael Burry: Burry, who famously predicted the 2008 financial crisis, has said that a US debt default would be “the mother of all crashes.” He has estimated that t<strong>he stock market could drop by as much as 80%</strong> in the event of a default.</li><li>Ray Dalio: Dalio, the founder of Bridgewater Associates, the world’s largest hedge fund, has said that a US debt default would be “a global financial catastrophe.” He has estimated that <strong>the global economy could contract by as much as 5%</strong> in the event of a default.</li><li>Paul Krugman: Krugman, a Nobel Prize-winning economist, has said that a US debt default would be “a disaster.” He has estimated that the US economy could <strong>enter a recession</strong> in the event of a default.</li></ul><blockquote>The editor’s note: Wow, an 80% drop in the stock market or a 5% contraction of the world economy, neither of them is acceptable. We must prevent that from happening under any circumstances, considering that the total size of the global stock market capitalization has exceeded $100 trillion.</blockquote><p>However, on the flip side, we should also not allow the government to repeatedly squander taxpayers’ money on flawed politically motivated agendas.</p><h3>Prompt 7: List some widely known criticisms of the US government’s excessive spending.</h3><p>Here are some widely known criticisms of the US government’s excessive spending, cited by Bard:</p><ul><li>It is unsustainable. The US government’s debt is now over $30 trillion, and it is growing at an unsustainable rate. If the government continues to spend more than it takes in, it will eventually default on its debt, which would have a devastating impact on the economy.</li><li>It crowds out private investment. When the government spends more money, it takes money away from the private sector. This makes it more difficult for businesses to borrow money and invest in new projects, which can slow economic growth.</li><li>It leads to inflation. When the government prints more money to finance its spending, it can lead to inflation. Inflation can make it more expensive for people to buy goods and services, which can reduce their standard of living.</li><li>It is unfair to future generations. The US government’s debt is not just a problem for today. It is a problem for future generations, who will have to pay off the debt. This means that future generations will have less money to spend on their own priorities, such as education and healthcare.</li></ul><h3>Conclusion</h3><p>So, I’d love to hear your thoughts on this: Should we consider raising or maintaining the current level?</p><p>What’s your take on whether the congressmen are making the right choices to control the government’s spending? Their aim is to prevent a larger default down the line, which could have a devastating impact on future generations. On the other hand, should we just hold off on making a decision and allow the government to continue printing a significant amount of money in things that, as they suggested, “could benefit our children and the future of humanity”? I’m curious, where do you think the solution lies? Is it within the White House, Capitol Hill, or perhaps the Fed? Your perspective is valuable!</p><p>Or maybe I’ll ask AI for ideas, that could make for an intriguing topic in the upcoming post, so stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cfab8c8a5d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Invest through the Good and Bad times — Part I]]></title>
            <link>https://medium.com/@richardhwlin/how-to-invest-through-the-good-and-bad-times-part-i-d89bf000da8e?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/d89bf000da8e</guid>
            <category><![CDATA[asset-allocation]]></category>
            <category><![CDATA[multi-asset]]></category>
            <category><![CDATA[investment]]></category>
            <category><![CDATA[portfolio-optimization]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Thu, 21 Jul 2022 01:53:02 GMT</pubDate>
            <atom:updated>2022-07-21T01:53:02.302Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to Invest through the Good and Bad times — Part I</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*peYXoLzSs2beBF0i" /><figcaption>Photo by <a href="https://unsplash.com/@mattpaul?utm_source=medium&amp;utm_medium=referral">Matt Paul Catalano</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>In this article, I’ll first revisit the Investment Clock Model I built last year. I’ll then discuss some of the updates I made recently, including a model risk parity portfolio and a risk weighted portfolio using the latest economy forecasts.</p><p><strong><em>Disclaimer: All the models and sample results presented in the article are only prototypes, not yet finalized, and are intended as educational materials only. It’s very important to do your own research before making any investment decision.</em></strong></p><h3><strong>Revisiting the Investment Clock Strategy</strong></h3><p><strong>What is Investment Clock Strategy?</strong></p><p>The Investment Clock strategy is a Merrill Lynch pioneered Macroeconomic research framework that tries to identify the different phases in a complete business cycle and assign the best asset classes in each phase.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OLGmdA81drAimMI9LXSiNw.jpeg" /></figure><p>The four business phases are <strong>Reflation, Recovery, Overheat and Stagflation</strong>. Since many economists agree that two major Macroeconomic signals, <strong>GDP Growth and Inflation</strong>, are the predominating forces in driving the the economy, our model also use them to define the four Investment Clock themes.</p><p>The model is intuitive and hence popular despite the simplicity in settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*8wYrEPQ73xIR_8gJWq1Eog.jpeg" /></figure><p><strong>Reviewing My First Model</strong></p><p>Instead of talking step by step on how I built my first Investment Clock model, which I have fully covered in my last blog <a href="https://medium.com/@richardhwlin/how-does-investment-clock-work-c7d8fbbeb7bd"><strong><em>here</em></strong></a>, I should assume that you have already read the article or grasped some good understandings of the model.</p><p>I’ll just briefly review some of the key model settings. First is on how to classify the four Investment Clock Regimes (business phases). What I used is a very simple fixed cut algorithm and I also used grid search to tune hyper-parameters. My best findings are with a <strong>GDP threshold at 2.5</strong> and <strong>Inflation threshold at 2</strong>.</p><pre>def InvestmentClockFixed(x,cuts={‘GDP’:2.5,’CPI’:2}):<br>   <br>   x_ = x.assign(Growth=None,Inflation=None,Theme=None)<br>   <br>   # define high and low growth<br>   x_.loc[x[‘GDP’]&lt;=cuts[“GDP”],’Growth’] = ‘low’<br>   x_.loc[x[‘GDP’]&gt;cuts[“GDP”],’Growth’] = ‘high’<br>   <br>   # define high and low inflation  <br>   x_.loc[x[‘CPI’]&lt;=cuts[“CPI”],’Inflation’] = ‘low’<br>   x_.loc[x[‘CPI’]&gt;cuts[“CPI”],’Inflation’] = ‘high’<br>   <br>   # define investment clock phases<br>   x_.loc[(x_.Growth==‘low’)&amp;(x_.Inflation==‘low’),’Theme’] = <br>      &#39;Reflection’<br>   x_.loc[(x_.Growth==’high’)&amp;(x_.Inflation==‘low’),’Theme’] =     <br>      ‘Recovery’<br>   x_.loc[(x_.Growth==’high’)&amp;(x_.Inflation==’high’),’Theme’] = <br>      ‘Overheat’<br>   x_.loc[(x_.Growth==‘low’)&amp;(x_.Inflation==’high’),’Theme’] = <br>      ‘Stagflation’</pre><pre>   return x_.dropna()</pre><p>Once we know which theme we are in, the traditional (wall street) investment wisdom tells us that the following sectors are very likely to outperform in their own favored Investment Clock Themes:</p><pre>def <strong>AssetPoolFixed</strong>():<br>return {<br> “Reflection”: [‘IYF’,’IYH’,’IYK’,’IYC’],<br> “Recovery”: [‘IYZ’,’IYW’,’IYM’,’IYC’],<br> “Overheat”: [‘IYJ’,’IYW’,’IYM’,’IYE’],<br> “Stagflation”: [‘IDU’,’IYH’,’IYK’,’IYE’]<br>}</pre><p>Now we can classify what phase of the business cycle we are in, and select the favored sectors based on the predefined asset pool. The last step is to equal weight the selected represented ETFs on a quarterly basis and run the backtest. The result shows that the strategy is pretty good in outperforming the S&amp;P 500 and sector equal-weight portfolio in the <strong>backtest</strong>. <em>Again the keyword here means “past performance is not the indication of future results” and “invest at your own risk”</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*z9ITYFgPTiSDqrCahTDOfQ.png" /></figure><p><strong>Still Many Ways to Improve</strong></p><p>There are for sure many ways I can improve to make it work more practically. For example,</p><ol><li><strong>Over-simplified Theme Classification</strong>: The current model uses a simple cutting method to detect the economic themes, such as GDP ≤2.5% and Inflation ≤2% should be classified as Reflection, etc. The overly simplified setting may cause the model to be too sensitive to the inputs, for example the grid search may suggest very different thresholds as soon as new data flow in.</li><li><strong>Short Price History: </strong>the ETFs only have about 20 years of price history. That is too short to cover the last hyperinflation era in 70’s.</li><li><strong>Subjective Investment Selection</strong>: The current model selects four predefined sector ETFs for each of the investment clock themes. It is based on the wall street analysts’ best experience. This is problematic because the candidates are all equities and lack of diversification. Also the status quo based on decades old experience may already be outdated and not optimized with today’s market dynamics.</li></ol><p>4. <strong>Inflexible Equal Weights: </strong>Equal weighting is perfect for simplicity. However, it lacks the ability to tilting portfolio risk based on investors’ risk appetite. Running portfolio optimization can better help for this purpose.</p><p>For the complete step by step discussion of my first Investment Clock model, I recommend you to read my last blog <a href="https://medium.com/@richardhwlin/how-does-investment-clock-work-c7d8fbbeb7bd"><strong><em>here</em></strong></a><strong>.</strong></p><h3><strong>Designing the New Investment Clock Model</strong></h3><p>The second generation of Investment Clock model is targeting to correct the four above problems. As I briefly work you through in the following section:</p><p><strong>Automatic Detection of Investment Clock Themes</strong></p><p>It involves two classic data cleansing techniques: PCA andK-Means. Principle Component Analysis (PCA) is used to reduce signal noise. A unsupervised classification such as K-means clustering (KMeans) is used to derive automatic, unbiased classification of economic phases. These two algorithms can be wrapped together using the Pipeline function from scikit-learn.</p><pre>from sklearn.pipeline import Pipeline<br>from sklearn.decomposition import PCA<br>from sklearn.cluster import KMeans</pre><pre>pipe = Pipeline([<br> (“pca”, PCA(n_components=2,whiten=True) ),<br> (“kmeans”, KMeans(n_clusters=4, random_state=0,algorithm=’auto’))<br>])</pre><pre>pipe.fit(signals)</pre><p>K-means calculates the Euclidean distances measuring the data to each centroid (the IC theme). I then use a sigmoid function to convert it to the theme probability:</p><pre>scores = pd.DataFrame( pipe.transform(data), <br>         index=signals.index, columns = [0,1,2,3])\<br>         .apply(lambda d: 1/(1+np.exp(d)) )\<br>         .apply(lambda x: x/sum(x), axis=1)</pre><p>K-means automatically generated numeric labels that are not easy to understand. To convert into the investment clock themes we familiar with, I used the following logic:</p><pre>stats = kmeans_signals.groupby(‘labels’).agg(<br> Num = pd.NamedAgg(‘labels’,’count’),<br> CPI = pd.NamedAgg(‘CPI’,’mean’),<br> GDP = pd.NamedAgg(‘GDP’,’mean’),<br>)</pre><pre>stats[‘GDP+CPI’] = stats.GDP + stats.CPI</pre><pre>stats.loc[stats.CPI==max(stats.CPI),’regime’] = ’Stagflation’</pre><pre>stats.loc[(stats.GDP==min(stats[stats.regime.isna()].GDP))<br> &amp; stats.regime.isna(),’regime’] = ’Reflection’</pre><pre>stats.loc[(stats[‘GDP+CPI’]==max(stats[stats.regime.isna()][‘GDP+CPI’])) &amp; stats.regime.isna() ,’regime’] = ’Overheat’</pre><pre>stats.loc[stats.regime.isna(),’regime’] = ’Recovery’</pre><p>As we can see below, the k-means derived themes actually quite make sense. Reflection is associated with a negative GDP growth (meaning the contraction of US economy), and Stagflation suffers with extraordinary high inflation. Recovery and Overheat are the two less extreme situations. Overheat clearly observed a higher nominal growth (GDP+CPI) than in the Recovery phase.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*_XfDbnzQIeX4bRxQWy5ZDw.png" /></figure><p>We can validate the results with visualizations. The top to bottom charts are: Theme probabilities, Theme clusters plotted across GDP and Inflation, and Theme and econ signal time series.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*dzl7ZZcdbzd2Uh3cghgoyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*DPcNttsXOJi10P-OZkmxlA.png" /></figure><p><strong>The New Investment Asset Pool</strong></p><p>Instead of section rotation, my new Investment Clock model is built on asset allocation. My desired asset classes are <strong>Equity, Bond, Commodity </strong>and <strong>Cash.</strong> Here are their represented ETF tickers:</p><pre>assets = { ‘equity’:’IVV’, <br>           ’bond’:’TLT’,<br>           ’cash’:’SHY’, <br>           ’commodity’:’DBC’, <br>           }</pre><pre>from yahoo_fin import stock_info as si<br>prices = {k:si.get_data(v) for k,v in assets.items()}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*XWSRYXZICWE-0HLQOuBWUA.png" /></figure><p><strong>Extending Asset History using Proxy Indices</strong></p><p>Unfortunately these ETFs only have less than 20 years price history and they are not sufficient to cover the important 70’s hyperinflation era. What I can use as substitutes for early year history are financial indexes from Fred:</p><ul><li>Equity: Wilshire 5000 Total Market Index (WILL5000IND)</li><li>Bond: Market Yield on U.S. Treasury Securities at 20-Year Constant Maturity (DGS20)</li><li>Commodity: Monthly Producer Price Index by All Commodities (PPIACO)</li><li>Cash: Market Yield on U.S. Treasury Securities at 2-Year Constant Maturity (DGS2)</li></ul><pre>from fredapi import Fred<br>fred = Fred(api_key=&#39;your_password&#39;)</pre><pre>factors = pd.DataFrame( dict(<br>    equity = fred.get_series(‘WILL5000IND’), <br>    bond = fred.get_series(‘DGS20’),<br>    commodity = fred.get_series(‘PPIACO’),<br>    cash = fred.get_series(‘DGS2’), <br> )).fillna(method=’ffill’).to_monthly().dropna()</pre><p>A few tips here: I actually did the following transformation to bring them more in line with the monthly total returns of those ETFs.</p><pre>factor_mrets = factors\<br>  .assign(equity=factors.equity.pct_change()*100)\<br>  .assign(bond=factors.bond/12-factors.bond.diff()*15)\<br>  .assign(commodity=factors.commodity.pct_change()*100)\<br>  .assign(cash=factors.cash/12-factors.bond.diff()*1.5)\<br>.fillna(0)</pre><pre>factors = (1+factor_mrets/100).cumprod()*100</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*7WS65cR2AA16EBn_q-aZ0w.png" /></figure><p>How well are they matching with the live ETFs? I could also scale them with the same level of ETF volatilities.</p><pre>std_y = totidx[dt_str:dt_end].to_monthly().pct_change().std()<br>std_x = factors[dt_str:dt_end].to_monthly().pct_change().std()<br>loadings = np.diag( np.array(std_y)/np.array(std_x) )</pre><pre>pd.DataFrame(loadings,index=Y_cols,columns=X_cols)\ .plot(kind=’barh’,stacked=True) `</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*EV_BKucQ9Zib_FYLmLcxjA.png" /></figure><p><strong>Portfolio Optimization and Efficient Frontiers</strong></p><p>Till now, we just finished all the data download, transform and cleaning work and next is the most fun part of the project — portfolio optimization.</p><p>Markowitz Portfolio Optimization is also known as Mean-Variance Optimization. It focus on either maximizing the return given risk budget or minimizing the risk given return target. A blended way is to define objective function with built-in tradeoffs between mean and variance. Interested readers can search Wikipedia for more literature and coding samples.</p><p>Here’s the MV Optimization code I used:</p><pre>def eff_frontier(mu,sigma, dr=0.0025, ):</pre><pre> pfRet = lambda x: x.dot(mu)<br> pfVar = lambda x: x.dot(sigma).dot(x)<br> <br> w0 = np.ones(len(mu))/len(mu)<br> mus = np.arange(start=0,stop=mu.max()+dr, step=dr, dtype=’float’)<br> <br> weights = np.zeros((len(mus),len(mu)))<br> pf_Ret = np.zeros(len(mus)) <br> pf_Var = np.zeros(len(mus)) <br> <br> for r in np.arange(len(mus)):<br>   exp_ret = mus[r]<br>   w_constraints = (<br>       {‘type’: ‘eq’, ‘fun’:lambda w:sum(w)-1.},<br>       {‘type’: ‘eq’, ‘fun’:lambda w:pfRet(w)-exp_ret},<br>   ) <br>   w_opt = minimize( pfVar,<br>                     w0,<br>                     method = ’SLSQP’,<br>                     constraints = w_constraints,<br>                     bounds = [(0,1)] * len(w) <br>                   )<br><br>   w= w_opt.x</pre><pre>   weights[r,] = w<br>   pf_Ret[r] = pfRet(w)<br>   pf_Var[r] = pfVar(w)<br> <br>  return weights,pf_Ret,pf_Var</pre><p>Just remember that what we need is not just one universal efficient frontier, but rather different efficient frontiers to cover all four economy phases. Indeed what really matters here is the correct expected return and covariance matrix under the different economy phases.</p><p>One way of doing that is to group the price history into different regimes (for example based on our derived signals) and calculate the return and covariance separately. But we might get less and uneven size of records which may undermine the reliability of the final result. The second way, which I prefer, is to calculate the weighted return mean and covariance using all available results. It’s easy to do with numpy embedded functions:</p><pre>mu = np.average(A,axis=0,weights=P )<br>sigma = np.cov( A.T, aweights=P )</pre><p><em>where P is the theme probability (a.k.a. score).</em></p><p>Now here’re the plots of efficient frontiers for each of the investment clock theme:</p><pre>for P_col in P_cols:</pre><pre> results = stats[stats.theme==P_col]<br> <br> fig,ax = plt.subplots(figsize=(12,4))<br> plt.suptitle(P_col + ”Optimized Portfolio”, fontsize=16)</pre><pre> results.plot(x=’ret’,y=Y_cols,kind=’area’,stacked=True,ax=ax)<br> ax.set_xlim(results.ret.min(),results.ret.max())<br> ax.set_ylim(0,100)<br> ax.set_ylabel(‘sector weight (%)’)<br> ax.set_xlabel(‘portolio return (% p.a.)’)<br> ax.legend(bbox_to_anchor=(1.1, .9), loc=2, borderaxespad=0.0)</pre><pre> ax2 = ax.twinx()<br> results.plot(x=’ret’, y=’risk’, color=’yellow’,<br>              marker=’o’, linewidth=3, ax=ax2)<br> ax2.set_ylabel(‘portfolio risk (% p.a.)’)<br> ax2.legend(bbox_to_anchor=(1.1, 1), loc=2, borderaxespad=0.0)</pre><pre> plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*J0GaOnv3WKsutE4BJtLNTA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*r7joFhziRd0R_rG4IWbUFA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*bzib1J3T2btnge0JboR3ng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*HQjRS0Y2ZeN_t3oLSG6BAw.png" /></figure><p><em>I have calculated based on the average forward three months return data. It’s very possible that you might get different results with different settings.</em></p><p>How to interpret these results? It’s interesting to see how asset allocations consistently evolve across the different regimes of the investment clock cycle. For example, Stagflation regime actually shows no value for bonds in a conventional allocation portfolio (portfolio risk targeted at 7% to 15%). Also against the common sense, Commodity is relatively most favored under the Overheat regime rather than under the Stagflation regime.</p><p><strong>Construct Your Own Favorite Portfolio Allocation</strong></p><p>With the efficient frontiers ready, we can now select the desired portfolio weightings based on any preferred risk return profile. For example, I can require an expected return of 500 bps above risk free rate, and a risk budget no higher than 15% p.a.</p><pre>min_ret = 5 # above cash<br>max_risk = 15</pre><pre>optimal_weights = None<br>for P_col in P_cols:<br>  optimal_weights = pd.concat([optimal_weights,<br>     stats.loc[ (stats.theme == P_col) <br>              &amp; (stats.ret &gt;= stats.cash + min_ret ) <br>              &amp; (stats.risk &lt;= max_risk) <br>            , ].sort_values(‘sharpe’,ascending=False).head(1)],)</pre><p>Now we have the optimized portfolio allocations with constraints of expected return at cash + 5% and risk at 15%. Two apparent takeaways are:</p><ol><li>You should invest in stocks in any occasion, and,</li><li>Definitely avoid bonds in a stagflation environment.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*HleBVM3FihK1NBesFVNC6Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*6UnJY-mlI2JQhBSVw03Ocw.png" /></figure><p><strong>Risk Parity Strategy</strong></p><p>If we just want a lazy man’s portfolio that can buy and hold under every economy condition, we could equally weight these four theme sub-portfolios. That gives us actually the <strong>Risk Parity </strong>weights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*bfjRRFets1S5gMxpuPSwDg.png" /></figure><p><strong>Update Portfolio with New Forecasts</strong></p><p>We can also update model weights based on the new Fed release or our own source of forecasts.</p><pre>def regime_forecast(GDP=2,CPI=7):<br> forecast = pd.DataFrame(index=[‘forecast’]).assign(GDP=GDP,CPI=CPI)<br> d = pipe.transform(forecast)<br> x = np.exp(1/d) <br> s = x/np.sum(x) <br> return(s)</pre><p>For example, with the somewhat extreme numbers GDP=-1.4 and CPI=8.5, we have the forecasted current regime tilted toward Stagflation. So the Probability Weighted Portfolio suggested more equities and less bonds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*PazPHKygPTpfnDZHbYKrYg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/347/1*vSviVr04TBHGOeuJN8OhhQ.png" /></figure><h3><strong>Project Summary</strong></h3><p>I re-engineered the Investment Clock model to make it also work for asset allocation. The new model uses K-means as the automatic regime detection algorithm, and uses the regime based MV optimization to allocate asset weights based on any specific risk and return constraints. Equal-weighting the theme sub-portfolios gives us the optimized Risk Parity weights that investors can hold through both good and bad times. Using the new GDP and CPI forecasts allow us to timely update the portfolio with theme based probabilities.</p><p>The MV optimization is largely unconstrained. The long only constraint allows for non negative asset weights and 100% total weights. I think it probably make sense for Part II to look at cases with more portfolio constraints applied.</p><p>Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d89bf000da8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Does Investment Clock Work?]]></title>
            <link>https://medium.com/@richardhwlin/how-does-investment-clock-work-c7d8fbbeb7bd?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c7d8fbbeb7bd</guid>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[investment-clock]]></category>
            <category><![CDATA[investment]]></category>
            <category><![CDATA[quantitative-strategies]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Fri, 20 Aug 2021 15:44:44 GMT</pubDate>
            <atom:updated>2021-08-21T02:40:28.000Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This research is inspired by a more detailed discussion made by Marco Ops: </em><a href="https://macro-ops.com/the-investment-clock/"><em>https://macro-ops.com/the-investment-clock/</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xati2_8D7F-yzeOz88zz_Q.jpeg" /></figure><h3><strong>Introduction: What is Investment Clock?</strong></h3><p>As per <strong>Marco Ops</strong>’ article, the Investment Clock is a <strong>Marco Investment Strategy</strong> first introduced by <strong>Merrill Lynch</strong>. It is a simple yet useful framework for understanding the various phases of a business cycle and finding the best-performing asset classes in each phase.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*8wYrEPQ73xIR_8gJWq1Eog.jpeg" /></figure><blockquote><strong>The Investment Clock splits the business cycle into four phases. Each phase is comprised of the direction of growth and inflation relative to their trends.</strong></blockquote><p><strong>Reflation</strong> — both the GDP growth and inflation are falling or lower than the trend. The stocks are suffering in a bear market but <strong>bonds</strong> suppose to be the most welcomed asset because of the generous monetary and fiscal support from governments and central banks (bail-out, cutting rates, and stimulus programs).</p><p><strong>Recovery</strong> —Growth starts to back on track while inflation still remains low. <strong>Stocks</strong> regain attractions with very attractive valuation and improving earnings. It is, of course, the most favorable asset at this stage.</p><p><strong>Overheat</strong> — Growth reaches the peak and slows down and inflation is rising. (<em>does it sound familiar if you follow these days’ headlines?</em>) Both stocks and bonds won’t perform well, but betting on <strong>commodities</strong> will be a proliferating and profitable strategy.</p><p><strong>Stagflation</strong> — Inflation is way out of control and that severely hurt consumer confidence. Central banks are forced to hike rates, and stocks, as one of the leading indicators of the economy, already fell. However, this stage of the cycle doesn’t happen that often in the last few decades, thanks to Fed’s “remarkable” economic interference policy, that is to print out enormous money to stimulate the economy meanwhile artificially set interest rates low to control the inflation (<em>however we just don’t know how long it could last</em>). Flying to safety asset, <strong>cash</strong>, is the best choice given the circumstance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OLGmdA81drAimMI9LXSiNw.jpeg" /></figure><h3><strong>Step 1: Define and Download Asset Pools</strong></h3><p>My sense is that Investment Clock should work the best within an Asset Allocation framework. But still, I’m going to reuse the sector ETF data that I’ve always been using in several of my previous experiments.</p><p>The ML research suggests the following sector rotation based on Investment Clock theory:</p><pre><strong><em>Reflation</em></strong><em> — Financials, Consumer Staples, Healthcare, and Consumer Discretionary;<br></em><strong><em>Recovery</em></strong><em> — Telecom, Tech, Basic Materials, and Consumer Discretionary;<br></em><strong><em>Overheat</em></strong><em> —Industrials, Tech, Basic Materials, and Energy;<br></em><strong><em>Stagflation</em></strong><em> — Utilities, Consumer Staples, Healthcare, and Energy.</em></pre><p>Sector rotation defined based on iShares ETFs:</p><pre>def <strong>AssetPoolFixed</strong>():<br>return {<br>  &quot;Reflection&quot;: [&#39;IYF&#39;,&#39;IYH&#39;,&#39;IYK&#39;,&#39;IYC&#39;],<br>  &quot;Recovery&quot;: [&#39;IYZ&#39;,&#39;IYW&#39;,&#39;IYM&#39;,&#39;IYC&#39;],<br>  &quot;Overheat&quot;: [&#39;IYJ&#39;,&#39;IYW&#39;,&#39;IYM&#39;,&#39;IYE&#39;],<br>  &quot;Stagflation&quot;: [&#39;IDU&#39;,&#39;IYH&#39;,&#39;IYK&#39;,&#39;IYE&#39;]<br>}</pre><p>Now we can download the historical prices using the codes I described in my previous <a href="https://medium.com/@richardhwlin/advanced-backtesting-with-bt-635ed441cb60">blog</a>.</p><h3><strong>Step 2: Download Economic Data</strong></h3><p>Our next step is to download the GDP and inflation data using <a href="https://github.com/mortada/fredapi"><strong>Fred API</strong></a>:</p><pre>from fredapi import Fred<br>fred = Fred(api_key=&quot;your_own_key&quot;)<br>signals = pd.DataFrame(<br>  dict(<br>    GDP = fred.get_series(‘GDPC1’),<em># Real GDP</em><br>    CPI = fred.get_series(‘CPILFESL’),<em> # Core CPI</em><br>  )<br>).fillna(method=’ffill’).pct_change(12).dropna()*100</pre><p>The above code downloads the US real GDP (quarterly) and Core CPI (monthly), then converted to YoY changes in percent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*6N2NQ-Ob8rByM8hYQvMD9Q.png" /></figure><h3><strong>Step 3: Define Investment Clock Phases</strong></h3><p>I used a very simple algorithm to define investment clock themes. For example, when real GDP Growth &gt;2.5% p.a., it’s High Growth, otherwise, it’s Low Growth. Similarly, when Core Inflation &gt;3% p.a., it’s High Inflation, otherwise, it’s Low Inflation. The Four phases of Investment Clock are separately defined based on the different combinations of Growth and Inflation status:</p><pre>def InvestmentClockFixed(x,cuts={‘GDP’:2.5,’CPI’:3}):<br>   <br>   x_ = x.copy().assign(Growth=None,Inflation=None,Theme=None)<br>   <br>   # define high and low growth<br>   x_.loc[x[‘GDP’]&lt;=cuts[“GDP”],’Growth’] = ‘low’<br>   x_.loc[x[‘GDP’]&gt;cuts[“GDP”],’Growth’] = ‘high’<br>   <br>   # define high and low inflation  <br>   x_.loc[x[‘CPI’]&lt;=cuts[“CPI”],’Inflation’] = ‘low’<br>   x_.loc[x[‘CPI’]&gt;cuts[“CPI”],’Inflation’] = ‘high’<br>   <br>   # define investment clock phases<br>   x_.loc[(x_.Growth==‘low’)&amp;(x_.Inflation==‘low’),’Theme’] = <br>&#39;Reflection’<br>   x_.loc[(x_.Growth==’high’)&amp;(x_.Inflation==‘low’),’Theme’] = ‘Recovery’<br>   x_.loc[(x_.Growth==’high’)&amp;(x_.Inflation==’high’),’Theme’] = ‘Overheat’<br>   x_.loc[(x_.Growth==‘low’)&amp;(x_.Inflation==’high’),’Theme’] = ‘Stagflation’</pre><pre>return x_.dropna()</pre><p>How does the history look like based on our defined phases? I have some test-run (see chart below). Indeed, stagflation is the least observed phase with only a few cases that happened between 70’s and 90’s, but none has been observed within the latest 30 years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*PpHiRIs7GUZ2WtW7RTYPVA.jpeg" /></figure><p>For your interest, the code to generate the above plot is here:</p><pre>themes = InvestmentClockFixed(signals,cuts={‘GDP’:2.5,’CPI’:3})</pre><pre>fig = plt.figure(figsize=(12,8))<br>fig.suptitle(‘Investment Clock Themes with Fixed Cuts’,fontsize=16)</pre><pre>ax1 = fig.add_axes((0,0.5,1,0.45))<br>ax2 = fig.add_axes((0,0,0.3,0.4))<br>ax3 = fig.add_axes((0.35,0,0.3,0.4))<br>ax4 = fig.add_axes((0.7,0,0.3,0.4))</pre><pre>y_lim = (-10,15)</pre><pre>colors = dict(<br>  Overheat=’#f4d19e’,<br>  Recovery=’#87ca9d’,<br>  Reflection=’#dbdbdb’,<br>  Stagflation=’#ffb3ba’<br>) </pre><pre>p1 = themes[[‘GDP’,’CPI’]].plot(ax=ax1)<br>for t in [&#39;Overheat&#39;,&#39;Recovery&#39;,&#39;Reflection&#39;,&#39;Stagflation&#39;]:<br>   ax1.fill_between(<br>      themes.index, y_lim[0],y_lim[1], <br>      where=(themes.Theme==t),<br>      interpolate = True,<br>      facecolor=colors[t], <br>      alpha = 0.5<br>   )</pre><pre>growth_counts=themes.Growth.value_counts()<br>ax2.pie(growth_counts,autopct=’%.1f %%’,<br>        labels=growth_counts.index)<br>ax2.set_title(‘Growth’)</pre><pre>inflation_counts=themes.Inflation.value_counts()<br>ax3.pie(inflation_counts,autopct=’%.1f %%’,<br>        labels=inflation_counts.index)<br>ax3.set_title(‘Inflation’)</pre><pre>theme_counts=themes.Theme.value_counts().sort_index(ascending=True)<br>ax4.pie(theme_counts,autopct=’%.1f %%’,<br>        labels=theme_counts.index,colors=colors.values())<br>ax4.set_title(‘Theme’)</pre><pre>plt.show()</pre><h3><strong>Step 4: Building the Investment Clock BT function</strong></h3><p><strong>It’s now the exciting moment to build our core backtest function!</strong> Friends following my blog know that I always use <a href="https://pmorissette.github.io/bt/"><strong>BT — the Flexible Backtesting Toolkit for Python</strong></a> to run all the strategy simulations, a.k.a. backtesting. It is surprisingly easy:</p><pre>import bt</pre><pre># step 1: build Investment Clock Class<br>class InvestmentClock(bt.Algo):</pre><pre>   def __init__(self, themes, <br>                lag = pd.DateOffset(days=0),<br>                pool = AssetPoolFixed()):<br>      super(InvestmentClock, self).__init__()<br>      self.Themes = themes<br>      self.lag = lag<br>      self.pool = pool</pre><pre>   def __call__(self, target):<br>      t0 = target.now — self.lag<br>      Theme = self.Themes[:t0][-1]<br>      target.temp[‘selected’] = self.pool[Theme]<br>      return True</pre><pre># step 2: build Investment Clock backtest<br>def bt_InvestmentClock(name,tickers,prices,signals,<br>                       cuts={‘GDP’:2.5,’CPI’:3}):<br>   <br>   themes = InvestmentClockFixed(signals,cuts)<br>   asset_pool = AssetPoolFixed()<br>   <br>   s = bt.Strategy(<br>        name,<br>        algos = [<br>          bt.algos.RunQuarterly(),<br>          bt.algos.SelectAll(),<br>          bt.algos.SelectThese(tickers),<br>          InvestmentClock( themes.Theme, <br>             lag=pd.DateOffset(days=0), <br>             pool = asset_pool ),<br>          bt.algos.WeighEqually(),<br>          bt.algos.Rebalance()<br>        ],<br>       )</pre><pre>return bt.Backtest(s, prices)</pre><p>For comparison purpose, I also have a passive backtest function that runs S&amp;P500 and Sector Equal-Weight strategies:</p><pre>def bt_passive(name,tickers,prices):<br>   s = bt.Strategy(name,<br>         algos = [<br>            bt.algos.RunQuarterly(),<br>            bt.algos.SelectAll(),<br>            bt.algos.SelectThese(tickers),<br>            bt.algos.WeighEqually(),<br>            bt.algos.Rebalance()<br>         ],<br>   )<br>   return bt.Backtest(s, prices)</pre><h3><strong>Step 5: Running the Investment Clock Backtest</strong></h3><p>Now I run the investment clock backtest with the fixed cuts of GDP at 2.5 (YoY % p.a.) and CPI at 3( YoY % p.a.):</p><pre>tickers = [&#39;IYC&#39;,&#39;IDU&#39;,&#39;IYZ&#39;,&#39;IYW&#39;,&#39;IYJ&#39;,&#39;IYH&#39;,<br>           &#39;IYF&#39;,&#39;IYE&#39;,&#39;IYK&#39;,&#39;IYM&#39;,&#39;IYR&#39;]</pre><pre>backtest_sp50 = bt_passive(‘SP50’,[‘IVV’],prices)<br>backtest_spew = bt_passive(‘SPEW’,tickers,prices)<br>backtest_spic = bt_InvestmentClock(‘SPIC’,tickers,prices,signals,<br>                                   cuts={‘GDP’:2.5,’CPI’:3})</pre><pre>report = bt.run(backtest_sp50, backtest_spew,backtest_spic)<br>report.display()</pre><p>The simulated results show that SPIC (Investment Clock) only slightly outperformed SPEW (Sector Equal-Weight) over the mid to long horizons, but underperformed within one year. Still very encouraging given such a simple setting it has!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*8ZLJjQ53S_BJjC-0aJ-XYQ.png" /></figure><pre>Stat                 SP50        SPEW        SPIC<br>-------------------  ----------  ----------  ----------<br>Start                2002-07-29  2002-07-29  2002-07-29<br>End                  2021-08-19  2021-08-19  2021-08-19<br>Risk-free rate       0.00%       0.00%       0.00%<br><br>Total Return         604.44%     562.83%     579.59%<br>Daily Sharpe         0.63        0.61        0.64<br>Daily Sortino        0.98        0.95        0.99<br>CAGR                 10.79%      10.43%      10.58%<br>Max Drawdown         -55.25%     -55.02%     -54.83%<br>Calmar Ratio         0.20        0.19        0.19<br><br>MTD                  0.32%       -0.51%      -2.46%<br>3m                   7.42%       4.08%       0.25%<br>6m                   13.55%      11.21%      4.27%<br>YTD                  18.43%      17.37%      8.39%<br>1Y                   32.46%      31.92%      24.21%<br>3Y (ann.)            17.64%      13.23%      13.50%<br>5Y (ann.)            17.23%      12.69%      13.68%<br>10Y (ann.)           16.86%      13.71%      15.25%<br>Since Incep. (ann.)  10.79%      10.43%      10.58%<br><br>Avg. Drawdown        -1.74%      -1.95%      -2.17%<br>Avg. Drawdown Days   21.91       22.26       25.66<br>Avg. Up Month        3.07%       3.07%       3.33%<br>Avg. Down Month      -3.60%      -3.68%      -3.39%<br>Win Year %           89.47%      84.21%      84.21%<br>Win 12m %            87.21%      84.47%      85.39%</pre><h3><strong>Step 6: Hyperparameter Tuning</strong></h3><p>How will you interpret a YoY real GDP growth reading of 2.5%? Is it still too low or high enough? I’ll have the same question for inflation. Economics is not a mere black and white natural scientific subject, but a social science topic with enormous human expectations embedded. We can still leverage on scientific methods to improve our predictions.</p><p>Scikit-learn has the built-in grid search module <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#examples-using-sklearn-model-selection-gridsearchcv"><strong>GridSearchCV</strong></a><strong>, </strong>however I haven’t learned how to apply that to my customized pediction model. So I decided to build my own.</p><p><strong>1. construct return histories for each of the 4 Investment Clock themes:</strong></p><pre>pool = AssetPoolFixed()</pre><pre>bt_reflection=bt_passive(‘Reflection’,pool[‘Reflection’],prices)<br>bt_recovery=bt_passive(‘Recovery’,pool[‘Recovery’],prices)<br>bt_overheat=bt_passive(‘Overheat’,pool[‘Overheat’],prices)<br>bt_stagflation=bt_passive(‘Stagflation’,pool[‘Stagflation’],prices)</pre><pre>report_cv = bt.run(backtest_sp50,<br>            bt_reflection,bt_recovery,bt_overheat,bt_stagflation)</pre><p><strong>2. build object function to score Investment Clock themes:</strong></p><pre>def objective(cut_gdp=2,cut_cpi=2,freq=’q’):</pre><pre>    themes = InvestmentClockFixed(signals,<br>                 cuts={‘GDP’:cut_gdp,’CPI’:cut_cpi})\<br>             .resample(freq).first()\<br>             .reset_index().rename(columns={‘index’:’Date’})</pre><pre>    scores = pd.DataFrame({k:report_cv[k].prices <br>             for k in asset_pool.keys()})\<br>         .div(report[‘SP50’].prices,axis=0)\<br>         .resample(freq).last().pct_change().dropna()\<br>         .reset_index().rename(columns={‘index’:’Date’})\<br>         .melt(id_vars=’Date’,var_name=’Theme’,value_name=’score’)</pre><pre>return pd.merge(scores,themes,on=[‘Date’,’Theme’]).score.sum()</pre><p><strong>3. run the exhaustive grid search:</strong></p><p><em>(Just be mindful that it will be slow especially on my pi4)</em></p><pre>output = pd.DataFrame()</pre><pre>for gdp_ in [1+.25*n for n in range(0,13)]:<br>   for cpi_ in [1+.25*n for n in range(0,13)]:<br>      output = output.append(<br>                  dict( <br>                        gdp=gdp_, cpi=cpi_,<br>                        score=objective(gdp_,cpi_,’q’)<br>                      ),<br>                  ignore_index=True<br>               )</pre><p><strong>4. build a pivot table to visualize the results:</strong></p><p><em>(The results are percentile ranked for ease of interpretation)</em></p><pre>pivot = output<br>        .assign(rank=output.score.rank(pct=True,ascending=True))\<br>        .pivot(‘cpi’,’gdp’,’rank’)\<br>        .sort_index(ascending=False)</pre><pre>sns.heatmap(pivot, linewidths = .5, annot=True , cmap=’jet’)<br>plt.title(‘Model Scores Based on Percentile Rank’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VCMZGMdH6Nl-JLIMxbcEEg.jpeg" /></figure><p>If you enlarge it you can see that <strong>GDP=2.5</strong> and <strong>CPI=2</strong> seem to provide us the best result (1 is the best spot with the maximized return of the entire backtest history).</p><p><strong>5. is it true? run backtest on optimized params and test it yourself!</strong></p><pre>backtest_best = bt_InvestmentClock(‘BEST’,tickers,prices,signals,<br>                                   cuts={‘GDP’:2.5,’CPI’:2})</pre><pre>report2 = bt.run( backtest_sp50, backtest_spew,<br>                  backtest_spic, backtest_best )</pre><pre>report2.plot()<br>plt.title(‘Investment Clock Strategy with Optimal Fixed Cuts’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*eWafiNNbmtVxaJJomm4-Vw.png" /></figure><pre>Stat                 SP50        SPEW        SPIC        BEST<br>-------------------  ----------  ----------  ----------  ----------<br>Start                2002-07-29  2002-07-29  2002-07-29  2002-07-29<br>End                  2021-08-19  2021-08-19  2021-08-19  2021-08-19<br>Risk-free rate       0.00%       0.00%       0.00%       0.00%<br><br>Total Return         604.44%     562.83%     579.59%     831.42%<br>Daily Sharpe         0.63        0.61        0.64        0.72<br>Daily Sortino        0.98        0.95        0.99        1.14<br>CAGR                 10.79%      10.43%      10.58%      12.42%<br>Max Drawdown         -55.25%     -55.02%     -54.83%     -47.49%<br>Calmar Ratio         0.20        0.19        0.19        0.26<br><br>MTD                  0.32%       -0.51%      -2.46%      -2.46%<br>3m                   7.42%       4.08%       0.25%       0.63%<br>6m                   13.55%      11.21%      4.27%       6.66%<br>YTD                  18.43%      17.37%      8.39%       10.88%<br>1Y                   32.46%      31.92%      24.21%      27.06%<br>3Y (ann.)            17.64%      13.23%      13.50%      12.52%<br>5Y (ann.)            17.23%      12.69%      13.68%      11.73%<br>10Y (ann.)           16.86%      13.71%      15.25%      14.80%<br>Since Incep. (ann.)  10.79%      10.43%      10.58%      12.42%<br><br>Avg. Drawdown        -1.74%      -1.95%      -2.17%      -2.20%<br>Avg. Drawdown Days   21.91       22.26       25.66       23.09<br>Avg. Up Month        3.07%       3.07%       3.33%       3.23%<br>Avg. Down Month      -3.60%      -3.68%      -3.39%      -3.33%<br>Win Year %           89.47%      84.21%      84.21%      89.47%<br>Win 12m %            87.21%      84.47%      85.39%      88.13%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*IiPTDGHWm8fWmAmE80BlVg.png" /></figure><h3>One Last Word</h3><p>The Investment Clock strategy is obviously very interesting to me at this moment. Almost all of the major central banks in the developed world are explicitly or implicitly deployed their <a href="https://en.wikipedia.org/wiki/Modern_monetary_theory"><strong>Modern Monetary Theory (MMT) </strong></a>— they endorse the overwhelmingly excessive borrowing &amp; spending without worrying about the burst of government debt.</p><p>Is there really such a “free lunch”? There will be multiple losers: First is the creditors (government bond buyers). Second is the emerging market exporters, as the developed world doesn’t produce. Also, don’t forget that it will be more and more difficult for younger generations (X, Y, Z) to buy houses because flooded liquidity already shot up the prices but governments are deliberately controlling the wages (inflation). Eventually, there might be greater and greater wealth inequality and social unrest (<em>well, we both know it but let the next elected governor deal with it</em>).</p><p>The MMT topic might be too far away, but let’s focus back on reality. In the coming years, the Fed will find it more and more difficult to put the goldilocks of growth and inflation targets under control. <strong>The market will see more and more surprises and any naive buy and hold strategy will be heavily disadvantaged.</strong> Fortify your investment (with active strategy) and prepare for the battle (on the capital market).</p><p>My model is not finalized and is just an experiment for the interest of data science and finance. It should not be used for any investment suggestion. I plan to publish more researches in the future to refine this strategy. Maybe lead to a more comprehensive strategic and tactic model that is similar to Bridgewater&#39;s famed “all-weather” strategy, but I’m free to share.</p><p>Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c7d8fbbeb7bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Backtest Validation]]></title>
            <link>https://medium.com/@richardhwlin/backtest-validation-22f52d1f74d?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/22f52d1f74d</guid>
            <category><![CDATA[etf]]></category>
            <category><![CDATA[investing]]></category>
            <category><![CDATA[quant]]></category>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[crossvalidation]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Sat, 25 Apr 2020 22:09:52 GMT</pubDate>
            <atom:updated>2020-04-25T22:09:52.615Z</atom:updated>
            <content:encoded><![CDATA[<h4>Continuing from the blog “<a href="https://medium.com/@richardhwlin/advanced-backtesting-with-bt-635ed441cb60">Advanced Backtesting with BT</a>”</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WCRoecyY_1SfZXtx" /><figcaption>Photo by <a href="https://unsplash.com/@stefanbc?utm_source=medium&amp;utm_medium=referral">Stefan Cosma</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>Backtesting without validation is just science-fiction.</blockquote><h3>Introduction</h3><p>In <a href="https://medium.com/@richardhwlin/advanced-backtesting-with-bt-635ed441cb60">my last blog</a>, which is also the third of the quant Investing series, I have demonstrated how to mimic the S&amp;P Sector Rotator strategy using BT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*XLUhcN40To2qJCcbcqJxHg.png" /></figure><pre>Stat                 SP500       SPESW       SPHMV<br>-------------------  ----------  ----------  ----------<br>Start                2002-07-29  2002-07-29  2002-07-29<br>End                  2020-01-31  2020-01-31  2020-01-31<br>Risk-free rate       1.00%       1.00%       1.00%</pre><pre>Total Return         405.13%     403.30%     474.86%<br>Daily Sharpe         0.55        0.55        0.52<br>Daily Sortino        0.86        0.85        0.82<br>CAGR                 9.69%       9.67%       10.51%<br>Max Drawdown         -55.25%     -55.02%     -61.36%<br>Calmar Ratio         0.18        0.18        0.17</pre><pre>MTD                  -0.00%      -1.03%      1.28%<br>3m                   6.67%       4.01%       5.30%<br>6m                   9.26%       5.77%       8.94%<br>YTD                  -0.00%      -1.03%      1.28%<br>1Y                   21.61%      15.21%      22.42%<br>3Y (ann.)            14.48%      9.62%       9.97%<br>5Y (ann.)            12.04%      8.57%       10.43%<br>10Y (ann.)           13.75%      11.83%      11.03%<br>Since Incep. (ann.)  9.69%       9.67%       10.51%</pre><pre><em>(omitted several lines here)</em></pre><pre>Avg. Drawdown        -1.71%      -1.92%      -2.53%<br>Avg. Drawdown Days   24.00       23.50       30.91<br>Avg. Up Month        2.92%       2.92%       3.62%<br>Avg. Down Month      -3.46%      -3.57%      -3.93%<br>Win Year %           83.33%      77.78%      72.22%<br>Win 12m %            86.50%      85.00%      76.50%</pre><p>From the above-pulled performance matrix, we can see that <strong>SPHMV</strong> did quite well, outperforming the <strong>S&amp;P 500</strong> Index, but did the <strong>High-Momentum-Value (HMV)</strong> algorithm add enough value? Although SPHMV’s total return is higher, its Sharpe ratio is somehow lower than the S&amp;P 500 Index. Since only performance couldn’t tell us much, we need to look for answers with the help of <strong>Cross-Validation techniques</strong>.</p><p><strong>How to Cross-Validate backtest results?</strong></p><p>Many of us are familiar with the cross-validation functions in <a href="https://scikit-learn.org/stable/modules/cross_validation.html"><strong><em>Scikit-learn</em></strong></a>, such as <strong>K-fold</strong> <strong>(CV)</strong> or <strong>Leave-one-out (LOOCV)</strong>. Unfortunately,<strong> the same technologies don’t work well when handling time series data</strong>. Time series are path-dependent data, which means that you can not test an event by training with another event that happens in the future.</p><p><a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html#sklearn.model_selection.TimeSeriesSplit"><strong>Time-series-split</strong></a> only works well with univariate time series. However, our strategy needs to select 3 winners from 11 rather than just 1 sector (multivariate TS). To CV multivariate TS, I think that Sampling should only be used in selecting the population (sectors). By applying time-series-split across the time (say, in total t samples) and leave-one-out across the population (say, p sample sets), we then can have t*p samples to fairly evaluate the model statistical significance.</p><p>there are very few resources on the internet to discuss backtest validation, especially in the finance domain. the closest concept I can find is written by <a href="https://towardsdatascience.com/@ccochrane_17940?source=post_page-----76adba623eb9----------------------">Courtney Cochrane</a> in 2018:</p><p><a href="https://towardsdatascience.com/time-series-nested-cross-validation-76adba623eb9">Time Series Nested Cross-Validation</a></p><h4>Common Time Series Cross-Validation Techniques</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QWM5YOHolFjQlpHql6WCaQ.png" /></figure><p>Here’s my implementation for the backtest validation:</p><p><strong>Step 1: Meta-labeling</strong></p><p>Meta-labeling, specific to financial backtesting, is to assign binary signals (True or False, Buy or Sell, Long or Short) based on assets’ performances. Assigning meta labels can help solve a Supervised Machine Learning problem. It can also be used to validate an existing model.</p><p>The goal of our SPHMV model is to detect sectors that can outperform (vs. S&amp;P 500). Any sector that can perform better than the benchmark in the forward 3 months period will be marked as TRUE, else as FALSE.</p><pre>RS = prices.div(prices[tickers[&#39;sp500&#39;]],axis=0)<br>y_true = RS.asfreq(&#39;q&#39;,method=&#39;ffill&#39;)\<br>.pct_change().shift(-1).dropna()\<br>.reset_index().rename(columns={&#39;index&#39;:&#39;Date&#39;})\<br>.melt(id_vars=&#39;Date&#39;,var_name=&#39;Ticker&#39;,value_name=&#39;return&#39;)\<br>.assign(actual=lambda x: x[&#39;return&#39;]&gt;0)</pre><p>We also need to assign binary labels for model predictions. Recall that the SPHMV model will only hold the top 3 scored sectors, so we will assign any weight&gt;0 as TRUE and weight=0 as FALSE.</p><pre>weights = report.get_security_weights(‘SPHMV’)<br>y_pred = weights.asfreq(‘q’,method=’ffill’).shift(-1).dropna()\<br> .reset_index().rename(columns={‘index’:’Date’})\<br> .melt(id_vars=’Date’,var_name=’Ticker’,value_name=’weight’)\<br> .assign(predicted=lambda x: x[‘weight’]&gt;0)</pre><p>Finally, we need to make sure the actuals and predictions are 1:1 matched. This is done using the <strong>merge </strong>function with the “<strong>how=’inner</strong>’” option.</p><pre>y = pd.merge(y_pred,y_true,how=’inner’,on=[‘Date’,’Ticker’])</pre><pre>plt.subplot(121)<br>y[‘actual’].value_counts(sort=False).plot(kind=’pie’,autopct=’%1.1f%%’)<br>plt.subplot(122)<br>y[‘predicted’].value_counts(sort=False).plot(kind=’pie’,autopct=’%1.1f%%’)<br>plt.suptitle(‘Meta-Labeling in Predicted and Actual Cases’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*2l0LLlPPTluR9nnVqlLm1Q.png" /></figure><p><strong>Step 2: Dealing with Sample Biases</strong></p><p>Ideally, we want the outperformance (TRUE) and underperformance (FALSE) cases to be equal (balanced/unbiased) in the actual dataset. Otherwise, we could use <strong><em>resampling</em></strong> to correct the bias. There’s a good tutorial on <a href="https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets"><strong><em>Kaggle</em></strong> </a>talking about how to deal with the sample biases.</p><p><a href="https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets">Resampling strategies for imbalanced datasets</a></p><h4>Basic Resampling Techniques</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*H6XodlitlGDl9YdbwaZLMw.png" /></figure><p>However, we need to be aware of some of the drawbacks:</p><blockquote>The simplest implementation of over-sampling is to duplicate random records from the minority class, which can cause overfitting. In under-sampling, the simplest technique involves removing random records from the majority class, which can cause loss of information.</blockquote><p>It appears that overall our samples are quite balanced (50.6% FALSE vs. 49.4% TRUE). But if I look at the counts on sector levels, the TRUEs and FALSEs can vary quite a bit:</p><pre>y.groupby(‘Ticker’)[‘actual’].value_counts().divide(70)\<br> .unstack().sort_values(by=True,ascending=True)\<br> .plot(kind=’barh’,stacked=True)</pre><pre>plt.xlim(0,1)<br>plt.legend(loc=’right’,bbox_to_anchor=(1.1,0.5))<br>plt.title(‘Sector Probability to Outperform S&amp;P 500 (Original)’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*N8aASdPDabOrJgC8Sn-miQ.png" /></figure><p>To fix this issue, I resampled an equal number of TRUE and FALSE cases in every sector:</p><pre>target = 35</pre><pre>y_balanced = y.groupby([&#39;actual&#39;,&#39;Ticker&#39;])\<br> .apply(lambda x: x.sample(target,replace=True,random_state=42))\<br> .reset_index(drop=True)</pre><pre>y_balanced.groupby(‘Ticker’)[‘actual’].value_counts()\<br> .divide(target*2).unstack()\<br> .sort_values(by=True,ascending=True)\<br> .plot(kind=’barh’,stacked=True)</pre><pre>plt.xlim(0,1)<br>plt.legend(loc=’right’,bbox_to_anchor=(1.1,0.5))<br>plt.title(‘Sector Probability to Outperform S&amp;P 500 (Resampled)’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*ShY1ZAdDC7oKS8T0b-aI6g.png" /></figure><p><strong>Step 3: Measuring Accuracy</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*-mJyoIJ7E3iIzyFQ-6NzyQ.png" /></figure><p>Depending on the goal of your model, data scientists commonly look at precision or recall scores to measure prediction accuracy.</p><p><strong>Precision</strong> is the percentage of True Positive cases among all the predicted positive cases, a.k.a. <strong>TP / (TP + FP)</strong>.</p><p><strong>Recall</strong> is the percentage of True Positive cases among all the true cases in the sample universe, a.k.a. <strong>TP / (TP + FN</strong>).</p><p>Specific to our sector prediction model: We can <strong>only select the top 3</strong> scored sectors but not all. So apparently <strong>Precision</strong>, which measures the success ratio among all of our model selections, is a better accuracy score than Recall, which needs us to pick as many winners as in the total universe.</p><p>The easiest way to review the model accuracy is to use the <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html"><strong>classification report</strong></a> function in the <strong>scikit-learn</strong> package. We can see that the precision score is only 50% in original and 51% in balanced samples. We also need to take into account the randomness of the sample generation in the resampling process. <strong><em>The result probably suggests that the model itself is either very weak or not working well enough in detecting the outperformers.</em></strong></p><pre>print(‘With Origianl Samples:’)<br>print(classification_report(y[‘actual’],y[‘predicted’]))<br>print(‘With Balanced Samples:’)<br>print(classification_report(y_balanced[‘actual’],y_balanced[‘predicted’]))</pre><pre>With Origianl Samples:<br>              precision    recall  f1-score   support</pre><pre>False       0.51      0.73      0.60       390<br>        True       <strong>0.50</strong>      0.27      0.35       380</pre><pre>accuracy                           0.50       770<br>   macro avg       0.50      0.50      0.48       770<br>weighted avg       0.50      0.50      0.48       770</pre><pre>With Balanced Samples:<br>              precision    recall  f1-score   support</pre><pre>False       0.50      0.72      0.59       385<br>        True       <strong>0.51</strong>      0.29      0.37       385</pre><pre>accuracy                           0.51       770<br>   macro avg       0.51      0.51      0.48       770<br>weighted avg       0.51      0.51      0.48       770</pre><p><strong>Step 4: Cross-validation with LOOCV</strong></p><p>What we did so far from step 1 to step 3 is just a single loop of the backtest validation. As I just described at the beginning of the blog, we still need to shuffle the sector population using Leave-One-Out (LOOCV) and rerun backtests multiply times to get a complete and unbiased cross-validation result.</p><p>So here is what I would do to implement the backtest LOOCV:</p><ol><li>Drop 1 and pick 10 instead of all 11 S&amp;P sectors;</li><li>Run the SPHMV backtest;</li><li>Calculate the precision scores (for both original and balanced samples);</li><li>Repeat 1 to 3 steps 11 times so every sector should be dropped once;</li><li>Calculate the average CV scores.</li></ol><p>The prerequisite step is to create two functions specifically for CV loops:</p><p><strong><em>SPHMV_CV(test_nm, sectors, benchmark, prices):</em></strong> for step 2 to run SPHMV backtest in loops, and</p><p><strong><em>backtest_CV(weights, prices):</em></strong> for step 3 to generate y_true and y_pred, the input used for calculating precision scores.</p><p>The codes in the functions are largely the same as I’ve already covered in my <a href="https://medium.com/@richardhwlin/advanced-backtesting-with-bt-635ed441cb60">last</a> blog and the early section of the blog. So I’ll just leave them as your homework and save some time and space here.</p><pre># LOOCV Implementation:</pre><pre>benchmark = tickers[‘sp500’]<br>sectors = tickers[‘sectors’]</pre><pre>report3 = {}<br>for t in sectors:<br> test_nm = ‘X’+t<br> cv_sectors = [i for i in sectors if i!=t]<br> cv_report = SPHMV_CV(test_nm,cv_sectors,benchmark,prices)<br> report3.update({test_nm:cv_report})</pre><pre>cv_prices = pd.concat([v.prices for k,v in report3.items()],axis=1)</pre><pre>cv_prices.plot()<br>plt.title(‘LOOCV Backtests’)<br>plt.legend(loc=’right’,bbox_to_anchor=(1.1,0.5))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*v1idSZkbddx0NDdWEXrLDw.png" /></figure><pre># Calculate CV Precision Scores:</pre><pre>cv_precision = dict()</pre><pre>for test_nm,cv_report in report3.items():<br> RS = prices.divide(prices[‘IVV’],axis=0)<br> weights = cv_report.get_security_weights(test_nm)<br> y,y_balanced = backtest_CV(weights,RS)<br> cv_precision[test_nm] = ( <br>    precision_score(y[‘actual’],y[‘predicted’]),<br>    precision_score(y_balanced[‘actual’],y_balanced[‘predicted’])<br> )</pre><pre>cv_precision = pd.DataFrame.from_dict( cv_precision, <br>               orient=’index’, columns=[‘original’,’balanced’])</pre><pre>cv_avg = cv_precision.mean(axis=0)</pre><p>cv_avg output:</p><pre>original    0.504762<br>balanced    0.504980</pre><p>We still got average CV scores that are very close to .50, which probably conclude our evaluation of the model with a “nothing work” to “very weak” score.</p><pre># Plot CV Precision Scores:<br>cv_precision.sort_values(by=’original’,ascending=False,inplace=True)</pre><pre>fig,axs = plt.subplots(1,2)</pre><pre>cv_precision[‘original’].plot(kind=’bar’,ax=axs[0])<br>axs[0].plot([0, 10], [cv_avg[0], cv_avg[0]], ‘r — ‘, lw=2)<br>axs[0].set_ylim(0.4,0.6)<br>axs[0].set_title(‘Original’)<br>axs[0].text(6,0.52,<br>            ’Average = {0:.4f}’.format(cv_avg[0]),fontsize=12)</pre><pre>cv_precision[‘balanced’].plot(kind=’bar’,ax=axs[1])<br>axs[1].plot([0, 10], [cv_avg[1], cv_avg[1]], ‘b — ‘, lw=2)<br>axs[1].set_ylim(0.4,0.6)<br>axs[1].set_title(‘Balanced’)<br>axs[1].text(6,0.52,<br>            ’Average = {0:.4f}’.format(cv_avg[1]),fontsize=12)</pre><pre>plt.suptitle(‘LOOCV Precision Scores’,fontsize=16)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*cvUn1fUeSf5bOPGiTCQp7A.png" /></figure><h3>Further Discussion</h3><p><em>One might have noticed that the red bar of </em><strong><em>XIYZ </em></strong><em>has the highest precision score of above 0.55 among all the loops. What does it mean? XIYZ means that I have dropped IYZ, the telecom sector, but kept all other sectors in the SPHMV sector rotator models. Does that mean the telecom sector is extremely difficult to predict using the Value+Momentum screen?</em></p><p><em>Another sample is IDU, the utility sector. Now I seem to get some clues.</em><strong><em> Telecom and Utility companies are high dividend payers. </em></strong><em>Investors allured to these stocks for defensive moves, seeking safe haven in the equity space with locked-in dividends. </em><strong><em>Picking them with a short-term-momentum based screen doesn’t seem to be well-fitting their long-term-value driven characteristics.</em></strong></p><p>The modified backtest result looked like below. <strong>Notice that I dropped Telecom and Utility sectors altogether, although dropping Telecom along can give me a better performance number. </strong>I made that discretionary call based on my discussion above. Also, remember that the original sample history we have is <strong>biased</strong>, so I prefer to rely on the<strong> balanced CV results</strong> to make my judgment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*bx_UAEeOZIfj-rpu-UUCRw.png" /><figcaption>Modified SPHMV Model Excluded Telecom and Utility Sectors</figcaption></figure><h3>Conclusion</h3><p>Making investment decisions based only on backtest performance is dangerous. Quantitative modeling is especially the case since it’s often been regarded as the black-box. The input data, usually retrieved from a limited time frame of history, is highly biased and can not produce reliable forecasts. One way to mitigate the risk of running wrong predictions is to recheck your data, using resampling techniques to adjust the sample biases. The other way is to rerun backtests many times but each with slightly different data input — a data analysis technique called cross-validation. Either way, nothing is more important than understanding the economic interpretation underlying your model — so becoming your domain master.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22f52d1f74d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced Backtesting with BT]]></title>
            <link>https://medium.com/@richardhwlin/advanced-backtesting-with-bt-635ed441cb60?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/635ed441cb60</guid>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[quantitative-strategies]]></category>
            <category><![CDATA[investment-research]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Wed, 15 Apr 2020 21:19:04 GMT</pubDate>
            <atom:updated>2020-04-15T21:19:04.671Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tNQU7BEF9ZeY29fL" /><figcaption>Photo by <a href="https://unsplash.com/@samsonyyc?utm_source=medium&amp;utm_medium=referral">Samson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>In my <a href="https://medium.com/@richardhwlin/get-hands-on-with-basic-backtests-fc66442d5d52"><em>first blog</em></a> and <a href="https://medium.com/@richardhwlin/flexible-backtesting-with-bt-7295c0dde5dd"><em>second blog</em></a>, I have introduced basic concepts and shared python codes for financial backtesting. <a href="https://pmorissette.github.io/bt/#"><em>BT</em></a>, the flexible backtesting API for Python, can make that task much quicker and easier.</p><p>In my third blog today, I’ll start looking at a real-world quant modeling problem, such as the <a href="https://eu.spindices.com/indices/strategy/sp-500-sector-rotator-daily-rc2-6-index-tr">S&amp;P 500 Sector Rotator Index (<strong><em>Ticker: SPXSRT6T</em></strong>)</a>.</p><h3>Bringing Rule-Based Access to Sector-Rotation</h3><p>To Understand the Sector Rotator Strategy better, let’s first watch a short video on the S&amp;P’s website:</p><p><a href="http://play.vidyard.com/kp29ECQF5FWX5AVRhpKSG8.html">play.vidyard.com/kp29ECQF5FWX5AVRhpKSG8.html</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplay.vidyard.com%2Fkp29ECQF5FWX5AVRhpKSG8.html%3Fautoplay%3D0%26custom_id%3D%26embed_button%3D0%26viral_sharing%3D0%26&amp;display_name=Vidyard&amp;url=https%3A%2F%2Fshare.vidyard.com%2Fwatch%2Fkp29ECQF5FWX5AVRhpKSG8%3F&amp;image=https%3A%2F%2Fcdn.vidyard.com%2Fthumbnails%2Fcustom%2FCcRcxm5IdrMdXIqkG_SR2w.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vidyard" width="600" height="337" frameborder="0" scrolling="no"><a href="https://medium.com/media/376c962b3d2b7be06d7212df14acd830/href">https://medium.com/media/376c962b3d2b7be06d7212df14acd830/href</a></iframe><p>Based on the video described, S&amp;P sectors behaved differently during the different stages of US economic cycles, so there’s a strong need for sector-rotation. The problem is on how-to. The challenge of successful sector-rotation is on the accurate prediction of macroeconomic indicators such as inflation, interest rate, GDP or unemployment, etc.</p><p>S&amp;P brings in a different approach by looking at two of the long-lasting investment principals, value and momentum, and uses a rule-based approach to select sectors.</p><p>The sector selection rule seems to be straightforward:</p><ol><li>Select Top 5 value scored (most fundamentally discounted) sectors based on 10 Years of historical data;</li><li>Select Top 3 momentum scored (best 6 months performance) sectors from the Top 5 value sectors, and equal-weight them.</li></ol><p>So by doing that, the index captures the sectors with the greatest relative values (across time) and the greatest relative momentum (across sectors).</p><p>3. Finally, the index can also provide better risk management by dynamically allocating assets to US T-bills should the index volatility exceeds a target level, i.e. 10%, 20%, etc.</p><h3>Data Download and Wrangling</h3><p>The first step for any quantitative research is, of course, investigating your data. Similar to many other quantitative finance blogs, I used iShares’ S&amp;P sector ETF line-ups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*2BweGHQ2EZuu7i_7y9hOUQ.png" /></figure><pre>from yahoo_fin import stock_info as si</pre><pre>tickers = {‘sp500’:’IVV’, ‘cash’:’SHY’, ‘sectors’:[‘IYC’,’IDU’,’IYZ’,’IYW’,’IYJ’,’IYH’,’IYF’,’IYE’,’IYK’,’IYM’,’IYR’]}</pre><pre># Download Data</pre><pre>symbols = tickers[&#39;sectors&#39;]+[tickers[&#39;sp500&#39;],tickers[&#39;cash&#39;]]<br>data = {k:si.get_data(k) for k in symbols}<br>prices = pd.DataFrame.from_dict({k:v.adjclose for k,v in data.items()}).dropna()</pre><pre># Plot Performance</pre><pre>prices[ [tickers[‘sp500’]] + tickers[‘sectors’] ].rebase().plot()<br>plt.title(‘S&amp;P Sector Performance’,{‘fontsize’:16})<br>plt.legend(ncol=3)<br>plt.show()</pre><pre># Print Simple Stats</pre><pre>mtest = prices[ [tickers[‘sp500’]] + tickers[‘sectors’] ].asfreq(‘m’,method=’ffill’).pct_change().dropna()<br>pd.DataFrame.from_dict({<br> ‘annualized monthly return’:mtest.mean()*12*100,<br> ‘annualized monthly volatility’:mtest.std()*np.sqrt(12)*100,<br> ‘annualized monthly sharpe’:mtest.mean() / mtest.std() * np.sqrt(12),<br>}).T.round(2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*CN50wd-lHhqmCjniWxfAjg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*DP25mA0C6kOJM4lCrntXmw.jpeg" /></figure><h3>Building the High-Momentum-Value Algo</h3><p>Now let’s go to the next step to build the sector rotation algorithm.</p><p><strong>Step 1: Building the Value Scores</strong></p><p>According to S&amp;P’s original <a href="https://eu.spindices.com/documents/methodologies/methodology-sp-500-high-momentum-value-sector-rotation.pdf?force_download=true">methodology document</a>, the overall sector value scores are set to be the simple average of the Z-scores of three fundamental factors:</p><ol><li><strong>Book-Value to Price Ratio</strong></li><li><strong>Earnings to Price Ratio</strong></li><li><strong>Sales to Price Ratio</strong></li></ol><p>Because of the difficulties of getting reliable historical financial data and producing sector aggregation (unless you’re willing to pay a decent amount to the financial data vendors), I just use the historical prices to approximate the value score calculation:</p><p><strong>Price Based Value Score = Current Price / Historical Highest Price -1</strong></p><p>This is essentially the same as a drawdown calculation. So I took the shortcut to directly call <strong>bt(ffn)</strong>’s embedded <a href="https://pmorissette.github.io/ffn/ffn.html?highlight=drawdown#ffn.core.to_drawdown_series"><strong>drawdown</strong></a><strong> </strong>function.</p><pre>class StatDrawdown(bt.Algo):</pre><pre>def __init__(self, lookback=pd.DateOffset(months=3),<br> lag=pd.DateOffset(days=0)):<br> super(StatDrawdown, self).__init__()<br> self.lookback = lookback<br> self.lag = lag</pre><pre>def __call__(self, target):<br> selected = target.temp[‘selected’]<br> t0 = target.now — self.lag<br> prc = target.universe.loc[(t0 — self.lookback):t0,selected]<br> target.temp[‘stat’] = prc.<strong>to_drawdown_series</strong>().iloc[-1] <br> return True</pre><p><strong>Step 2: Build the Momentum Scores</strong></p><p>In S&amp;P’s original <a href="https://eu.spindices.com/documents/methodologies/methodology-sp-500-high-momentum-value-sector-rotation.pdf?force_download=true">methodology document</a>, <strong>the momentum scores are calculated using the six months’ total return and adjusted for the sector index’s daily volatility</strong>. The document itself didn’t state precisely whether it applied the square root rule when adjusting for risk. For Simplicity (<em>and soundness of mathematics</em>), I decided to use <strong>bt(ffn)</strong>’s <a href="https://pmorissette.github.io/ffn/_modules/ffn/core.html#calc_information_ratio"><strong>information ratio</strong></a> function to calculate the Momentum Score.</p><pre>class StatInfoRatio(bt.Algo):</pre><pre>def __init__(self, benchmark, lookback=pd.DateOffset(months=3),<br> lag=pd.DateOffset(days=0)):<br> super(StatInfoRatio, self).__init__()<br> self.benchmark = benchmark<br> self.lookback = lookback<br> self.lag = lag</pre><pre>def __call__(self, target):<br> selected = target.temp[‘selected’]<br> t0 = target.now — self.lag<br> prc = target.universe.loc[(t0 — self.lookback):t0,selected].pct_change().dropna()<br> bmk = target.universe.loc[(t0 — self.lookback):t0,self.benchmark].pct_change().dropna()<br> target.temp[‘stat’] = pd.Series({p:prc[p].<strong>calc_information_ratio</strong>(bmk) for p in prc})<br> return True</pre><p><strong>Step 3: Build Strategy and Run Backtest</strong></p><p>Once I have created the Algo classes for Value and Momentum Scores calculation, I’m ready to wrap up all steps to build the SPHMV strategy and backtest.</p><p>First, I created a passive benchmark that is just <strong>S&amp;P 500</strong>:</p><pre>sp500 = bt.Strategy(‘SP500’,<br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.SelectThese([tickers[‘sp500’]]),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()],<br>)</pre><p>Secondly, I created another benchmark that<strong> equally weights eleven S&amp;P sectors</strong>:</p><pre>spesw = bt.Strategy(‘SPESW’,<br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.SelectThese(tickers[‘sectors’]),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()],<br>)</pre><p>Now, this is the <strong>S&amp;P High Momentum Value</strong> Strategy:</p><pre>sphmv = bt.Strategy(‘SPHMV’,<br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.SelectThese(tickers[‘sectors’]),<br> StatDrawdown(lookback=pd.DateOffset(years=10)),<br> bt.algos.SelectN(5,sort_descending=False),<br> StatInfoRatio( benchmark=tickers[‘sp500’],<br>                lookback=pd.DateOffset(months=7),<br>                lag=pd.DateOffset(months=1) ),<br> bt.algos.SelectN(3,sort_descending=True), <br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()],<br>)</pre><p>Finally, it’s time to <strong>run backtests</strong> for all three strategies:</p><pre># run backtests<br>backtest_sp500 = bt.Backtest(sp500,prices)<br>backtest_spesw = bt.Backtest(spesw,prices)<br>backtest_sphmv = bt.Backtest(sphmv,prices)<br>report = bt.run(backtest_sp500, backtest_spesw, backtest_sphmv)</pre><pre># Plot performance<br>report.plot()<br>plt.title(&#39;S&amp;P Sector Rotator Strategy&#39;)<br>plt.show()</pre><pre># Plot Sector Weights<br>report.backtests[&#39;SPHMV&#39;].security_weights.plot.area()<br>plt.title(&quot;Security Weights for S&amp;P Sector Rotator Strategy&quot;)<br>plt.legend(ncol=2,loc=&#39;right&#39;,bbox_to_anchor=(1.2,0.5))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*XLUhcN40To2qJCcbcqJxHg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vB56FENcKXheoJXOB9YO_w.png" /></figure><h3>Add Daily Volatility Control</h3><p>To add volatility control, we need to implement the <strong>Pooled Portfolio</strong> concept as I introduced in my <a href="https://medium.com/@richardhwlin/flexible-backtesting-with-bt-7295c0dde5dd"><em>second blog</em></a>.</p><p>First, to calculate the target equity weights based on volatility. I’m tracking the rolling 20 days volatility of the S&amp;P 500 with a maximum annualized target of 20%.</p><pre><strong>target_vol = 20</strong><br>real_vol = prices[<strong>tickers[‘sp500’]</strong>].pct_change()\<br>           .<strong>rolling(20)</strong>.std().multiply(np.sqrt(252)*100)<br><strong>w = real_vol.map(lambda x: min(1,target_vol/x) )</strong><br>target_weight = pd.DataFrame({‘SPHMV’:w,’SHY’:1-w})</pre><pre>target_weight.plot()<br>plt.title(“Allocation Weights for Daily Volatility Control”)<br>plt.legend(loc=’right’,bbox_to_anchor=(1.1,0.5))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*xghJg0KlO1p0biUYMDAHFA.png" /></figure><p>Now to apply the target weight on a pooled portfolio with SPHMV and SHY (cash) using the <a href="https://pmorissette.github.io/bt/bt.html#bt.algos.WeighTarget"><strong><em>bt.algos.WeighTarget()</em></strong></a>.</p><pre># daily volatility control</pre><pre>spdvc = bt.Strategy(‘SPDVC’,<br> algos = [bt.algos.RunDaily(),<br> bt.algos.SelectAll(),<br> bt.algos.<strong>WeighTarget</strong>(target_weight),<br> bt.algos.Rebalance()],<br> children = [sphmv,tickers[‘cash’]] <br>)</pre><pre>backtest_spdvc = bt.Backtest(spdvc,prices)<br>report2 = bt.run(backtest_sp500, backtest_sphmv,backtest_spdvc)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*62qKIJmZpw04bre5BO3ryQ.png" /></figure><pre><strong>report2.set_date_range(end=’2020–1–31&#39;)<br>report2.set_riskfree_rate(0.01)<br>report2.display()</strong></pre><pre>Stat                 SP500       SPHMV       SPDVC<br>-------------------  ----------  ----------  ----------<br>Start                2002-07-29  2002-07-29  2002-07-29<br>End                  2020-01-31  2020-01-31  2020-01-31<br>Risk-free rate       1.00%       1.00%       1.00%<br><br>Total Return         405.13%     474.86%     572.18%<br>Daily Sharpe         0.55        0.52        0.67<br>Daily Sortino        0.86        0.82        1.09<br>CAGR                 9.69%       10.51%      11.50%<br>Max Drawdown         -55.25%     -61.36%     -41.35%<br>Calmar Ratio         0.18        0.17        0.28</pre><pre>MTD                  -0.00%      1.28%       1.28%<br>3m                   6.67%       5.30%       5.30%<br>6m                   9.26%       8.94%       8.61%<br>YTD                  -0.00%      1.28%       1.28%<br>1Y                   21.61%      22.42%      22.05%<br>3Y (ann.)            14.48%      9.97%       8.62%<br>5Y (ann.)            12.04%      10.43%      9.68%<br>10Y (ann.)           13.75%      11.03%      10.51%<br>Since Incep. (ann.)  9.69%       10.51%      11.50%<br><br><em>(omitted several lines here)</em><br><br>Avg. Drawdown        -1.71%      -2.53%      -2.32%<br>Avg. Drawdown Days   24.00       30.91       28.27<br>Avg. Up Month        2.92%       3.62%       3.35%<br>Avg. Down Month      -3.46%      -3.93%      -3.33%<br>Win Year %           83.33%      72.22%      72.22%<br>Win 12m %            86.50%      76.50%      76.50%</pre><p>Adding Daily Volatility Control seems to be beneficial to the strategy performance - our new portfolio SPDVC is outperforming both the SP500 and SPHMV. However, just keep in mind that <strong><em>timely execution is the key to this type of strategy</em></strong> as this is a daily tracked strategy and very sensitive to the price fluctuations. <strong><em>In another word, good numbers from backtesting are very difficult to be fully materialized in real-life trade books.</em></strong></p><h3>Conclusion:</h3><p>In today’s blog, I have demonstrated how to use BT to tackle the real-world quantitative investing questions — starting with a relatively easy task to replicate the <strong><em>S&amp;P Sector Rotator</em></strong> <strong><em>Strategy</em></strong>.</p><p>By building customized algorithms and using BT’s advanced functions such as <strong><em>SelectN</em></strong> or <strong><em>WeighTarget</em></strong>, I was able to implement the High-Momentum-Value and Daily-Volatility-Control Algos and successfully run the backtests.</p><p>The backtest result seems to outperform the benchmark S&amp;P 500 and a sector-equal-weighted S&amp;P. But, keep in mind that backtest numbers are more like science fiction if they remain untested. For my next blog, I’m going to share some thoughts on how to validate the backtest results.</p><p>Thanks for reading and stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=635ed441cb60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Flexible Backtesting with BT]]></title>
            <link>https://medium.com/@richardhwlin/flexible-backtesting-with-bt-7295c0dde5dd?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7295c0dde5dd</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[backtesting]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Thu, 12 Mar 2020 19:32:26 GMT</pubDate>
            <atom:updated>2020-03-12T19:35:57.887Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A1tHKL3sFsoOfW0P" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><blockquote><strong>Quantitative investing can be Simple, Easy, Awesome.</strong></blockquote><p>In my first blog “<a href="https://medium.com/@richardhwlin/get-hands-on-with-basic-backtests-fc66442d5d52"><em>Get Hands-on with Basic Backtests</em></a>”, I have demonstrated how to use python to quickly backtest some simple quantitative strategies.</p><p>Backtesting more sophisticated strategies is also easy if you can use open-sourced third-party APIs such as <a href="https://pmorissette.github.io/bt/#"><strong>B</strong>T</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*62LSRIH5LdtuvTVUfQMVsw.png" /><figcaption>bt — Flexible Backtesting for Python</figcaption></figure><h3>What is bt?</h3><p><a href="https://pmorissette.github.io/bt/#"><strong>B</strong>T</a> is a flexible backtesting framework for Python used to test quantitative trading strategies. This framework allows you to easily create strategies that mix and match different Algos. It aims to foster the creation of easily testable, re-usable and flexible blocks of strategy logic to facilitate the rapid development of complex trading strategies. It saves quants tons of time in development and lets them focus on the important part of the job — research.</p><p>Now let’s explore the rich functionalities in <strong>BT</strong> together!</p><h3>Data Collection and Data Wrangling</h3><p><strong><em>bt.data.get</em></strong> is the data download function in <strong>BT</strong> package:</p><pre># download prices using bt<br>import bt</pre><pre>tickers = {<br>‘equity’:[‘ITOT’, ’IVV’, ’IJH’, ’IJR’, ’IUSG’, ’IUSV’, ’IJK’, ’IJJ’, ’IJS’, ’IJT’, ’OEF’, ’IWC’],<br>‘bond’:[‘AGG’, ’LQD’, ’GOVT’, ’MBB’, ’MUB’, ’TIP’, ’SHY’, ’IEF’, ’TLT’, ’HYG’, ’FLOT’, ’CMBS’],<br>}</pre><pre>prices = <strong><em>bt.data.get</em></strong>(tickers[‘equity’] + tickers[‘bond’], clean_tickers=False)</pre><p>It is also useful to align prices with bt’s <strong><em>rebase</em></strong> function:</p><pre># display Equity ETFs</pre><pre>prices[tickers[‘equity’]].<strong><em>rebase()</em></strong>.plot()<br>plt.title(‘Equity ETF Prices’,{‘fontsize’:16})<br>plt.legend(ncol=3)<br>plt.show()</pre><pre># performance stats for backtests</pre><pre>mtest = prices[tickers[‘equity’]].asfreq(‘m’,method=’ffill’).pct_change().dropna()<br>pd.DataFrame.from_dict({<br> ‘annualized monthly return’:mtest.mean()*12*100,<br> ‘annualized monthly volatility’:mtest.std()*np.sqrt(12)*100,<br> ‘annualized monthly sharpe’:mtest.mean() / mtest.std() * np.sqrt(12),<br>}).T.round(2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*hVBJmeGiPcT0s8Xef9IMdA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*4DIzjFSUJ_sqxczzt5jpaQ.png" /></figure><pre># Display Bond ETFs </pre><pre>prices[tickers[‘bond’]].<strong><em>rebase()</em></strong>.plot()<br>plt.title(‘Bond ETF Prices’,{‘fontsize’:16})<br>plt.legend(ncol=3)<br>plt.show()</pre><pre># performance stats for backtests</pre><pre>mtest = prices[tickers[‘bond’]].asfreq(‘m’,method=’ffill’).pct_change().dropna()<br>pd.DataFrame.from_dict({<br> ‘annualized monthly return’:mtest.mean()*12*100,<br> ‘annualized monthly volatility’:mtest.std()*np.sqrt(12)*100,<br> ‘annualized monthly sharpe’:mtest.mean() / mtest.std() * np.sqrt(12),<br>}).T.round(2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*-kWrRB-s5J44eFVEuE9mRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*xz2Vy_vyfvIvpfDUOp2wEw.png" /></figure><p>You can use <strong>BT</strong>’s embedded <strong><em>ffn.calc_stats</em></strong> function to calculate a comprehensive group of pre-packaged performance statistics:</p><pre>pd.DataFrame.from_dict(<br>	{k:v.stats for k,v in <strong><em>bt.ffn.calc_stats</em></strong>(prices).items()}<br>)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gsknVwvLxBQV1XqgKKuWlg.png" /></figure><p>It saved me so much time in just coding all these performance and risk calculations. <strong><em>Can’t love anymore!</em></strong></p><h3>The Simplest Form of Portfolio</h3><p>In my first blog “<a href="https://medium.com/@richardhwlin/get-hands-on-with-basic-backtests-fc66442d5d52"><em>Get Hands-on with Basic Backtests</em></a>”, I have shown how to set up fixed-weighted portfolios such as the <strong>80% equity / 20% bond</strong> for <strong>aggressive </strong>portfolio, the <strong>60% equity / 40% bond</strong> for <strong>moderate </strong>portfolio and the <strong>40% equity / 60% bond</strong> for <strong>conservative </strong>portfolio. The same setup is equally simple and straightforward in <strong>BT</strong>.</p><p>Step 1: Define Strategy in <strong>bt.Strategy</strong></p><pre>aggressive = <strong><em>bt.Strategy</em></strong>(‘aggressive’, <br> <strong><em>algos = [</em></strong><br> bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.WeighSpecified(ITOT=0.8,AGG=0.2),<br> bt.algos.Rebalance(),<br> <strong><em>]</em></strong><br>)</pre><pre>moderate = <strong><em>bt.Strategy</em></strong>(‘moderate’, <br> <strong><em>algos = [</em></strong><br> bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.WeighSpecified(ITOT=0.6,AGG=0.4),<br> bt.algos.Rebalance(),<br> <strong><em>]</em></strong><br>)</pre><pre>conservative = <strong><em>bt.Strategy</em></strong>(‘conservative’, <br> <strong><em>algos = [</em></strong><br> bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.WeighSpecified(ITOT=0.4,AGG=0.6),<br> bt.algos.Rebalance(),<br> <strong><em>]</em></strong><br>)</pre><p>Step 2: Define Backtest in <strong>bt.Backtest</strong></p><pre>backtest_aggressive = <strong>bt.Backtest</strong>(aggressive,prices)<br>backtest_moderate = <strong>bt.Backtest</strong>(moderate,prices)<br>backtest_conservative = <strong>bt.Backtest</strong>(conservative,prices)</pre><p>Step 3: Run Backtests with <strong>bt.run</strong></p><pre>report = <strong><em>bt.run</em></strong>(backtest_aggressive, backtest_moderate, backtest_conservative)</pre><pre><strong><em>report.plot()</em></strong><br>plt.title(“Fixed Weighted Portfolio”)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*8Y70AGgk_5DhGb7biqthIw.png" /></figure><pre>aggressive<br>0% [############################# ] 100% | ETA: 00:00:00<br>moderate<br>0% [############################# ] 100% | ETA: 00:00:00<br>conservative<br>0% [############################# ] 100% | ETA: 00:00:00</pre><p><strong><em>Voila! Backtesting can’t be easier with BT!</em></strong></p><p>What is even better with BT is its well-designed report functions. For example, to show lookback returns:</p><pre><strong><em>report.display_lookback_returns()</em></strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*5WkmpDETouJGwoBAQ-EclA.png" /></figure><p>And the nice layout of monthly returns:</p><pre><strong><em>report.display_monthly_returns</em></strong>(backtest=&#39;aggressive&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*Zx_viWgsJ_bWWCL5U39C-A.png" /></figure><p>Or to print the complete performance stats with customizable risk-free rate setting:</p><pre><strong><em>report.set_riskfree_rate</em></strong>(0.01)<br><strong><em>report.display</em></strong>()</pre><pre>Stat                 aggressive    moderate    conservative<br>-------------------  ------------  ----------  --------------<br>Start                2012-02-23    2012-02-23  2012-02-23<br>End                  2020-03-06    2020-03-06  2020-03-06<br>Risk-free rate       1.00%         1.00%       1.00%<br><br>Total Return         122.07%       96.32%      72.53%<br>Daily Sharpe         0.90          0.99        1.15<br>Daily Sortino        1.42          1.58        1.86<br>CAGR                 10.44%        8.76%       7.03%<br>Max Drawdown         -15.80%       -11.46%     -7.33%<br>Calmar Ratio         0.66          0.76        0.96<br><br>MTD                  -0.38%        0.15%       0.66%<br>3m                   -4.06%        -1.63%      0.76%<br>6m                   0.60%         1.89%       3.04%<br>YTD                  -6.15%        -3.27%      -0.40%<br>1Y                   8.21%         9.67%       10.98%<br>3Y (ann.)            8.44%         7.97%       7.37%<br>5Y (ann.)            8.17%         7.29%       6.31%<br>10Y (ann.)           10.44%        8.76%       7.03%<br>Since Incep. (ann.)  10.44%        8.76%       7.03%<br><br>Daily Sharpe         0.90          0.99        1.15<br>Daily Sortino        1.42          1.58        1.86<br>Daily Mean (ann.)    10.50%        8.71%       6.94%<br>Daily Vol (ann.)     10.52%        7.76%       5.19%<br>Daily Skew           -0.43         -0.42       -0.37<br>Daily Kurt           3.25          3.01        2.59<br>Best Day             3.59%         2.51%       1.57%<br>Worst Day            -3.43%        -2.54%      -1.66%<br><br>Monthly Sharpe       1.02          1.12        1.27<br>Monthly Sortino      1.80          2.05        2.49<br>Monthly Mean (ann.)  10.33%        8.60%       6.87%<br>Monthly Vol (ann.)   9.12%         6.78%       4.62%<br>Monthly Skew         -0.62         -0.55       -0.43<br>Monthly Kurt         0.90          0.78        0.58<br>Best Month           6.94%         5.44%       3.93%<br>Worst Month          -6.81%        -4.66%      -3.32%<br><br>Yearly Sharpe        0.75          0.79        0.86<br>Yearly Sortino       3.89          5.45        10.80<br>Yearly Mean          10.35%        8.48%       6.64%<br>Yearly Vol           12.54%        9.42%       6.56%<br>Yearly Skew          -0.03         -0.01       0.17<br>Yearly Kurt          -1.55         -1.52       -1.00<br>Best Year            26.14%        21.66%      17.21%<br>Worst Year           -6.15%        -3.27%      -1.48%<br><br>Avg. Drawdown        -1.15%        -0.90%      -0.56%<br>Avg. Drawdown Days   14.39         13.95       12.94<br>Avg. Up Month        2.06%         1.61%       1.18%<br>Avg. Down Month      -2.58%        -1.84%      -1.18%<br>Win Year %           75.00%        75.00%      75.00%<br>Win 12m %            95.40%        95.40%      96.55%</pre><h3>The More Active Rotation Portfolios</h3><p>we can also use the <strong>bt.algos</strong> functions to backtest more sophisticated active portfolios. For example, the similar price momentum strategies I demonstrated in my <a href="https://medium.com/@richardhwlin/get-hands-on-with-basic-backtests-fc66442d5d52">first blog</a> can also be easily replicated under the <strong>BT</strong> framework:</p><pre>m3m = bt.Strategy(‘3m’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=4,lookback=pd.DateOffset(months=3)),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()]<br>)</pre><pre>m6m = bt.Strategy(‘6m’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=4,lookback=pd.DateOffset(months=6)),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()]<br>)</pre><pre>m9m = bt.Strategy(‘9m’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=4,lookback=pd.DateOffset(months=9)),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()]<br>)</pre><pre>m1y = bt.Strategy(‘1y’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=4,lookback=pd.DateOffset(years=1)),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()]<br>)</pre><pre>backtest_m3m = bt.Backtest(m3m,prices[tickers[‘equity’]])<br>backtest_m6m = bt.Backtest(m6m,prices[tickers[‘equity’]])<br>backtest_m9m = bt.Backtest(m9m,prices[tickers[‘equity’]])<br>backtest_m1y = bt.Backtest(m1y,prices[tickers[‘equity’]])</pre><pre>report2 = bt.run(backtest_m3m,backtest_m6m,backtest_m9m,backtest_m1y)</pre><pre>report2.prices[&#39;2013-3-31&#39;:].plot()<br>plt.title(“Equity Rotation Portfolio”)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*8-w8VS7QW56A3r76e82-xw.png" /></figure><h3>Portfolio Optimization</h3><p>In addition to the Equal-weights,<strong> BT</strong> also supports several advanced portfolio construction techniques such as Mean-Variance Optimization, Equal Risk Contribution, and Inversed Volatility.</p><pre>MeanVar = bt.Strategy(‘MeanVar’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.WeighMeanVar(lookback=pd.DateOffset(years=1),),<br> bt.algos.Rebalance()]<br>)</pre><pre>ERC = bt.Strategy(‘ERC’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.WeighERC(lookback=pd.DateOffset(years=1),),<br> bt.algos.Rebalance()]<br>)</pre><pre>InvVol = bt.Strategy(‘InvVol’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.WeighInvVol(lookback=pd.DateOffset(years=1),),<br> bt.algos.Rebalance()]<br>)</pre><pre>backtest_mv = bt.Backtest(MeanVar,prices[tickers[‘equity’]])<br>backtest_erc = bt.Backtest(ERC,prices[tickers[‘equity’]])<br>backtest_iv = bt.Backtest(InvVol,prices[tickers[‘equity’]])</pre><pre>report3 = bt.run(backtest_mv,backtest_erc,backtest_iv)</pre><pre>report3.prices[‘2013–3–31’:].plot()<br>plt.title(“Equity Alt-Weighted Portfolio”)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*SRi6XMaFrjDY4PhB3P48JQ.png" /></figure><h3>Creating Pooled Portfolio Strategy</h3><p>Portfolio of Portfolios, including Fund of Funds (FoFs) or ETF of ETFs, are pooled portfolio structures aiming to achieve broad diversification and minimal risk. It usually involves two layers of investment decisions: asset allocation and sector/security selection.</p><p>In the following example, I use <strong><em>80% Equity / 20% Bond fixed allocation</em></strong> and overlay with <strong><em>price momentum based active sector</em></strong> strategies. You’ll see that it’s easy to do with the <strong><em>children</em></strong> parameter.</p><pre>equity = bt.Strategy(‘equity’,<br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=2,lookback=pd.DateOffset(months=3)),<br> bt.algos.WeighInvVol(lookback=pd.DateOffset(years=1),),<br> bt.algos.Rebalance()],<br> <strong><em>children = tickers[‘equity’]</em></strong><br>)</pre><pre>bond = bt.Strategy(‘bond’,<br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.RunAfterDate(‘2013–3–31’),<br> bt.algos.SelectAll(),<br> bt.algos.SelectMomentum(n=2,lookback=pd.DateOffset(months=3)),<br> bt.algos.WeighEqually(),<br> bt.algos.Rebalance()],<br> <strong><em>children = tickers[‘bond’]</em></strong><br>)</pre><pre>pooled = bt.Strategy(‘pooled’, <br> algos = [bt.algos.RunQuarterly(),<br> bt.algos.SelectAll(),<br> bt.algos.WeighSpecified(equity=0.8,bond=0.2),<br> bt.algos.Rebalance()], <br> <strong><em>children = [equity,bond]</em></strong><br>)</pre><pre>backtest_equity = bt.Backtest(equity,prices)<br>backtest_bond = bt.Backtest(bond,prices)<br>backtest_pooled = bt.Backtest(pooled,prices)</pre><pre>report4 = bt.run(backtest_equity, backtest_bond, backtest_pooled)</pre><pre>report4.prices[‘2013–3–31’:].plot()<br>plt.title(“Pooled Portfolio — aggressive(80/20)”)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*sV5A46pMY7BPkRRmm4irVg.png" /></figure><p>Here’re the underlying security holdings over time:</p><pre>report4.get_security_weights(‘pooled’)[‘2013–3–31’:].plot.area()<br>plt.title(“Security Weights for Pooled Strategy”)<br>plt.legend(ncol=2,loc=’right’,bbox_to_anchor=(1.2,0.5))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nCqbOsZWGNk8cmqFJtYKvA.png" /></figure><p>And historical drawdowns:</p><pre>report4.backtests[‘pooled’].stats.drawdown[‘2013–3–31’:].plot()<br>plt.title(“Drawdowns for Pooled Strategy”)<br>plt.show()</pre><pre>print(‘Top 5 Drawdowns:’)<br>report4.backtests[‘pooled’].stats.drawdown_details.sort_values(‘drawdown’).head(5)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*GtvYj7YMDWUOvaMgRG4tQw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*1zRHOEsJyqFHxWRoa3Ua9g.png" /></figure><p>One last block of codes is to show the nicely formatted print for single strategy performance:</p><pre><strong><em>print(“Performance Summary for Pooled Strategy:\n”)<br>report4.backtests[‘pooled’].stats.set_riskfree_rate(0.01)<br>report4.backtests[‘pooled’].stats.display()</em></strong></pre><pre>Performance Summary for Pooled Strategy:<br><br>Stats for price from 2012-02-23 00:00:00 - 2020-03-11 00:00:00<br>Annual risk-free rate considered: 1.00%<br>Summary:<br>Total Return      Sharpe  CAGR    Max Drawdown<br>--------------  --------  ------  --------------<br>68.57%              0.53  6.70%   -20.06%<br><br>Annualized Returns:<br>mtd     3m       6m      ytd      1y      3y     5y     10y    incep.<br>------  -------  ------  -------  ------  -----  -----  -----  --------<br>-9.34%  -14.00%  -9.36%  -15.72%  -2.85%  3.05%  5.60%  6.70%  6.70%<br><br>Periodic:<br>        daily    monthly    yearly<br>------  -------  ---------  --------<br>sharpe  0.53     0.56       0.46<br>mean    7.18%    7.07%      7.61%<br>vol     11.61%   10.93%     14.32%<br>skew    -0.92    -0.28      -0.39<br>kurt    8.22     1.31       -0.94<br>best    4.18%    9.91%      25.06%<br>worst   -7.30%   -9.34%     -15.72%<br><br>Drawdowns:<br>max      avg       # days<br>-------  ------  --------<br>-20.06%  -1.98%     23.37<br><br>Misc:<br>---------------  ------<br>avg. up month    2.62%<br>avg. down month  -1.86%<br>up year %        62.50%<br>12m up %         83.91%<br>---------------  ------</pre><h3>Conclusion</h3><p>In this blog I have demonstrated the rich functionalities of BT — the open-source API of Flexible Backtesting for Python.</p><p>BT is capable of conducting backtestings in various ways: I started from fixed weighted portfolios, price momentum based active portfolios, to mean-variance optimization and minimum volatility weighted portfolios. In the last example I showed how to construct a pooled portfolio with BT.</p><p>BT also provides comprehensive risk and performance measures. Along with all the nicely designed charts, tables and reports, BT is one of the best friends for quants.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7295c0dde5dd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Get Hands-on with Basic Backtests]]></title>
            <link>https://medium.com/@richardhwlin/get-hands-on-with-basic-backtests-fc66442d5d52?source=rss-19f76801d2b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc66442d5d52</guid>
            <category><![CDATA[learning-to-code]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Richard L]]></dc:creator>
            <pubDate>Tue, 03 Mar 2020 05:15:34 GMT</pubDate>
            <atom:updated>2020-03-03T15:58:55.132Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KcBbHrhlI-vP4Vsb" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><blockquote><strong>Quantitative investing can be Simple, Easy, Awesome.</strong></blockquote><p>For quantitative researchers, their day to day responsibilities are just one of the three following tasks: <strong><em>Security Selection, Portfolio and Risk Management, </em></strong>and<strong><em> Automatic Trade Execution</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ctXavBrb6dSfXOqDkTBHwA.jpeg" /></figure><p><strong><em>Backtesting</em></strong>, according to <a href="https://www.investopedia.com/terms/b/backtesting.asp">Investopedia</a>, is the general method for seeing how well a strategy or model would have done ex-post. Backtesting involves in every stage of the quantitative research process — from security selection, portfolio construction to automated trading. Backtesting normally is conducted with historical data, but I believe it actually can be more valuable with the simulated data, another term a.k.a <strong><em>Scenario Analysis </em></strong>or <strong><em>Stress Test</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*qsTdh4aAMb9-IiUgiuESrQ.png" /><figcaption>Source: <a href="https://www.investopedia.com/terms/b/backtesting.asp">https://www.investopedia.com/terms/b/backtesting.asp</a></figcaption></figure><h3>Data Collection and Data Wrangling</h3><p>Let me have a word for friends just starting to study quantitative finance: in order to get free pricing data, the easiest way is from <strong><em>Yahoo! Finance:</em></strong></p><p><a href="https://finance.yahoo.com/">Yahoo Finance - Stock Market Live, Quotes, Business &amp; Finance News</a></p><pre># download prices from yahoo finance</pre><pre>from yahoo_fin import stock_info as si<br>assets = { ‘equity’:’IVV’, ’bond’:’AGG’, ’cash’:’SHY’, ’em’:’EEM’, ’dm’:’EFA’ }<br>prices = {k:si.get_data(v) for k,v in assets.items()}</pre><p>The raw price data I downloaded includes seven fields: <strong><em>open, close, high, low, adjclose, volume, ticker</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/533/1*1S2pZAcU4RbeBeYsn_UPYQ.png" /></figure><p>I prefer to use the field <strong><em>adjclose</em></strong> as the most accurate version of <strong><em>total return</em></strong> because it counts for the reinvestment of dividends and adjusted historical prices for all corporate actions.</p><pre># transfer and plot total retun prices</pre><pre>totidx = pd.DataFrame.from_dict({<br>    k:v.adjclose for k,v in prices.items()<br>})\<br>.dropna()\<br>.apply(lambda x: x/x.iloc[0]*1000)</pre><pre>totidx.plot()<br>plt.title(‘Total Return Index with Base Value = 1000’, {‘fontsize’:16})<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*uGgb4eGhE42-v2U824R8DA.png" /></figure><p>It&#39;s also worth reviewing some quick <strong><em>Performance Statistics</em></strong>. <strong><em>Emerging Market</em></strong> (em) observed the highest return but is also the most volatile. It looks that <strong><em>US Equity</em></strong> (equity) is the best-performed asset class within our review history.</p><pre># show some basic performance stats</pre><pre>mret = totidx.asfreq(‘m’,method=’ffill’).pct_change().dropna()<br>pd.DataFrame.from_dict({<br> ‘annualized monthly return’:mret.mean()*12*100,<br> ‘annualized monthly volatility’:mret.std()*np.sqrt(12)*100,<br> ‘annualized monthly sharpe’:mret.mean() / mret.std() * np.sqrt(12),<br>}).T.round(2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/1*gJWr7rtBHjk_nkYpieK5Cw.png" /></figure><h3>The Simplest Form of Portfolio</h3><p>The simplest form of a portfolio is a fixed-weighted two-asset portfolio, which you only need to choose the allocation between equity and bond and keep them constant forever.</p><p>Financial advisors usually start with asking for your risk tolerance: are you more <strong><em>aggressive</em></strong>, <strong><em>moderately aggressive</em></strong> or<strong><em> conservative</em></strong>? These will be transformed into different weights between equity and bond. I will cover more about this topic in the future under the category of <strong><em>Robo-Advising</em></strong>.</p><pre># create fixed-weighted portfolios based on risk-tolerance</pre><pre>reb_dates = totidx.asfreq(‘q’).index</pre><pre>weights = dict()<br>weights[‘aggressive’]= pd.DataFrame( data={‘equity’:.8,’bond’:.2}, index=reb_dates, columns=totidx.columns ).fillna(0)<br>weights[‘moderate’]= pd.DataFrame( data={‘equity’:.6,’bond’:.4}, index=reb_dates, columns=totidx.columns ).fillna(0)<br>weights[‘conservative’]= pd.DataFrame( data={‘equity’:.4,’bond’:.6}, index=reb_dates, columns=totidx.columns ).fillna(0)</pre><pre># plot average weights<br>pd.concat({k:v.mean() for k,v in weights.items()},axis=1)\<br> .loc[[‘equity’,’bond’]]\<br> .plot.pie(autopct=’%1.1f%%’,subplots=True,legend=False)</pre><pre>plt.suptitle(‘Average Weights with Different Risk Tolerances’)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*zEdCc5B4NV6WbvIIeavEWQ.png" /></figure><p>Let’s see how they perform with a simple backtesting algorithm.</p><pre># simple backtest function<br>def backtest(w,prices, v0=1000,t=0,start_date=None):<br> <br> p = prices[w.columns]<br> v = pd.DataFrame(data = v0*np.ones(len(p)),index=p.index)<br> <br> for i in w.index:<br> # i = rebalance date ; t = trade offset, d = effective date<br> d = i + timedelta(t) <br> w_i = np.array(w.loc[:i].tail(1)).flatten()<br> v_i = np.array(v.loc[:d].tail(1)).flatten()<br> p_i = p.loc[d:]<br> <br> v.update( (p_i/p_i.iloc[0]).dot(v_i * w_i) ) <br> <br> if start_date is not None:<br> v = v[start_date:]<br> v = v/v.iloc[0]*v0<br> <br> return(v)</pre><pre><br># run backtest<br>test1 = pd.DataFrame.from_dict({k:backtest(v,totidx)[0] for k,v in weights.items()})</pre><pre>test1.plot()<br>plt.title(‘Backtest Performance of Fixed-Weighted Portfolios’,{‘fontsize’:16})<br>plt.show()</pre><pre># performance review<br>mret = test1.asfreq(‘m’,method=’ffill’).pct_change().dropna()<br>pd.DataFrame.from_dict({<br> ‘annualized monthly return’:mret.mean()*12*100,<br> ‘annualized monthly volatility’:mret.std()*np.sqrt(12)*100,<br> ‘annualized monthly sharpe’:mret.mean() / mret.std() * np.sqrt(12),<br>}).T.round(2)</pre><p>So we have an annualized return of 8.5% with 10% volatility for our aggressive portfolio composed of 80% equity and 20% bond. The result is pretty good for history including the 2008 Financial Crisis, the 2019 Trade War Saga and most-recently the CoronaVirus Outbreak Fear. If you calculate CAGR with the consideration of compounding, the return could be even better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*f8laj8aZgNRFOweKdlUdRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/1*MCehndIRetamgTkLdzwgFQ.png" /></figure><h3>The More Active Asset Rotation Portfolio</h3><p>Now we need to derive some valuation scores for our assets. The simplest of such models is <strong><em>price momentum</em></strong>. There are different ways to evaluate the attractiveness of assets, sectors or individual stocks. I will cover more topics about it in the future in categories such as <strong><em>smart beta</em></strong> or <strong><em>factor investing</em></strong>.</p><pre># create momentum ranks</pre><pre>mtridx = totidx.asfreq(‘m’,method=’ffill’)[[‘equity’,’dm’,’em’,’bond’]]</pre><pre>mranks=dict()<br>mranks[‘3m’]= mtridx.pct_change(3).dropna()\<br>                    .rank(axis=1,ascending=False)<br>mranks[‘6m’]= mtridx.pct_change(6).dropna()\<br>                    .rank(axis=1,ascending=False)<br>mranks[‘9m’]= mtridx.pct_change(9).dropna()\<br>                    .rank(axis=1,ascending=False)<br>mranks[‘12m’]= mtridx.pct_change(12).dropna()\<br>                    .rank(axis=1,ascending=False)</pre><p>The next step is to convert factor scores into weights. You can do equal weights for simplicity, but here I provided sample codes that assign the top-ranked security to 40% of the total portfolio, the second to 30%, the third to 20% and the fourth to the remaining 10%.</p><pre># convert ranks to weights (1=0.4, 2=0.3, 3=0.2 , 4=0.1)<br>s = {1:0.4,2:0.3,3:0.2,4:0.1}</pre><pre>def rankWeights(m,s):<br> w = pd.DataFrame(data = 0,index=m.index,columns=m.columns)<br> for k,v in s.items(): w[m==k]=v<br> w[‘cash’]=1-w.sum(axis=1).round(2)<br> return w</pre><pre># add weights to list<br>for k,v in mranks.items():<br> weights[k]=rankWeights(v,s).asfreq(‘q’,method=’ffill’).dropna()</pre><p>The average rotation weights are similar across assets and momentum models, from 20% to 29%. This suggests that their performance should also be close to the fixed-weighted aggressive (80/20) and moderate (60/40) portfolios. But will they?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*t8EiUIrE4sf2nED_yOSXnA.png" /></figure><p>The answer is NO. Rotation portfolios’ performances are much worst than the moderate portfolio, with less return and higher volatilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*adTdFxrJI9gS1bgykyHbXg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*DzNuy8kDJeYEc_dQP-1jZQ.png" /></figure><h3>Mean-Variance Optimized Portfolio</h3><p>In the last backtest, we look at implementing the <strong>Mean-Variance (Markovitz) Optimization</strong> for portfolio construction.</p><p>Here’s the function to calculate the efficient frontier:</p><pre>def eff_frontier(mu, sigma, clip=(0,1), dr=0.0025, symbols=list(mu.index)):<br> <br> pfRet = lambda x: x.dot(mu)<br> pfVar = lambda x: x.dot(sigma).dot(x) <br> <br> w = w0 = np.ones(len(mu))/len(mu)<br> mus = np.arange(start=mu.min(), stop=mu.max()+dr, step=dr, dtype=’float’)<br> results = pd.DataFrame(index=mus, columns=[‘return’,’risk’,’sharpe’] + symbols)<br> <br> for r in mus:<br> w_constraints = (<br> {‘type’: ‘eq’, ‘fun’:lambda x:sum(x)-1.},<br> {‘type’: ‘eq’, ‘fun’:lambda x:pfRet(x)-r},<br> ) <br> w_opt = minimize(pfVar, w, method=’SLSQP’, constraints=w_constraints, bounds = [clip]*len(w) )<br> w = w_opt.x<br> results.loc[r]=[ pfRet(w), np.sqrt(pfVar(w)), pfRet(w) / np.sqrt(pfVar(w)) ] + list(w)<br> <br> return results.reset_index(drop=True)</pre><p>I also wrote a wrapper function to calculate return and covariance matrix, solve for the efficient frontier, find the optimized weights for the best Sharpe ratio, minimized volatility given expected return, and maximized return given expected risk.</p><pre>def optimization(prices,clip=(0,1),exp_ret=0.07,exp_vol=0.15,freq=’m’,lookback=12,yearbase=12):<br> <br> if lookback is not None:<br> mret = prices.asfreq(freq,method=’ffill’).pct_change().tail(lookback).dropna()<br> else:<br> mret = prices.asfreq(freq,method=’ffill’).pct_change().dropna()<br> <br> mu = mret.mean()*yearbase<br> sigma = mret.cov()*yearbase<br> results = eff_frontier(mu,sigma,clip)<br> <br> return {<br> ‘eff_frontier’: results,<br> ‘best_sharpe’: results.sort_values(by=’sharpe’,ascending=False).head(1),<br> ‘min_vol’: results.sort_values(by=’risk’,ascending=True).loc[results[‘return’]&gt;=exp_ret].head(1),<br> ‘max_ret’: results.sort_values(by=’return’,ascending=False).loc[results[‘risk’]&lt;=exp_vol].head(1),<br> }</pre><p>Here are the plots for efficient frontier and weights:</p><pre>opt_weights = optimization(mtridx,exp_ret=0.02,exp_vol=0.2,lookback=None)</pre><pre>results = opt_weights[‘eff_frontier’]<br>pf1 = opt_weights[‘best_sharpe’]<br>pf2 = opt_weights[‘min_vol’]<br>pf3 = opt_weights[‘max_ret’]</pre><pre>plt.plot(results[‘risk’],results[‘return’],’ — ‘,label=’eff frontier’)<br>plt.scatter(pf1[‘risk’],pf1[‘return’],label=’best sharpe’,s=250,marker=’*’)<br>plt.scatter(pf2[‘risk’],pf2[‘return’],label=’min vol’,s=250,marker=’*’)<br>plt.scatter(pf3[‘risk’],pf3[‘return’],label=’max ret’,s=250,marker=’*’)<br>plt.title(‘Efficient Frontier’,{‘fontsize’:16})<br>plt.xlabel(‘Expected Return’)<br>plt.ylabel(‘Expected Risk’)<br>plt.legend()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*QDgn32TVbZgXH7fHP8ZVjw.png" /></figure><pre>results.set_index(‘return’)[mtridx.columns].plot.area()</pre><pre>plt.axvline(x=pf1[‘return’].values,c=’black’,ls=’ — ‘)<br>plt.text(x=pf1[‘return’].values-0.001,y=0.6,s=’best sharpe’,rotation=’vertical’,fontsize=12)</pre><pre>plt.axvline(x=pf2[‘return’].values,c=’black’,ls=’ — ‘)<br>plt.text(x=pf2[‘return’].values-0.001,y=0.6,s=’min vol’,rotation=’vertical’,fontsize=12)</pre><pre>plt.axvline(x=pf3[‘return’].values,c=’black’,ls=’ — ‘)<br>plt.text(x=pf3[‘return’].values-0.001,y=0.6,s=’max ret’,rotation=’vertical’,fontsize=12)</pre><pre>plt.title(‘Efficient Frontier Weights’)<br>plt.legend(bbox_to_anchor=[1.1,0.5],loc=’right’ )<br>plt.xlim(results[‘return’].min(),results[‘return’].max())<br>plt.ylim(0,1)</pre><pre>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*eEPmKdstkoWcIj8E3tlHdg.png" /></figure><p>Now we are ready to run the optimization backtests. Here’re the results of three portfolios with constraints of 5% minimal expected return, 10% maximal expected risk and 50% maximum single asset weight.</p><pre>symbols=[‘equity’,’bond’,’dm’,’em’]</pre><pre>reb_dates = totidx.asfreq(‘q’).index<br>w1=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)<br>w2=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)<br>w3=pd.DataFrame(data = 0,index=reb_dates,columns=symbols)</pre><pre>for d in reb_dates:<br> p = totidx.loc[:d,symbols]<br> if len(p)&gt;=250:<br> opt_w = optimization(p, clip=(0,0.5), exp_ret=0.05, exp_vol=0.10, lookback=36)<br> w1.loc[d] = opt_w[‘best_sharpe’][symbols].values.flatten().tolist()<br> w2.loc[d] = opt_w[‘min_vol’][symbols].values.flatten().tolist() if len(opt_w[‘min_vol’])&gt;0 else 0<br> w3.loc[d] = opt_w[‘max_ret’][symbols].values.flatten().tolist() if len(opt_w[‘max_ret’])&gt;0 else 0</pre><pre>w1[‘cash’]=1-w1.sum(axis=1).round(2) <br>w2[‘cash’]=1-w2.sum(axis=1).round(2) <br>w3[‘cash’]=1-w3.sum(axis=1).round(2)</pre><pre>weights[‘best_sharpe’]=w1<br>weights[‘min_vol’]=w2<br>weights[‘max_ret’]=w3</pre><p>The optimized backtests with return and risk estimates derived from the ex-posted return history couldn’t prove that they are superior to the simple fixed-weighted portfolios. But I actually like optimization, because it’s quite challenging to do right. I’ll see if I can post some more sophisticated examples in the future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*lna_iXT3WImhbwxopDuJkA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*9zEa_BLHrbdjNRxAYrL3Fg.png" /></figure><h3>Conclusion</h3><p>I have demonstrated how to use python (without relying on any 3rd party API) to quickly test quantitative strategies.</p><p>Within the three tested portfolio construction techniques: fixed-weights, asset-rotation, and optimization, fixed-weights seem to be the winner for their simplicity, robustness and superior risk-return trade-offs. But, of course, there are lots of rooms for the other two to improve (which I hope I can address individually in the future).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*H-k2aSPL2iaQKR11B_NsBg.png" /></figure><p><strong>My last words</strong></p><p>Thanks for your patience to finish reading my first Medium blog. I look forward to sharing with you more interesting quantitative finance and data science topics in the future.</p><p>Best regards,</p><p>Richard</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc66442d5d52" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>