<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Facundo Joel Allia Fernandez on Medium]]></title>
        <description><![CDATA[Stories by Facundo Joel Allia Fernandez on Medium]]></description>
        <link>https://medium.com/@facujallia?source=rss-b77229e22a72------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*ZqDC6lYidnmnuoBM</url>
            <title>Stories by Facundo Joel Allia Fernandez on Medium</title>
            <link>https://medium.com/@facujallia?source=rss-b77229e22a72------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 18:31:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@facujallia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Designing a Fixed Income Carry Trade Screener with Python and Central Bank APIs]]></title>
            <link>https://medium.com/@facujallia/designing-a-fixed-income-carry-trade-screener-with-python-and-central-bank-apis-406da2340e71?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/406da2340e71</guid>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[argentina]]></category>
            <category><![CDATA[macroeconomics]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Sun, 01 Mar 2026 20:47:53 GMT</pubDate>
            <atom:updated>2026-03-01T20:47:53.479Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How a quantitative framework, a custom BCRA API, and Claude Code turned a Jupyter notebook into a production investment tool</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jLWY0u3AIaeenFzv.jpg" /></figure><p>Between January 2024 and late 2025, systematic carry trade strategies in Argentine peso instruments delivered annualized returns of <strong>15% to 30% in USD terms</strong> for investors who sized positions correctly and managed maturity timing around key macro events.</p><p>That number sounds implausible for a country associated with recurring currency crises. The mechanism is not exotic: when nominal peso rates significantly exceed actual peso depreciation over a holding period, the dollar-adjusted return on peso instruments is positive. The interesting development under the current Argentine administration is that the depreciation side of that equation became, for the first time in years, <em>modelable in advance</em>. The introduction of an explicit exchange rate band created a quantifiable worst-case FX scenario for any future date — and that constraint is what makes this a screener problem rather than a speculation problem.</p><p>This article documents the quantitative framework, the data infrastructure built to support it, and the production tool that came out of it.</p><h3>The macro context that created the opportunity</h3><p>Javier Milei took office in December 2023 with a fiscal deficit above 5% of GDP and a central bank with negative net international reserves. The policy response was sharp: a primary fiscal surplus was achieved within the first quarter — the first sustained surplus in over a decade.</p><p>After a one-time 54% devaluation at the start of the administration to partially unify the official and parallel rates, the BCRA implemented a crawling peg at a pre-announced monthly rate. In April 2025, the framework evolved into a band-based floating regime — the institutional innovation that makes quantitative carry analysis tractable.</p><h3>The two components of carry: rates and FX</h3><p>For zero-coupon sovereign instruments such as treasury bills, the return in pesos is fixed at purchase.</p><p>You buy the instrument at price <strong>X</strong> and receive a payoff <strong>P</strong> at maturity. The return in pesos is simply:</p><p><strong>Return (ARS) = (P / X) — 1</strong></p><p>There is no reinvestment risk and no path dependency. The local currency return is fully determined at entry.</p><p>The only uncertainty comes from the exchange rate.</p><p>If the exchange rate at entry is <strong>M0</strong>, and the exchange rate at maturity is <strong>MT</strong>, the realized return in USD becomes:</p><p><strong>Return (USD) = (P / X) × (M0 / MT) — 1</strong></p><p>This equation captures the entire carry trade. One component is deterministic. The other is stochastic but, under a band regime, bounded. That structure is what makes scenario analysis tractable — and what separates this from unstructured currency speculation.</p><h3>Exchange rate regimes create modelable constraints</h3><p>Since April 2025, Argentina’s exchange rate operates within a managed float with explicit bands.</p><p>The ceiling evolves according to deterministic rules:</p><p><strong>Phase 1 (April–December 2025):</strong> Fixed 1% monthly depreciation, compounded daily.</p><pre>Phase 1 (Apr–Dec 2025): ceiling = 1,400 × (1.01)^(days/30)          </pre><p><strong>Phase 2 (January 2026 onward):</strong> Monthly depreciation linked to <strong>inflation two months prior</strong> (T-2 rule) — aligning the nominal depreciation allowance with realized disinflation.</p><pre>Phase 2 (Jan 2026+):<br>ceiling_today = ceiling_yesterday × (1 + inflation_T-2)^(1/days_in_month)</pre><p>From a quantitative standpoint, constraints transform uncertainty into modelable scenarios. The ceiling becomes a worst-case FX assumption: if the peso stays within the band through a position’s maturity, the maximum depreciation is known today.</p><p>📊 <strong>Exchange rate vs. projected band ceiling</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m8h9AqtStdVNDtQTwRNcNQ.png" /></figure><p>FX rate (historical) plotted alongside the projected ceiling derived from the band model. This defines the upper bound of modeled FX scenarios.</p><h3>The instruments: deterministic local returns</h3><p>The carry trade uses peso-denominated sovereign instruments — <strong>LECAPs </strong>(Letras Capitalizables) and <strong>BONCAPs </strong>(Bonos Capitalizables) — which trade and settle directly in pesos through registered brokers.</p><p>The FX conversion is an investor-level decision. A dollar-based investor sells USD at the FX rate to fund the entry, and buys USD again at the FX rate prevailing at maturity. This distinction matters for modeling: the peso return is fully deterministic at purchase, and the USD return is a function exclusively of USD/ARS at entry versus USD/ARS at exit.</p><p>These instruments behave as zero-coupon bonds: purchased at a discount, redeemed at a fixed payoff at maturity. Their payoff is fixed at issuance — only the investor’s FX round-trip determines the USD return. This allows direct comparison across the entire maturity curve.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B1JmTUJGTRJhNQ0AtSNpIA.png" /></figure><h3>Building the scenario matrix</h3><p>The core output is a matrix: rows are instruments ordered by maturity, columns are exchange rate scenarios at maturity, each cell is the realized USD return. Scenarios above the band ceiling are flagged as structurally invalid — they imply the BCRA abandoned the band, which changes the investment thesis entirely.</p><p>📊 <strong>Carry return heatmap across FX scenarios</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*fEn0RYnuWeXf2X_degewEg.png" /></figure><p>This transforms carry evaluation from intuition into a measurable surface. Longer-duration instruments accumulate more peso yield over time, which translates into positive carry across a wider range of FX scenarios — including the most adverse ones. Short-duration instruments, by contrast, have less time for the peso rate to compound, making them sensitive to even moderate depreciation. The risk-return tradeoff is immediately visible in the matrix: duration is your friend in a carry trade, as long as the regime holds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*oGFjokN_Sh5dmKwwcak2ng.png" /></figure><h3>The investment simulator</h3><p>Beyond the matrix, the tool includes a position-level simulator: you input a dollar amount and select an instrument, and the app returns the projected outcome in USD across every exchange rate scenario.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*Ly5Q7-MMuSwKHNOprCQuKg.png" /></figure><p>This is the component most used in practice by clients: rather than reading the matrix abstractly, you enter the actual position size and see the dollar outcome under each scenario. It makes the risk concrete — not “this instrument returns -3% in the $1,700 scenario” but “if you invest $10,000 and FX (MEP) reaches $1,700, you recover $9,700.”</p><p><strong>Want to try it with live data?</strong> The tool is publicly available — no login, no signup, prices and projections update automatically:</p><p>👉 <a href="https://logos-serviciosfinancieros.com.ar/recursos/analisis-carry-trade">logos-serviciosfinancieros.com.ar/recursos/analisis-carry-trade</a></p><blockquote>If you want to build something similar — for Argentina or any other market with a managed exchange rate regime — the rest of this article covers how it works under the hood: where the data comes from, how the band model is implemented, and what it takes to go from a notebook to a production system. The architecture is replicable and the core infrastructure is open source.</blockquote><h3>Under the hood: data, models, and infrastructure</h3><p>Two inputs power the model in real time:</p><p><strong>Bond prices and exchange rate</strong> — sourced from <a href="https://data912.com/">data912.com</a>, a public API that provides real-time prices for Argentine sovereign instruments as well as the exchange rate needed to run the calculations. The original code comparing bond prices against their cashflows was developed by <a href="https://twitter.com/JohnGalt_is_www">JohnGalt_is_www</a>, whose work was the foundation this project built on.</p><p>This was enough to run the model under the original crawling peg regime. But when the BCRA shifted to the band-based float in April 2025, a new variable entered the equation: the Phase 2 band ceiling evolves according to future inflation, which means you need <em>forward inflation estimates</em> to project where the ceiling will be at each instrument’s maturity date.</p><p>Market prices are available via API. Inflation expectations, as it turns out, are not.</p><p>The BCRA publishes the <strong>REM (Relevamiento de Expectativas de Mercado)</strong> — a monthly survey that consolidates macroeconomic forecasts from approximately 45 economists and research firms. It covers inflation, exchange rate, interest rates, GDP, fiscal balance, and more. It is the most authoritative consensus estimate available for Argentine macro variables, and the closest thing the market has to a shared forward curve for inflation.</p><p>The problem: the REM is distributed exclusively as an Excel file posted on the BCRA’s website. No API, no structured access, no programmatic interface. Every month, a new file. That was the gap that motivated building the REM API from scratch.</p><h3>The REM BCRA API</h3><p>The <a href="https://github.com/facundoallia/rem-bcra-api">rem-bcra-api</a> is an open-source pipeline that converts the BCRA’s monthly Excel into a public REST API with global edge delivery — no authentication, no cost.</p><pre>BCRA Excel<br>    ↓<br>GitHub Actions<br>    ↓<br>Python + Pandas<br>    ↓<br>Cloudflare R2<br>    ↓<br>Cloudflare Workers<br>    ↓<br>REST API</pre><p>GitHub Actions downloads and processes the file automatically between the 1st and 15th of each month. Cloudflare Workers serves the data globally with a 1-hour cache.</p><p>You can try the API right now — no authentication, no setup:</p><pre>import requests<br><br>data = requests.get(<br>    &quot;https://bcra-rem-api.facujallia.workers.dev/api/ipc_general&quot;<br>).json()<br><br># {&quot;periodo&quot;: &quot;2026-03&quot;, &quot;mediana&quot;: 2.8, &quot;percentil_25&quot;: 2.5, &quot;percentil_75&quot;: 3.1}</pre><p>Or directly in your browser: <a href="https://bcra-rem-api.facujallia.workers.dev/api/ipc_general">bcra-rem-api.facujallia.workers.dev/api/ipc_general</a></p><p>Beyond inflation, the API covers the full REM: exchange rate projections, interest rates, GDP, exports, fiscal balance, unemployment. Every endpoint has a _top10 variant with individual forecasts from the best-performing economists in the survey — useful for dispersion analysis when consensus is unusually wide.</p><p>The carry analyzer consumes this with an adaptive cache strategy: 12-hour TTL during days 1–10 of the month (the REM update window), and month-end TTL when data is stable. Inflation data priority inside the model:</p><pre>1. Published INDEC data      (Nov 2025: 2.5%, Dec 2025: 2.8%, Jan 2026: 2.9%)<br>2. REM monthly median        (~45 economists consensus)<br>3. Implied from REM annual   (prorated from annual target if monthly missing)<br>4. Conservative fallback     (2.0% monthly)</pre><h3>Building with Claude Code</h3><p>Both projects — the REM API and the carry analyzer — were built with Claude Code as a core part of the workflow.</p><p>The value is not code generation in isolation. It is the ability to hold financial logic and systems design in the same context simultaneously. Designing the Phase 2 band model and verifying the T-2 indexation logic, debugging the REM Excel parser across file versions with irregular structure that changes between publications, iterating on the API response schema, building the FX three-level fallback chain — all of that requires switching rapidly between financial reasoning and engineering decisions. Claude Code compresses that feedback loop substantially.</p><p>This is what AI-assisted engineering actually looks like in practice — not replacing judgment, but amplifying the output of a single person with domain expertise across two fields.</p><h3>Why this framework matters beyond Argentina</h3><p>Carry trades are often discussed qualitatively. The underlying mechanics are fully quantifiable.</p><p>The interaction between deterministic yield and constrained exchange rate evolution creates a system that can be modeled, simulated, and monitored. The same architecture applies directly to: Brazil’s mini-auction exchange rate system, Turkey’s rate corridor, any emerging market managed float where a known constraint makes FX evolution bounded.</p><p>The REM API pipeline — GitHub Actions + Python + Cloudflare — can be pointed at any central bank that publishes consensus survey data in Excel format. The carry framework is market-agnostic once the data inputs are in place.</p><p>Both repositories are open source. The API requires no authentication. Once the pipeline exists, analyzing new instruments as they come to market is trivial. The system updates itself.</p><p><em>Not investment advice. Sovereign credit risk, exchange rate regime risk, and liquidity risk apply to all instruments discussed.</em></p><p><strong>Links</strong></p><ul><li>Carry Trade Analyzer — <a href="https://github.com/facundoallia/carry-trade-analyzer">github.com/facundoallia/carry-trade-analyzer</a></li><li>REM BCRA API — <a href="https://github.com/facundoallia/rem-bcra-api">github.com/facundoallia/rem-bcra-api</a></li><li>Live tool —<a href="https://logos-serviciosfinancieros.com.ar/recursos/analisis-carry-trade"> logos-serviciosfinancieros.com.ar/</a></li><li>API — <a href="https://bcra-rem-api.facujallia.workers.dev/">https://bcra-rem-api.facujallia.workers.dev</a></li></ul><p><em>Follow me on Medium for more on quantitative finance, AI-assisted engineering, and fixed income in emerging markets.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=406da2340e71" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Effective Trading Screener with Python and Tradingview]]></title>
            <link>https://medium.com/@facujallia/building-an-effective-trading-screener-with-python-and-tradingview-9329db9afaa9?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/9329db9afaa9</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[emerging-markets]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[investing]]></category>
            <category><![CDATA[trading]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Mon, 26 Jun 2023 23:22:24 GMT</pubDate>
            <atom:updated>2023-06-26T23:22:24.426Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/612/0*AbvpRn0gEMkD2dBI.jpg" /></figure><p>Trading screeners are indispensable tools for traders and investors seeking opportunities in the vast financial markets. By applying specific criteria and indicators to filter through a multitude of instruments, we can uncover potential trades with higher probabilities of success.</p><p>To accomplish this, we will leverage the `<strong>tradingview_ta</strong>` library, a powerful Python library that provides access to a wide range of updated and popular indicators used by traders worldwide. This ensures that we stay ahead of the game by incorporating the latest market insights into our screening process. We’ll also utilize the `<strong>yfinance</strong>` library to download historical quotes for our selected instruments effortlessly. This gives us a solid historical data foundation for comprehensive analysis and testing.</p><p>This article will focus on a practical application using <strong>Argentine ADRs</strong> (American Depositary Receipts). However, it’s important to note that the principles and techniques we explore can be applied to a wide range of instruments available on Tradingview. Whether you’re interested in stocks, futures, ETFs, cryptocurrencies, or any other tradable asset, this screener can adapt to your needs.</p><p>In the first part, we create a graph where we can visually see how many technical indicators give purchase or sale for each of the analyzed actions. Then, in the second section, we download the detail of each of the technical indicators to filter and search for trading opportunities in specific assets.</p><p><strong>1. Import libraries</strong></p><p>We will import essential libraries such as pandas, numpy, tradingview_ta, yfinance, mplfinance, datetime and matplotlib. These libraries will enable us to handle data, perform screeners, visualize results, and create plots.</p><pre># import the libraries that we going to use for the analysis<br>import pandas as pd<br>import numpy as np<br>from tradingview_ta import TA_Handler, Interval, Exchange<br>import yfinance as yf<br>import mplfinance as mpf<br>from datetime import datetime, timedelta<br>import matplotlib.pyplot as plt</pre><p><strong>2. Define stocks list</strong></p><p>In the code snippet provided, we have a list of ticker symbols representing various Argentine ADRs (American Depositary Receipts). ADRs are stocks of foreign companies that trade on U.S. exchanges, allowing investors to gain exposure to international markets without the need to directly purchase shares on foreign exchanges.</p><p>By including these ticker symbols in our code, we are essentially creating a watchlist or a set of instruments that we want to analyze using our trading screener. These tickers will serve as the basis for retrieving historical price data, applying indicators, and ultimately screening for potential trading opportunities.</p><pre>tickers = [ &#39;YPF&#39;,<br>            &#39;BBAR&#39;,<br>            &#39;GGAL&#39;,<br>            &#39;MELI&#39;,<br>            &#39;SUPV&#39;,<br>            &#39;BMA&#39;,<br>            &#39;DESP&#39;,<br>            &#39;LOMA&#39;,<br>            &#39;PAM&#39;,<br>            &#39;CEPU&#39;,<br>            &#39;TGS&#39;,<br>            &#39;TEO&#39;,<br>            &#39;BIOX&#39;,<br>            &#39;CRESY&#39;,<br>            &#39;IRS&#39;,<br>            &#39;EDN&#39;<br>        ]</pre><p><strong>3. Retrieve data</strong></p><p>In this step, we are retrieving data for each ticker symbol in the tickers list. We iterate through the list and perform the following actions for each ticker:</p><p>We create an instance of the TA_Handler class from the tradingview_ta library. This handler allows us to interact with TradingView’s technical analysis functionalities.</p><p>Also, we specify the parameters for the <strong>TA_Handler</strong> instance:</p><ul><li><strong>symbol</strong>: The ticker symbol for which we want to retrieve data.</li><li><strong>screener</strong>: The region or exchange from which we want to fetch data. In this case, we set it to “america” to focus on American stocks.</li><li><strong>exchange</strong>: The specific stock exchange where the ticker is listed. We initially search for data on the NYSE (New York Stock Exchange).</li><li><strong>interval</strong>: The time interval of the data we want to retrieve. Here, we set it to “1d” for daily data.</li></ul><p>We use the <em>get_analysis().summary</em> method to retrieve a summary of the technical analysis for the specified ticker. This summary contains various indicators and metrics calculated based on historical price data. Then, we append the retrieved data for the ticker to the tickers_data list.</p><p>If an exception occurs during the retrieval process, indicating that no data is found for the ticker on the NYSE, we handle the exception and search for the data on the NASDAQ (NASDAQ Stock Market), which is another popular U.S. stock exchange.</p><p>Finally, we print a message indicating that the data has been successfully imported.</p><p>By executing this code, we gather technical analysis data for each ticker from the specified stock exchanges. This data will later be used for applying indicators, screening for trading opportunities, and generating insights in our trading screener.</p><pre>tickers_data = []<br><br># Iterate through each ticker<br>for ticker in tickers:<br>    try:<br>        # Retrieve data for the ticker from NYSE<br>        data = TA_Handler(<br>            symbol=ticker,<br>            screener=&quot;america&quot;,<br>            exchange=&quot;NYSE&quot;,<br>            interval=&quot;1d&quot;<br>        )<br>        data = data.get_analysis().summary<br>        tickers_data.append(data)<br>        <br>    except Exception as e:<br>        # If no data is found for the ticker in NYSE, search in NASDAQ<br>        print(f&quot;No data found for ticker {ticker} in NYSE. Searching in NASDAQ...&quot;)<br>        data = TA_Handler(<br>            symbol=ticker,<br>            screener=&quot;america&quot;,<br>            exchange=&quot;NASDAQ&quot;,<br>            interval=&quot;1d&quot;<br>        )<br>        data = data.get_analysis().summary<br>        tickers_data.append(data)<br><br>print(&quot;Data successfully imported.&quot;)</pre><p>After retrieving the data for all the tickers, we proceed to process and analyze it to extract the relevant information in order to create a plot with buy, sell and neutral indicators.</p><p><strong>4. Process data</strong></p><p>We iterate through each element in the tickers_data list, which contains the analyzed data for each ticker. We extract the specific information we need, such as the recommendation, buy, sell, and neutral values, and store them in separate lists.</p><p>We then create a dictionary called data to organize the extracted information, associating it with the corresponding ticker symbol. This dictionary is used to create a Pandas DataFrame called df, where each column represents a specific piece of information: Ticker, Recommendations, Buys, Sells, and Neutrals.</p><pre>recommendations = []<br>buys = []<br>sells = []<br>neutrals = []<br><br># Iterate through each data in tickers_data<br>for data in tickers_data:<br>    recommendation = data.get(&#39;RECOMMENDATION&#39;)<br>    buy = data.get(&#39;BUY&#39;)<br>    sell = data.get(&#39;SELL&#39;)<br>    neutral = data.get(&#39;NEUTRAL&#39;)<br>    <br>    recommendations.append(recommendation)<br>    buys.append(buy)<br>    sells.append(sell)<br>    neutrals.append(neutral)<br><br>data = {<br>    &#39;Ticker&#39;: tickers,<br>    &#39;Recommendations&#39;: recommendations,<br>    &#39;Buys&#39;: buys,<br>    &#39;Sells&#39;: sells,<br>    &#39;Neutrals&#39;: neutrals<br>}<br><br>df = pd.DataFrame(data)</pre><p>To establish an order for the recommendations, we define a dictionary called order_categories, which assigns numerical values to each category. The Order column is created in the DataFrame by mapping the values from the Recommendations column to the corresponding categories in the order_categories dictionary. The DataFrame is then sorted based on the Order column in ascending order, ensuring a logical order of the recommendations.</p><p>Finally, the Order column is dropped from the DataFrame since it is no longer needed, and the sorted DataFrame is displayed, presenting the information in a tabular format.</p><p>By processing and analyzing the data in this manner, we can gain valuable insights into the recommendations and categorizations for each ticker, aiding us in making informed trading decisions.</p><pre># Define the order of categories<br>order_categories = {<br>    &#39;STRONG_BUY&#39;: 5,<br>    &#39;BUY&#39;: 4,<br>    &#39;NEUTRAL&#39;: 3,<br>    &#39;SELL&#39;: 2,<br>    &#39;STRONG_SELL&#39;: 1<br>}<br><br># Assign a numerical value to each category in a new column &quot;Order&quot;<br>df[&#39;Order&#39;] = df[&#39;Recommendations&#39;].map(order_categories)<br>df = df.sort_values(&#39;Order&#39;, ascending=True).reset_index(drop=True)<br><br># Drop the &quot;Order&quot; column if not needed in the final output<br>df = df.drop(&#39;Order&#39;, axis=1)<br><br># Display the sorted dataframe<br>df</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*H4vttFKg4TgHhVjF8twcTQ.png" /></figure><p><strong>5. Plot the data</strong></p><p>We can visually represent the recommendations through a horizontal bar plot using Matplotlib. This allows us to display the buy, sell, and neutral signals for each ticker in a clear and intuitive manner.</p><p>In the code snippet, we start by defining the colors for the bars, background, and text elements to create an appealing visual style. Then, we create the figure and axes objects, specifying the size of the plot. The background color of the plot is set to enhance the overall presentation.</p><p>Annotations are included to specify the recommendation for each ticker. These annotations appear adjacent to the corresponding bar, providing additional context. The horizontal grid lines are removed to declutter the plot and focus attention on the essential elements.</p><pre># Create the figure and axes<br>fig, ax = plt.subplots(figsize=(10, 6))<br>fig.set_facecolor(&#39;#111111&#39;)<br>ax.set_facecolor(&#39;#111111&#39;)<br><br># Add the horizontal bars for buys, neutrals, and sells<br>ax.barh(df.index, df[&quot;Buys&quot;], height=0.25, color=&#39;#00BFFF&#39;, label=&#39;Indicators signaling buy&#39;)<br>ax.barh(df.index, df[&quot;Neutrals&quot;], height=0.25, color=&#39;#808080&#39;, label=&#39;Neutral indicators&#39;, left=df[&quot;Buys&quot;])<br>ax.barh(df.index, df[&quot;Sells&quot;], height=0.25, color=&#39;#FFA500&#39;, label=&#39;Indicators signaling sell&#39;, left=df[&quot;Buys&quot;] + df[&quot;Neutrals&quot;])<br><br># Set the axes labels<br>ax.set_yticks(df.index)<br>ax.set_yticklabels(df[&quot;Ticker&quot;], fontsize=8, color=&#39;white&#39;)<br>ax.set_xlabel(&#39;Number of indicators&#39;, fontsize=10, color=&#39;white&#39;)<br><br># Add title with larger font size and additional spacing<br>ax.set_title(&#39;Indicators with buy, sell, and neutral recommendations by ticker&#39;, fontsize=14, color=&#39;white&#39;, pad=20)<br><br># Add the annotations<br>for i, recommendation in enumerate(df[&quot;Recommendations&quot;]):<br>    ax.annotate(recommendation, xy=(25, i), xytext=(26.5, i),<br>                color=&#39;white&#39;, fontsize=8, va=&#39;center&#39;, ha=&#39;left&#39;)<br><br># Remove the horizontal grid lines<br>ax.yaxis.grid(False)<br><br># Configure the tick and axis styles<br>ax.tick_params(axis=&#39;x&#39;, colors=&#39;white&#39;)<br>ax.tick_params(axis=&#39;y&#39;, colors=&#39;white&#39;)<br>ax.spines[&#39;left&#39;].set_color(&#39;white&#39;)<br>ax.spines[&#39;bottom&#39;].set_color(&#39;white&#39;)<br>ax.spines[&#39;left&#39;].set_linewidth(0.5)<br>ax.spines[&#39;bottom&#39;].set_linewidth(0.5)<br>ax.spines[&#39;right&#39;].set_visible(False)<br>ax.spines[&#39;top&#39;].set_visible(False)<br><br># Adjust the spacing<br>plt.tight_layout()<br><br># Set the text color of the legend to white<br>legend = ax.legend(loc=&#39;lower center&#39;, bbox_to_anchor=(0.5, -0.175), ncol=3, fontsize=8)<br>for text in legend.get_texts():<br>    text.set_color(&#39;#111111&#39;)<br><br># Display the plot<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*BMKQhh6jk4Apzcf9YYXfyA.png" /></figure><h3><strong>Section 2: Unveiling Trading Opportunities through Indicator Filtering</strong></h3><p>While the collective signals offer a broad perspective, our approach goes beyond surface-level analysis. In the second section, we leverage the power of technical indicators by diving deeper into their individual details. We download and analyze the specific data points for each indicator, enabling us to apply advanced filtering techniques.</p><p>In the first step, we involve importing technical indicator data for a list of stock tickers from either the NYSE or NASDAQ exchange and storing it in a pandas DataFrame called df_adrs_indicators. The code iterates over each ticker, attempts to retrieve the indicator data from the NYSE, and if it fails, searches for the data in the NASDAQ. The retrieved data is then concatenated with the existing DataFrame. Finally, the column names are converted to strings, and a success message is printed. This process allows for the consolidation of technical indicator data for multiple stocks, enabling further analysis and identification of trading opportunities.</p><pre>df_adrs_indicators = pd.DataFrame()<br><br>for ticker in tickers:<br>    try:<br>        data = TA_Handler(<br>            symbol=ticker,<br>            screener=&quot;america&quot;,<br>            exchange=&quot;NYSE&quot;,<br>            interval=&quot;1d&quot;<br>        )<br>        data = data.get_analysis().indicators<br><br>    except Exception as e:<br>        print(f&quot;No data found for ticker {ticker} in NYSE. Searching in NASDAQ...&quot;)<br>        data = TA_Handler(<br>            symbol=ticker,<br>            screener=&quot;america&quot;,<br>            exchange=&quot;NASDAQ&quot;,<br>            interval=&quot;1d&quot;<br>        )<br>        data = data.get_analysis().indicators<br>        <br>    df_temp = pd.DataFrame(data, index=[ticker])<br>    df_adrs_indicators = pd.concat([df_adrs_indicators, df_temp])<br><br>df_adrs_indicators.columns = df_adrs_indicators.columns.astype(str)<br>print(&quot;Data successfully imported.&quot;)</pre><p>As a result we obtain the dataframe df_adrs_indicators. It has 91 columns with information on many technical indicators, extremely useful information to filter the assets according to the criteria that we consider relevant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*Q8L-XV3t8AYpU96zoJ2tnA.png" /></figure><p>For example, once we have imported the technical indicator data into the df_adrs_indicators DataFrame, we can proceed to filter the stocks based on specific criteria. In this code snippet, we apply multiple filters to identify stocks that meet certain conditions.</p><p>First, we filter stocks that have their Exponential Moving Average (EMA) 10-day value crossing above their EMA 20-day value. This is done by creating a new DataFrame called df_filtered using the condition df_adrs_indicators[‘EMA10’] &gt; df_adrs_indicators[‘EMA20’].</p><p>Next, we further narrow down the selection by filtering for stocks with a Relative Strength Index (RSI) value above 75. The df_filtered DataFrame is updated with this condition: df_filtered[df_filtered[‘RSI’] &gt; 75.</p><p>Subsequently, we apply a filter for stocks with a positive Moving Average Convergence Divergence (MACD), specifically when the MACD line crosses above the signal line. The df_filtered DataFrame is again updated with the condition: df_filtered[df_filtered[‘MACD.macd’] &gt; df_filtered[‘MACD.signal’].</p><p>Lastly, we filter for stocks with an Average Directional Index (ADX) above 30, indicating a strong trend. The df_filtered DataFrame is updated with the final condition: df_filtered[df_filtered[‘ADX’] &gt; 30.</p><pre># Filter stocks with EMA crossing SMA<br>df_filtered = df_adrs_indicators[df_adrs_indicators[&#39;EMA10&#39;] &gt; df_adrs_indicators[&#39;EMA20&#39;]]<br><br># Filter stocks with RSI above 70 (overbought)<br>df_filtered = df_filtered[df_filtered[&#39;RSI&#39;] &gt; 75]<br><br># Filter stocks with positive MACD (signal line crossover)<br>df_filtered = df_filtered[df_filtered[&#39;MACD.macd&#39;] &gt; df_filtered[&#39;MACD.signal&#39;]]<br><br># Filter stocks with ADX above a certain threshold (indicating a strong trend)<br>df_filtered = df_filtered[df_filtered[&#39;ADX&#39;] &gt; 30]<br><br># Print the stocks that meet the criteria<br>filtered_shares = df_filtered.index.to_list()<br><br>print(&#39;Shares filtered:&#39;, filtered_shares)</pre><blockquote>Shares filtered: [‘YPF’, ‘BMA’, ‘CRESY’, ‘EDN’]</blockquote><p>We can also observe the price movements of the filtered shares using <strong>mplfinance</strong>, a Python library that provides a high-level interface for creating financial charts and visualizations. It is built on top of the popular Matplotlib library and offers a convenient way to generate candlestick charts, line plots, bar plots, and other financial plot types.</p><pre># Get the current date<br>end_date = datetime.now().strftime(&quot;%Y-%m-%d&quot;)<br><br># Calculate the start date (3 months ago from today)<br>start_date = (datetime.now() - timedelta(days=3 * 30)).strftime(&quot;%Y-%m-%d&quot;)<br><br># Calculate the number of rows and columns based on the desired layout<br>num_rows = (len(filtered_shares) + 1) // 2<br>num_cols = 2<br><br># Create a figure with subplots<br>fig, axes = plt.subplots(nrows=num_rows, ncols=num_cols, figsize=(12, 8))<br>fig.suptitle(&quot;Daily Candlestick Charts&quot;, fontsize=14)<br><br># Iterate over each stock and add a subplot for each one<br>for i, symbol in enumerate(filtered_shares):<br>    # Calculate the row and column index for the current subplot<br>    row = i // num_cols<br>    col = i % num_cols<br><br>    # Get historical data from Yahoo Finance<br>    data = yf.download(symbol, start=start_date, end=end_date)<br><br>    # Plot the candlestick chart in the subplot<br>    mpf.plot(data, type=&#39;candle&#39;, ax=axes[row, col], volume=False, show_nontrading=False)<br><br>    # Set the title of the subplot<br>    axes[row, col].set_title(symbol)<br><br># Adjust the spacing between subplots<br>plt.tight_layout()<br><br># Show the plot<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqkPA_KOZI4i3iTf2MJj6A.png" /></figure><p>You can see the complete code in my <a href="https://github.com/facundoallia">GitHub profile</a></p><p>Also, I invite you to follow me on my<a href="https://medium.com/@facujallia"> Medium profile</a> to be aware of the articles I publish.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9329db9afaa9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unleashing the Power of Time Series Forecasting in Calendar Spread Strategies: An Application in…]]></title>
            <link>https://medium.com/@facujallia/unleashing-the-power-of-time-series-forecasting-in-calendar-spread-strategies-an-application-in-2cc568933f3a?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cc568933f3a</guid>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[derivatives]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Mon, 22 May 2023 03:14:50 GMT</pubDate>
            <atom:updated>2023-05-22T03:14:50.392Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Unleashing the Power of Time Series Forecasting in Calendar Spread Strategies: An Application in the Argentine Corn Market</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S9y9Rmvs8Z6m2B7KFnJ1CA.png" /></figure><p>In the fast-paced world of financial markets, traders are constantly seeking innovative strategies to gain a competitive edge. One such strategy that has gained popularity is calendar spread trading. <strong>Calendar spreads </strong>involve simultaneously buying and selling options or futures contracts with different expiration dates but the same underlying asset. These spreads can offer unique profit opportunities based on the relative pricing dynamics of the contracts over time.</p><p>To navigate the complexities of calendar spread trading and identify profitable opportunities, traders have turned to the power of<strong> time series forecasting</strong>. By analyzing historical market data and applying advanced forecasting techniques, traders can gain valuable insights into future price movements and make more informed trading decisions.</p><p>In this article, we will delve into the realm of time series forecasting applied specifically to calendar spread strategies using the<strong> futures prices of corn in the Argentine market</strong>. We will focus on the futures contracts traded in <a href="https://www.matbarofex.com.ar/">Matba Rofex</a>, where corn futures play a significant role in agricultural commodity trading.</p><p><strong>The code can be modified to implement in other agricultural commodities</strong> such as soybeans, wheat, etc. The seasonality exhibited by agricultural futures can be useful for forecasting calendar spreads but also the code can be adjusted to apply to other non-agricultural commodities, including crude oil, gold, natural gas, etc. Even can compare two different products to analyze a pair trading strategy.</p><p>Join us on this journey as we unlock the potential of time series forecasting and explore the fascinating world of calendar spread strategies. Let’s dive in and discover how this powerful combination can take your trading game to new heights. The article is structured into different sections:</p><p><strong><em>About the Data</em></strong></p><p><strong><em>1. Library Setup</em></strong></p><p><strong><em>2. Data Processing for Product Analysis</em></strong></p><p><strong><em>3. Data merge and spread calculation</em></strong></p><p><strong><em>4. Filter the data</em></strong></p><p><strong><em>5. Forecast future values using Meta Prophet</em></strong></p><p><strong><em>6. Trend and seasonality analysis</em></strong></p><p><strong><em>7. Conclusions</em></strong></p><h3>About the Data</h3><p>The data used for this analysis originates from the <a href="http://datacenter.matba.com.ar/futuros.aspx">Data Center</a> of Matba Rofex, the primary derivatives market in Argentina. The Data Center provides a comprehensive repository of historical market data, including futures contracts traded on various commodities. Each dataframe within our dataset corresponds to a specific futures contract, representing the trading activity over a particular time period. In our analysis, we focus on the daily close prices of corn futures.</p><h3><strong>1. Library Setup</strong></h3><p>We will import essential libraries such as pandas, numpy, calendar, Prophet, matplotlib, and plotly.graph_objects. These libraries will enable us to handle data, perform forecasting, visualize results, and create interactive plots.</p><pre># Import the libraries that we going to use for the analysis<br>import pandas as pd<br>import numpy as np <br>import calendar<br>from prophet import Prophet<br>import matplotlib.pyplot as plt<br>import plotly.graph_objects as go</pre><p>By utilizing <strong>Meta Prophet</strong>, a state-of-the-art forecasting library, we can harness the power of time series analysis to unlock the potential of calendar spread trading in the Argentine corn futures market.<a href="https://facebook.github.io/prophet/"> Meta Prophet</a>, based on the popular Prophet library, is a cutting-edge tool that simplifies the forecasting process and provides accurate predictions for time series data. Its flexibility and ease of use make it a valuable asset for data scientists and traders alike.</p><h3><strong>2. Data Processing for Product Analysis</strong></h3><p>In order to process the data, we present the code snippet for the process_product_data function, which is responsible for processing product data related to a specific product, month, and range of years. The function takes parameters such as product (the name of the product), month (the specific month of interest), beg_year (the starting year), end_year (the ending year), and last_date (an optional parameter for filtering data after a specified date).</p><p>The function first generates a list of years within the specified range. It then iterates over each year and reads the corresponding CSV file containing the product data. The relevant columns (‘Fecha’ for date and ‘Cierre’ for closing price) are selected and renamed appropriately. The closing price values are standardized by replacing commas with periods, and the date column is converted to a datetime format. If the last_date parameter is provided and the current year is not the first year in the range, the function filters the data to include only records with dates after the last_date.</p><p>The processed dataframes for each year are stored in a list, and the last date from the last dataframe is recorded for potential future use. Finally, the function concatenates all the dataframes into a single dataframe and returns it.</p><pre>def process_product_data(product, month, beg_year, end_year, last_date=None):<br><br>    import pandas as pd<br>    <br>    # Initialize an empty list to store dataframes<br>    dataframes = []<br><br>    # Create a list of years from beg_year to end_year<br>    years = list(range(beg_year, end_year + 1, 1))<br><br>    # Loop through each year<br>    for year in years:<br>        # Generate file path based on product, month, and year<br>        file_path = f&#39;Data/{product}/{month}/{year}.csv&#39;<br>        <br>        # Read the CSV file and select the desired columns<br>        df = pd.read_csv(file_path, encoding=&#39;latin1&#39;, usecols=[&#39;Fecha&#39;, &#39;Cierre&#39;])<br>        <br>        # Rename the columns for consistency<br>        df = df[[&#39;Fecha&#39;, &#39;Cierre&#39;]].rename(columns={&#39;Fecha&#39;: &#39;date&#39;, &#39;Cierre&#39;: &#39;close&#39;})<br>        <br>        # Add a column to store the expiration year<br>        df[&#39;expiration&#39;] = year<br>        <br>        # Replace commas with periods in the &#39;close&#39; column<br>        df[&#39;close&#39;] = df[&#39;close&#39;].str.replace(&#39;,&#39;, &#39;.&#39;).astype(float)<br>        <br>        # Convert the &#39;date&#39; column to datetime format<br>        df[&#39;date&#39;] = pd.to_datetime(df[&#39;date&#39;], format=&#39;%d/%m/%Y&#39;)<br><br>        # Filter rows based on the last_date for subsequent years<br>        if last_date and year &gt; years[0]:<br>            df = df[df[&#39;date&#39;] &gt; last_date]<br><br>        # Append the dataframe to the list<br>        dataframes.append(df)<br><br>        # Check if the DataFrame has at least one record<br>        if not df.empty:<br>            last_date = df[&#39;date&#39;].iloc[-1]<br><br>    # Concatenate the dataframes in the list into a single dataframe<br>    return pd.concat(dataframes)</pre><p>We perform data processing for two sets of corn futures contracts. The first set, denoted as ‘<em>prod_1</em>’, focuses on corn futures with a contract month of April (month 4) and covers the period from 2018 to 2024. The second set, ‘<em>prod_2</em>&#39;, analyzes corn futures with a contract month of December (month 12) and encompasses the time frame from 2018 to 2023.</p><p><em>This part of the code can be modified in order to analyze other products, months, or years. In the folder ‘Data’ of the </em><a href="https://github.com/facundoallia/calendar-spread-time-series-forecasting"><em>GitHub version</em></a><em>, you can find the historical prices for corn, soy, and wheat from 2010 to the date (21–5–2023). Feel free to replace the parameters to analyze calendar spreads with different combinations of products and months.</em><strong><em> Remember to refresh the latest contracts.</em></strong></p><pre># Define the parameters for product 1<br>prod_1 = &#39;corn&#39;<br>month_1 = 4<br>beg_1 = 2018<br>end_1 = 2024<br><br># Define the parameters for product 2<br>prod_2 = &#39;corn&#39;<br>month_2 = 12<br>beg_2 = 2018<br>end_2 = 2023</pre><p>By calling the <em>process_product_data</em> function for each set, we process the respective corn futures data, extracting the daily close prices and organizing them into dataframes. These dataframes, <em>df_1</em> and <em>df_2</em>, now contain the processed historical price data for the specified corn futures contracts.</p><pre># Process data for prod_1<br>df_1 = process_product_data(prod_1, month_1, beg_1, end_1)<br><br># Process data for prod_2<br>df_2 = process_product_data(prod_2, month_2, beg_2, end_2)</pre><p>We can plot the dataframes to visualize the historical prices of each contract by defining the function <em>plot_futures_data</em>. The function separates the contracts according to their expiration year and graphs them in a different color to be able to distinguish them. We use the Plotly library to achieve interactive graphics.</p><pre>def plot_futures_data(df, month):<br>    <br>    # Create a list of unique expiration values<br>    expirations = df[&#39;expiration&#39;].unique()<br><br>    # Create a trace for each expiration value and add it to a list<br>    traces = []<br>    for expiration in expirations:<br>        df_plot = df[df[&#39;expiration&#39;] == expiration]<br>        trace = go.Scatter(x=df_plot[&#39;date&#39;], y=df_plot[&#39;close&#39;], mode=&#39;lines&#39;, name=str(expiration))<br>        traces.append(trace)<br><br>    # Create layout for the plot<br>    layout = go.Layout(<br>        title=&#39;Evolution of historical prices: &#39; + month,<br>        xaxis=dict(title=&#39;Date&#39;),<br>        yaxis=dict(title=&#39;Close&#39;),<br>        template=&#39;plotly_dark&#39;,  # Apply a dark theme<br>        legend=dict(orientation=&#39;h&#39;, yanchor=&#39;bottom&#39;, y=-0.3),  # Adjust legend position<br>    )<br><br>    # Create figure object and add line traces and layout<br>    fig = go.Figure(data=traces, layout=layout)<br><br>    # Display the plot<br>    fig.show()<br><br>plot_futures_data(df_1, &#39;April&#39;)<br>plot_futures_data(df_2, &#39;December&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QWLlMZ8H5t5O4ixQd7gu6w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eUf8a6PS2n8fRezb3ezs1w.png" /></figure><h3><strong>3. Data merge and spread calculation</strong></h3><p>In order to merge the dataframes, we create column names for the two sets based on the product and month. <em>column_1</em> is defined as the concatenation of the product name (<em>prod_1</em>) and the abbreviated month name, while c<em>olumn_2</em> is similarly defined for the second set. Next, <strong>we merge the two dataframes</strong>, <em>df_1</em> and <em>df_2</em>, on the <em>date</em> column using the <em>pd.merge()</em> function. The merge operation combines the dataframes based on matching dates, resulting in a single dataframe, <em>df</em>, with both sets of corn futures data. The suffixes, defined as <em>column_1</em> and <em>column_2</em>, are added to the column names to differentiate between the two sets.</p><p>To calculate the spread between two sets of corn futures contracts, we subtract the <em>close</em> values of <em>column_2</em> from <em>column_1</em>. The calculated <strong>spread</strong> is stored in a new column named <em>spread</em> within the dataframe <em>df</em>. The <em>pd.to_numeric()</em> function is used to ensure that the close values are treated as numeric data for accurate subtraction, with any non-numeric values coerced to NaN.</p><p>By executing this code, we obtain a merged dataframe<strong> </strong><em>df</em> that<strong> contains the historical price data for both sets of corn futures contracts and the calculated spread</strong>. This allows us to further analyze and develop trading strategies based on the price differentials between the two contract months.</p><pre># Create column names based on product and month<br>column_1 = f&#39;_{prod_1}_{calendar.month_name[month_1][:3].lower()}&#39;<br>column_2 = f&#39;_{prod_2}_{calendar.month_name[month_2][:3].lower()}&#39;<br><br># Merge the two dataframes on &#39;date&#39; column with appropriate suffixes<br>df = pd.merge(df_1, df_2, on=&#39;date&#39;, suffixes=(column_1, column_2))<br><br># Calculate the spread by subtracting the &#39;close&#39; values of column_2 from column_1<br>df[&#39;spread&#39;] = pd.to_numeric(df[&#39;close&#39;+str(column_1)], errors=&#39;coerce&#39;) - pd.to_numeric(df[&#39;close&#39;+str(column_2)], errors=&#39;coerce&#39;)</pre><h3><strong>4. Filter the data</strong></h3><p>When comparing prices of contracts from different months, there are two key scenarios to consider:</p><p>1. <strong>Comparing contracts within the same year</strong>: For instance, we may compare contracts like July 2023 and April 2023. In such cases, our analysis focuses solely on capturing price differences between these specific months within a single year.</p><p>2. <strong>Comparing contracts across different years</strong>: Let’s take the example of comparing April 2024 and December 2023 contracts. This scenario involves comparing months from different years. To accurately analyze a calendar spread strategy with such variations, it is essential to filter out spread values associated with contracts from the same year.</p><p>In this case, analyzing a calendar spread strategy that involves months of a different year (April 2024 vs December 2023) <strong>we must filter the observed spread values whose contracts are from the same year</strong>. By applying this filtering approach, we ensure that our analysis examines the precise price differentials between the desired contract months.</p><pre># Create an empty list to store spread names<br>spread_names = []<br><br># Iterate over each index in the dataframe<br>for index in df.index:<br>    # Construct the spread name using various components<br>    spread_name = str(prod_1) + &#39;_&#39; + calendar.month_name[month_1].lower() + &#39;_&#39; + str(df.iloc[index,2]) + &#39;_vs_&#39; + str(prod_2) + &#39;_&#39; + calendar.month_name[month_2].lower() + &#39;_&#39; + str(df.iloc[index,4])<br>    # Append the spread name to the list<br>    spread_names.append(spread_name)<br><br># Assign the spread names to the &#39;spread_name&#39; column in the dataframe<br>df[&#39;spread_name&#39;] = spread_names<br><br># Create a mask to compare the expiration values of column_1 and column_2<br>mask = df[&#39;expiration&#39;+str(column_1)] == df[&#39;expiration&#39;+str(column_2)]<br># In case of analyzing contracts within the same year, replace the line of code:<br># mask = df[&#39;expiration&#39;+str(column_1)] != df[&#39;expiration&#39;+str(column_2)]<br><br># Drop rows from the dataframe where the expiration values match<br>df = df.drop(df[mask].index)</pre><p>We can plot the dataframe to visualize the historical spreads of each combination of contracts using Plotly.</p><pre># Create a list of unique spread_name values<br>spread_names = df[&#39;spread_name&#39;].unique()<br><br># Create a trace for each spread_name value and add it to a list<br>traces = []<br>for name in spread_names:<br>    df_plot = df[df[&#39;spread_name&#39;] == name]<br>    trace = go.Scatter(x=df_plot[&#39;date&#39;], y=df_plot[&#39;spread&#39;], mode=&#39;lines&#39;, name=str(name))<br>    traces.append(trace)<br><br># Create layout for the plot<br>layout = go.Layout(<br>    title=&#39;Calendar Spread Prices&#39;,<br>    xaxis=dict(title=&#39;Date&#39;),<br>    yaxis=dict(title=&#39;Close&#39;),<br>    template=&#39;plotly_dark&#39;,  # Apply a dark theme<br>    legend=dict(orientation=&#39;h&#39;, yanchor=&#39;bottom&#39;, y=-0.5),  # Adjust legend position<br>)<br><br># Create figure object and add line traces and layout<br>fig = go.Figure(data=traces, layout=layout)<br><br># Display the plot<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T8U1rwTrrhJ7-okoX5wdsw.png" /></figure><p>As we can see in the previous graph,<strong> we can readily discern a distinct pattern in the spread values</strong> during the years 2018, 2021, and 2022. The pattern reveals an upward trend from April to November, followed by a subsequent decline towards December.</p><p>In contrast, the years 2019 and 2020 exhibit a contrasting behavior in the spreads. Notably, during October and November, there is a noticeable decrease in the spread values, setting these years apart from the aforementioned pattern.</p><p>These observations shed light on the dynamic nature of the spread values across different years, providing valuable insights for our analysis of calendar spread strategy. Now that we know more about the behavior of the spread in past years, let’s use this information to forecast the value of the spread in the coming months to analyze a possible calendar spread strategy.</p><h3><strong>5. Forecast future values using Meta Prophet</strong></h3><p>Prophet, a widely adopted forecasting tool, is highly suitable for analyzing the calendar spread strategy of corn futures. This powerful procedure utilizes an additive model to forecast time series data. It excels in capturing non-linear trends by incorporating yearly, weekly, and daily seasonality, alongside holiday effects.</p><p>Prophet shines when applied to time series data with prominent seasonal patterns and a substantial historical dataset spanning multiple seasons. Its strength lies in its ability to handle missing data gracefully and adapt to shifts in the trend. Furthermore, Prophet exhibits robust performance in handling outliers, ensuring accurate and reliable forecasts.</p><p>The input to Prophet is always a dataframe with two columns: <strong>ds</strong> and<strong> y</strong>. The ds (datestamp) column should be of a format expected by Pandas, ideally YYYY-MM-DD for a date. The y column must be numeric and represents the measurement we wish to forecast. We start from the analyzed dataframe to transform it to the format required by Prophet.</p><pre># Create an empty DataFrame df_obs<br>df_obs = pd.DataFrame()<br><br># Assign the &#39;date&#39; column from df to &#39;ds&#39; column in df_obs<br>df_obs[&#39;ds&#39;] = df[&#39;date&#39;]<br><br># Assign the &#39;spread&#39; column from df to &#39;y&#39; column in df_obs, converting it to float<br>df_obs[&#39;y&#39;] = df[&#39;spread&#39;].astype(float)</pre><p>We<strong> initialize the Prophet model</strong> by creating an instance, denoted as <strong>m</strong>. Once the model is instantiated, we proceed to <strong>fit it to our dataset</strong>, <em>df_obs</em>. Fitting the model involves training it on historical data, enabling the model to learn the underlying patterns and dynamics of the corn futures market. This step is crucial for the Prophet model to generate accurate forecasts based on the observed data.</p><pre># Create a Prophet instance<br>m = Prophet()<br><br># Fit the Prophet model to the observed data<br>m.fit(df_obs)</pre><p>Then, <strong>we generate a future dataframe </strong>to facilitate forecasting using the previously fitted Prophet model. By creating this future dataframe, we extend the time horizon beyond the observed data, allowing us to make predictions for an additional 200 periods into the future.</p><pre># Create a future dataframe with 200 additional periods<br>future = m.make_future_dataframe(periods=200)</pre><p>Finally, we utilize the previously fitted Prophet model to <strong>generate forecasts for the future timeframe using the future dataframe</strong>. By invoking the <em>predict()</em> function from the Prophet model, <strong>we generate a forecast based on the future dataframe</strong>. This function applies the trained model to the future data, leveraging the learned patterns and seasonality to project the expected values of the target variable. In this case, we are interested in forecasting the price of corn futures for the next <strong>200 days</strong>.</p><p>The resulting forecast object contains various columns, including ‘ds’ (representing the dates), <em>yhat</em> (the predicted values), <em>yhat_lower</em> (the lower bound of the prediction), and <em>yhat_upper</em> (the upper bound of the prediction).</p><pre># Generate the forecast using the trained model<br>forecast = m.predict(future)</pre><p>By utilizing the <em>plot_plotly()</em> function, we can represent the predicted values in an interactive and engaging manner</p><pre>plot_plotly(m, forecast)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*FkOu_lzDmr2pNYv6e5zc3w.png" /></figure><p>To enhance the visualization of the forecasts, we can apply a simple yet effective technique known as <strong>smoothing</strong>. By implementing a <strong>moving average</strong> with a window size of 7 periods, we can achieve a smoother representation of the forecasted values. The moving average calculates the average of a specific number of consecutive periods, in this case, 7 periods. By taking the average, we can reduce the impact of short-term fluctuations and highlight the underlying trends in the data.</p><pre>smoothed_forecast = pd.DataFrame()<br><br>smoothed_forecast[&#39;ds&#39;] = forecast[&#39;ds&#39;]<br><br># Smooth the &#39;yhat&#39;, &#39;yhat_lower&#39;, and &#39;yhat_upper&#39; columns using a rolling mean with window size 7<br>smoothed_forecast[&#39;yhat&#39;] = forecast[&#39;yhat&#39;].rolling(window=7, min_periods=1, center=True).mean()<br>smoothed_forecast[&#39;yhat_lower&#39;] = forecast[&#39;yhat_lower&#39;].rolling(window=7, min_periods=1, center=True).mean()<br>smoothed_forecast[&#39;yhat_upper&#39;] = forecast[&#39;yhat_upper&#39;].rolling(window=7, min_periods=1, center=True).mean()<br><br>plot_plotly(m, smoothed_forecast)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*EIqensiI5CRF5oTzF6p3ZQ.png" /></figure><p>Upon visual analysis of the forecast graph generated with Prophet, we observe a<strong> discernible pattern in the predicted values</strong>. Looking ahead for the next few months, there is a noticeable upward trend until November, followed by a subsequent decline towards December. <strong>This pattern aligns closely with the historical trends observed in previous years, suggesting a consistent behavior in the calendar spread strategy for corn futures</strong>. By recognizing and understanding these patterns, we can gain valuable insights to inform our trading decisions and optimize our strategies accordingly.</p><h3><strong>6. Trend and seasonality analysis</strong></h3><p>Additionally, <strong>by invoking the <em>plot_components_plotly()</em> function, we can explore the distinct components of the forecast, including trends, seasonality, and any holiday effects</strong>. This visual representation, allows us to examine each component separately and gain insights into their contributions to the overall forecasted values.</p><p>Analyzing the trend component helps us identify the long-term direction of the forecast while examining the seasonality component uncovers any recurring patterns or periodic fluctuations. Additionally, understanding the impact of holiday effects enables us to account for any anomalies or deviations that occur during specific time periods.</p><pre>plot_components_plotly(m, forecast)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*6wdXkEJNn5l-HfU6bBiO-w.png" /></figure><p>When examining the <strong>long-term trend</strong>, we can identify interesting patterns in the data. Between the years 2018 and 2019, we observe a gradual increase in the trend component, transitioning from -5 to 0. However, since 2019 there is a significant downward movement until mid-2021, with values plummeting to -17. Since then, the trend has been on a steady rise, currently hovering around zero. The forecasted values for the upcoming months continue this upward trajectory, reaching a peak of 6.5 in December.</p><p>As for the <strong>monthly seasonal component</strong>, we should note that the observations begin in April of each year. At the start of the year, there are some noticeable fluctuations; however, these should be disregarded due to the limited data available. In April, the monthly seasonal component is around 3.5. It then dips to -8 in May, recovers its previous levels, experiences a slight decline towards early December, and undergoes a sharp increase in the final month.</p><p>Regarding the <strong>weekly component</strong>, no discernible pattern emerges. It’s worth noting that trading of the asset does not take place on Saturdays and Sundays, making these days irrelevant in our analysis.</p><h3><strong>7. Conclusions</strong></h3><p>Based on the generated forecasts,<strong> it appears favorable to consider a long position for the strategy</strong>. This involves purchasing an April 2024 contract and simultaneously selling a December 2023 contract, aiming to capitalize on potential increases in the price differential between the two.</p><p>However, it’s essential to emphasize that<strong> the analysis conducted here should not be construed as an investment recommendation</strong>. It’s crucial to acknowledge that the forecast solely relies on historical observations and does not encompass other factors that may impact the spread’s price. Nevertheless, the analysis provides a foundation for exploring a potential calendar spread strategy.</p><p>Remember, it’s prudent to conduct further research, consider additional market factors, and assess your individual risk tolerance and investment objectives before making any trading decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cc568933f3a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stock classification using k-means clustering]]></title>
            <link>https://medium.com/@facujallia/stock-classification-using-k-means-clustering-8441f75363de?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/8441f75363de</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[k-means-clustering]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[stock-market]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Fri, 28 Apr 2023 03:52:28 GMT</pubDate>
            <atom:updated>2023-04-28T03:52:28.351Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M6APxIGLM-b9HpUzoWPVvw.png" /></figure><p><em>I am writing this article to share the study that I carried out last year in the final postgraduate project in quantitative finance “Classification of shares through k-means clustering”. To see the full paper visit </em><a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4403662"><em>SSRN</em></a><em>.</em></p><p>This work presents an approach to using the K-means algorithm for stock classification, with the aim of helping investors to diversify their investment portfolios. It´s divided into 4 parts:</p><ol><li>Introducing the k-means algorithm</li><li>Clustering of stocks by return and volatility</li><li>Clustering of shares by price-earnings ratio and dividend rate</li><li>3-dimensional analysis using the k-means++ algorithm by return, volatility, and PER.</li></ol><h3>Introducing the k-means algorithm</h3><p>The term k-means was first used by MacQueen in 1967, although the idea dates back to Steinhaus in 1957. K-means is an unsupervised classification (clustering) algorithm that groups objects into k groups based on their characteristics.</p><p>Clustering is done by minimizing the sum of distances between each object and the centroid of its group or cluster. Quadratic distance is often used. The algorithm consists of three steps:</p><ol><li>Initialization: once the number of groups, k, has been chosen, k centroids are established in the data space, for example, choosing them randomly.</li><li>Assign objects to centroids: each data object is assigned to its nearest centroid.</li><li>Centroid update: the position of the centroid of each group is updated, taking as the new centroid the position of the average of the objects belonging to said group.</li></ol><p>Steps 2 and 3 are repeated until the centroids do not move, or move below a threshold distance at each step.</p><h3>Clustering of stocks by return and volatility</h3><p>We analyze the S&amp;P 500 index to cluster stocks based on return and volatility. This index comprises 500 large-cap US companies from various sectors, traded on NYSE or Nasdaq. Due to its representation of the US’s largest publicly traded firms, it serves as a suitable dataset for algorithmic k-means clustering.</p><pre>#Import the libraries that we are going to need to carry out the analysis:<br>import numpy as np <br>import pandas as pd<br>import pandas_datareader as dr<br>import yfinance as yf<br><br>from pylab import plot,show<br>from matplotlib import pyplot as plt<br>import plotly.express as px<br><br>from numpy.random import rand<br>from scipy.cluster.vq import kmeans,vq<br>from math import sqrt<br>from sklearn.cluster import KMeans <br>from sklearn import preprocessing</pre><h4>Load Data</h4><p>We calculate the annual average return and volatility for each company by obtaining their adjusted closing prices during 01/02/2020–12/02/2022 and inserting them into a dataframe, which is then annualized (assuming 252 market days per year).</p><pre># Define the url<br>sp500_url = &#39;https://en.wikipedia.org/wiki/List_of_S%26P_500_companies&#39;<br><br># Read in the url and scrape ticker data<br>data_table = pd.read_html(sp500_url)<br>tickers = data_table[0][&#39;Symbol&#39;].values.tolist()<br>tickers = [s.replace(&#39;\n&#39;, &#39;&#39;) for s in tickers]<br>tickers = [s.replace(&#39;.&#39;, &#39;-&#39;) for s in tickers]<br>tickers = [s.replace(&#39; &#39;, &#39;&#39;) for s in tickers]<br><br># Download prices<br>prices_list = []<br>for ticker in tickers:<br>    try:<br>        prices = dr.DataReader(ticker,&#39;yahoo&#39;,&#39;01/01/2020&#39;)[&#39;Adj Close&#39;]<br>        prices = pd.DataFrame(prices)<br>        prices.columns = [ticker]<br>        prices_list.append(prices)<br>    except:<br>        pass<br>    prices_df = pd.concat(prices_list,axis=1)<br>prices_df.sort_index(inplace=True)<br><br># Create an empity dataframe<br>returns = pd.DataFrame()<br><br># Define the column Returns<br>returns[&#39;Returns&#39;] = prices_df.pct_change().mean() * 252<br><br># Define the column Volatility<br>returns[&#39;Volatility&#39;] = prices_df.pct_change().std() * sqrt(252)</pre><h4>Determine the optimal number of clusters</h4><p>The Elbow curve method is a technique used to determine the optimal number of clusters for K-means clustering. The method works by plotting the sum of squared errors (SSE) for different values of k (number of clusters). The optimal number of clusters is the value of k at which the SSE starts to decrease at a slower rate. The optimal number of clusters is determined by finding the elbow or the point at which the SSE reaches its minimum value. In this case, the optimal number of clusters is 4.</p><pre># Format the data as a numpy array to feed into the K-Means algorithm<br>data = np.asarray([np.asarray(returns[&#39;Returns&#39;]),np.asarray(returns[&#39;Volatility&#39;])]).T<br>X = data<br>distorsions = []<br>for k in range(2, 20):<br>    k_means = KMeans(n_clusters=k)<br>    k_means.fit(X)<br>    distorsions.append(k_means.inertia_)<br>fig = plt.figure(figsize=(15, 5))<br><br>plt.plot(range(2, 20), distorsions)<br>plt.grid(True)<br>plt.title(&#39;Elbow curve&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xBjvuhmo4yJCWhXCRo9YAw.png" /></figure><h4>K-means clustering</h4><p>Once the optimum number of clusters has been defined, we proceed to create them. In the first instance, the centroids are defined using the sklearn library. For the creation of 4 groups of actions, the K-means algorithm iteratively assigns data points to the groups based on their similarity of characteristics, or “features”, in this case, Average Annualized Return and Average Annualized Volatility.</p><pre># Computing K-Means with K = 4 (4 clusters)<br>centroids,_ = kmeans(data,4)<br><br># Assign each sample to a cluster<br>idx,_ = vq(data,centroids)<br><br># Create a dataframe with the tickers and the clusters that&#39;s belong to<br>details = [(name,cluster) for name, cluster in zip(returns.index,idx)]<br>details_df = pd.DataFrame(details)<br><br># Rename columns<br>details_df.columns = [&#39;Ticker&#39;,&#39;Cluster&#39;]<br><br># Create another dataframe with the tickers and data from each stock<br>clusters_df = returns.reset_index()<br><br># Bring the clusters information from the dataframe &#39;details_df&#39;<br>clusters_df[&#39;Cluster&#39;] = details_df[&#39;Cluster&#39;]<br><br># Rename columns<br>clusters_df.columns = [&#39;Ticker&#39;, &#39;Returns&#39;, &#39;Volatility&#39;, &#39;Cluster&#39;]</pre><p>The algorithm initially randomly assigns the data points to the clusters and then calculates the centroid of each cluster, which is the mean of all the data points within the cluster. Then, it compares the data points to the centroid and reassigns them to groups accordingly. This process is repeated until the centroid of each cluster remains relatively stable, at which point the algorithm stops and each cluster is assigned a label. The end result is a set of 4 groups, each containing stocks that have similar returns and volatility.</p><pre># Plot the clusters created using Plotly<br>fig = px.scatter(clusters_df, x=&quot;Returns&quot;, y=&quot;Volatility&quot;, color=&quot;Cluster&quot;, hover_data=[&quot;Ticker&quot;])<br>fig.update(layout_coloraxis_showscale=False)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kdN9G0EeIyKG6JVzxOWGEg.png" /></figure><h4>Outlier treatment</h4><p>When creating the clusters, four outliers or outliers are detected in a scatter plot. Outliers are data points that are significantly different from the rest of the data points in the data set. Often, they can lead to inaccurate results when using an algorithm, since they don’t fit the same pattern as the other data points. Therefore, it is important to segregate and remove outliers to improve the accuracy of the model.</p><p>Outlier removal can help the algorithm focus on the most representative data points and reduce the effect of outliers on the results. This can help increase the accuracy of the model and ensure that the data points are grouped correctly. The removed tickers are:</p><ul><li>MRNA</li><li>ENPH</li><li>TSLA</li><li>CEG</li></ul><pre># Identify and remove the outliers stocks<br>returns.drop(&#39;MRNA&#39;,inplace=True)<br>returns.drop(&#39;ENPH&#39;,inplace=True)<br>returns.drop(&#39;TSLA&#39;,inplace=True)<br>returns.drop(&#39;CEG&#39;,inplace=True)<br><br># Recreate data to feed into the algorithm<br>data = np.asarray([np.asarray(returns[&#39;Returns&#39;]),np.asarray(returns[&#39;Volatility&#39;])]).T</pre><p>Once the outliers have been eliminated, we repeat the steps performed for clustering using the K-means algorithm to obtain more accurate clusters.</p><pre># Computing K-Means with K = 4 (4 clusters)<br>centroids,_ = kmeans(data,4)<br><br># Assign each sample to a cluster<br>idx,_ = vq(data,centroids)<br><br># Create a dataframe with the tickers and the clusters that&#39;s belong to<br>details = [(name,cluster) for name, cluster in zip(returns.index,idx)]<br>details_df = pd.DataFrame(details)<br><br># Rename columns<br>details_df.columns = [&#39;Ticker&#39;,&#39;Cluster&#39;]<br><br># Create another dataframe with the tickers and data from each stock<br>clusters_df = returns.reset_index()<br><br># Bring the clusters information from the dataframe &#39;details_df&#39;<br>clusters_df[&#39;Cluster&#39;] = details_df[&#39;Cluster&#39;]<br><br># Rename columns<br>clusters_df.columns = [&#39;Ticker&#39;, &#39;Returns&#39;, &#39;Volatility&#39;, &#39;Cluster&#39;]<br><br># Plot the clusters created using Plotly<br>fig = px.scatter(clusters_df, x=&quot;Returns&quot;, y=&quot;Volatility&quot;, color=&quot;Cluster&quot;, hover_data=[&quot;Ticker&quot;])<br>fig.update(layout_coloraxis_showscale=False)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*geCxN9tEsF1U7mTfDHLa2Q.png" /></figure><p>The graph shows 4 clusters that were generated using a K-means algorithm with 2 variables: average annualized return and average annualized volatility. These variables are used to measure the risk and return of a stock. The 4 clusters represent 4 groups of actions with different levels of risk and return in the period under study.</p><p>Clustering is useful for identifying peer groups among stocks, thus allowing differentiation between stocks with different levels of risk and return. This is useful for investors looking to diversify their investment portfolios, as it allows them to identify groups of stocks with different levels of risk and return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*EsTlNu_efSRWxTj5Pai0vg.png" /></figure><p>Investors could use the 4 clusters to select a mix of stocks with different levels of risk and return based on their investment objectives. This will help them diversify their portfolio and reduce the risk of their investment since they will be investing in a variety of assets with different levels of risk.</p><h3>Clustering of shares by price-earnings ratio and dividend rate</h3><p>According to Zhao Gao:</p><blockquote>In machine learning, the development of some algorithms is already quite mature, […] such as the k-Means algorithm. Said algorithm can be applied to investments in variable income and achieve very good results. For example, segregating two types of companies in the market. On the one hand, mature companies or “value stocks” generally have low P/E ratios and high dividend rates. The second category, “growth” companies are companies with broad development prospects, but also uncertainties in the future, generally have high P/E ratios and low dividend rates. If you can accurately distinguish between blue-chip stocks and high-growth stocks in the market, you can provide a good benchmark for investors. (Zhao Gao 2020).</blockquote><p>Following this conceptual line, it is possible to apply a clustering similar to the one carried out previously, exchanging the variables Average Annualized Return and Average Annualized Volatility for PER (Price-Earnings Ratio) and Dividend Rate (Dividend Yield). In this way, we could differentiate between “value” companies and “growth” companies.</p><h4>Load Data</h4><p>The trailing price-to-earnings (P/E) ratio is a relative valuation multiple that is based on the last 12 months of actual earnings. It is calculated by taking the current share price and dividing it by the earnings per share (EPS) for the last 12 months. While the dividend rate, or dividend rate, is the amount of cash that a company returns to its shareholders annually as a percentage of the company’s market value.</p><pre># Download trailingPE and dividendRate <br>trailingPE_list = []<br>dividendRate_list = []<br><br>for t in tickers:<br>   <br>    tick = yf.Ticker(t)<br>    ticker_info = tick.info<br><br>    try:<br>        trailingPE = ticker_info[&#39;trailingPE&#39;]<br>        trailingPE_list.append(trailingPE)<br>    except:<br>        trailingPE_list.append(&#39;na&#39;)<br><br>    try:<br>        dividendRate = ticker_info[&#39;dividendRate&#39;]<br>        dividendRate_list.append(dividendRate)<br>    except:<br>        dividendRate_list.append(&#39;na&#39;)<br><br># Create a datafrane to contain the data<br>sp_features_df = pd.DataFrame()<br><br># Add the ticker, trailingPE and dividendRate data<br>sp_features_df[&#39;Ticker&#39;] = tickers<br>sp_features_df[&#39;trailingPE&#39;] = trailingPE_list<br>sp_features_df[&#39;dividendRate&#39;] = dividendRate_list<br><br># Shares with &#39;na&#39; as dividend rate has no dividend so we have to assign 0 as dividend rate in this cases<br>sp_features_df[&quot;dividendRate&quot;] = sp_features_df[&quot;dividendRate&quot;].fillna(0)<br><br># filter shares with &#39;na&#39; as trailingPE <br>df_mask = sp_features_df[&#39;trailingPE&#39;] != &#39;na&#39; <br>sp_features_df = sp_features_df[df_mask]<br><br># Convert trailingPE numbers to float type<br>sp_features_df[&#39;trailingPE&#39;] = sp_features_df[&#39;trailingPE&#39;].astype(float)<br><br># Removes the rows that contains NULL values <br>sp_features_df=sp_features_df.dropna()</pre><h4>Determine the optimal number of clusters</h4><p>Once the Price-Earnings Ratio and Dividend Rate data have been obtained, we can reapply the Elbow method to determine the optimal number of clusters</p><pre># Format the data as a numpy array to feed into the K-Means algorithm<br>data = np.asarray([np.asarray(sp_features_df[&#39;trailingPE&#39;]),np.asarray(sp_features_df[&#39;dividendRate&#39;])]).T<br>X = data<br>distorsions = []<br>for k in range(2, 20):<br>    k_means = KMeans(n_clusters=k)<br>    k_means.fit(X)<br>    distorsions.append(k_means.inertia_)<br>fig = plt.figure(figsize=(15, 5))<br><br>plt.plot(range(2, 20), distorsions)<br>plt.grid(True)<br>plt.title(&#39;Elbow curve&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yp3ti4KYwoOv6Ra9JQ4EVw.png" /></figure><p>The optimal number of clusters is 3.</p><h4>K-means clustering</h4><p>Once the optimum number of clusters has been defined, we proceed to create them. In the first instance, the centroids are defined using the sklearn library. For the creation of groups of actions, the K-means algorithm iteratively assigns data points to the groups based on their similarity of characteristics, or “features”, in this case, Price-Earnings Ratio and Dividend Rate.</p><pre># Computing K-Means with K = 3 (3 clusters)<br>centroids,_ = kmeans(data,3)<br><br># Assign each sample to a cluster<br>idx,_ = vq(data,centroids)<br><br># Create the clusters from the numpy array &#39;data&#39;<br>cluster_1 = data[idx==0,0],data[idx==0,1]<br>cluster_2 = data[idx==1,0],data[idx==1,1]<br>cluster_3 = data[idx==2,0],data[idx==2,1]<br><br># Create a dataframe with the tickers and the clusters that&#39;s belong to<br>details = [(name,cluster) for name, cluster in zip(sp_features_df.index,idx)]<br>details_df = pd.DataFrame(details)<br><br># Rename columns<br>details_df.columns = [&#39;Ticker&#39;,&#39;Cluster&#39;]<br><br># Create another dataframe with the tickers and data from each stock<br>clusters_df = sp_features_df<br><br># Bring the clusters information from the dataframe &#39;details_df&#39;<br>clusters_df[&#39;Cluster&#39;] = details_df[&#39;Cluster&#39;]<br><br># Rename columns<br>clusters_df.columns = [&#39;Ticker&#39;, &#39;trailingPE&#39;, &#39;dividendRate&#39;, &#39;marketCap&#39;, &#39;Cluster&#39;]<br><br># Plot the clusters created using Plotly<br>fig = px.scatter(clusters_df, x=&quot;dividendRate&quot;, y=&quot;trailingPE&quot;, color=&quot;Cluster&quot;, hover_data=[&quot;Ticker&quot;])<br>fig.update(layout_coloraxis_showscale=False)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ogfj8HA6h7g3cKIHRmEp7Q.png" /></figure><p>When making a first approximation clustering by trailing price-to-earnings (P/E) and dividend rate, the presence of outliers and excessive dispersion among the observations is evident, so we proceed to filter the actions and normalize the data to eliminate these distortions.</p><h4>Outlier treatment</h4><p>We see as a result a scattered and unclear clustering. Therefore, eliminating outliers and normalizing the data will be necessary to achieve more accurate clusters.</p><p>First, we apply a filter to include only stocks with price-to-earnings less than 200 and dividend rate less than 5.</p><pre>df_mask = (sp_features_df[&#39;trailingPE&#39;] &lt; 200) &amp; (sp_features_df[&#39;dividendRate&#39;] &lt; 5)<br>sp_features_df = sp_features_df[df_mask]</pre><p>Then, we apply MaxAbsScaler. MaxAbsScaler scales each feature by its maximum absolute value. This estimator scales and translates each feature individually such that the maximal absolute value of each feature in the training set will be 1. It does not shift/center the data and thus does not destroy any sparsity.</p><pre># Import the MaxAbsScaler class <br>max_abs_scaler = preprocessing.MaxAbsScaler()<br><br># Extract the &#39;trailingPE&#39; column and reshape it to a column vector<br>trailingPE_array = np.array(sp_features_df[&#39;trailingPE&#39;].values).reshape(-1,1)<br><br># Extract the &#39;dividendRate&#39; column and reshape it to a column vector<br>dividendRate_array = np.array(sp_features_df[&#39;dividendRate&#39;].values).reshape(-1,1)<br><br># Extract the &#39;marketCap&#39; column and reshape it to a column vector<br>marketCap_array = np.array(sp_features_df[&#39;marketCap&#39;].values).reshape(-1,1)<br><br># Apply the MaxAbsScaler and store the normalized values in new columns<br>sp_features_df[&#39;trailingPE_norm&#39;] = max_abs_scaler.fit_transform(trailingPE_array)<br>sp_features_df[&#39;dividendRate_norm&#39;] = max_abs_scaler.fit_transform(dividendRate_array)<br>sp_features_df[&#39;marketCap_norm&#39;] = max_abs_scaler.fit_transform(marketCap_array)</pre><p>Once MaxAbsScaler is applied, we perform the Elbow method again with the normalized variables as input:</p><pre># Format the data as a numpy array to feed into the K-Means algorithm<br>data = np.asarray([np.asarray(sp_features_df[&#39;trailingPE_norm&#39;]),np.asarray(sp_features_df[&#39;dividendRate_norm&#39;])]).T<br>X = data<br>distorsions = []<br>for k in range(2, 20):<br>    k_means = KMeans(n_clusters=k)<br>    k_means.fit(X)<br>    distorsions.append(k_means.inertia_)<br>fig = plt.figure(figsize=(15, 5))<br><br>plt.plot(range(2, 20), distorsions)<br>plt.grid(True)<br>plt.title(&#39;Elbow curve&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EBohIzDaw-c-xXwoxOHTcQ.png" /></figure><p>Once the pertinent modifications have been made, it is possible to obtain 4 clusters generated by the K-means algorithm according to the trailing price-to-earnings (P/E) and dividend rate of each share.</p><pre># Computing K-Means with K = 4 (4 clusters)<br>centroids,_ = kmeans(data,4)<br><br># Assign each sample to a cluster<br>idx,_ = vq(data,centroids)<br><br># Create a dataframe with the tickers and the clusters that&#39;s belong to<br>details = [(name,cluster) for name, cluster in zip(sp_features_df.index,idx)]<br>details_df = pd.DataFrame(details)<br><br>clusters_df = pd.DataFrame()<br>clusters_df[&#39;Ticker&#39;] = sp_features_df[&#39;Ticker&#39;]<br>clusters_df[&#39;trailingPE_norm&#39;] = sp_features_df[&#39;trailingPE_norm&#39;]<br>clusters_df[&#39;dividendRate_norm&#39;] = sp_features_df[&#39;dividendRate_norm&#39;]<br>clusters_df[&#39;marketCap_norm&#39;] = sp_features_df[&#39;marketCap_norm&#39;]<br>clusters_df[&#39;Cluster&#39;] = details_df[1].values<br><br># Plot the clusters created using Plotly<br>fig = px.scatter(clusters_df, x=&quot;dividendRate_norm&quot;, y=&quot;trailingPE_norm&quot;, color=&quot;Cluster&quot;, hover_data=[&quot;Ticker&quot;])<br>fig.update(layout_coloraxis_showscale=False)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X5LikWRAOKYkXBP_MaM_TA.png" /></figure><p>It is possible to graphically verify that the algorithm assigned a greater weight to the dividend rate variable when creating the clusters. In this way, 4 sets of actions are distinguished: C1 with Null or very low, C2 with low, C3 with a medium-high and C4 with a high dividend rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*tjr9XGUySQ4j7KJS_gOE1Q.png" /></figure><h3>3-dimensional clustering with K-means++</h3><p>We can extend the analysis of S&amp;P500 stocks by applying k-means++ binning. This algorithm ensures a more intelligent initialization of the centroids and improves the quality of the clustering. Other than initialization, the rest of the algorithm is the same as the standard K-means algorithm. That is, K-means++ is the standard K-means algorithm together with more intelligent initialization of the centroids.</p><p>It is possible to take into account 3 variables for clustering. In K-means clustering, points are grouped into clusters based on the distance between the points. This means that to extend a clustering from two dimensions to three, you must add a third dimension to the data and calculate the distance between points in that dimension as well.</p><p>Clusters are now defined as data sets that have similar distances in all three dimensions, instead of just two. This allows a better separation of the clusters and a better fit for the distribution of the data.</p><h4>Load Data</h4><p>As in the first application, the Average Annualized Return and Average Annualized Volatility data are selected, but now the Price to Book variable is also added for the 3-dimensional analysis:</p><pre># Download priceToBook and marketCap<br>priceToBook_list = []<br>marketCap_list = []<br>tickers_clean = tickers<br><br>for t in tickers:<br><br>    try:<br><br>        tick = yf.Ticker(t)<br>        ticker_info = tick.info<br><br>        priceToBook = ticker_info[&#39;priceToBook&#39;]<br>        marketCap = ticker_info[&#39;marketCap&#39;]<br><br>        priceToBook_list.append(priceToBook)<br>        marketCap_list.append(marketCap)<br><br>    except:<br><br>        tickers_clean = tickers.remove(ticker)<br>        print(&#39;The stock ticker {} is not on database&#39;.format(ticker))<br><br># Create a datafrane to contain the data        <br>priceToBook_df = pd.DataFrame()<br><br># Add the ticker, priceToBook and marketCap data<br>priceToBook_df[&#39;Ticker&#39;] = tickers_clean<br>priceToBook_df[&#39;priceToBook&#39;] = priceToBook_list<br>priceToBook_df[&#39;marketCap&#39;] = marketCap_list<br><br># Merge dataframes<br>clusters3d_df = pd.merge(clusters_df, priceToBook_df)<br><br># Removes the rows that contains NULL values <br>clusters3d_df.dropna(inplace=True)<br><br># Drop the column with the old clusterization<br>clusters3d_df.drop([&#39;Cluster&#39;], axis=1, inplace=True)<br><br># Order columns<br>clusters3d_df = clusters3d_df[[&#39;Ticker&#39;, &#39;marketCap&#39;, &#39;Returns&#39;, &#39;Volatility&#39;, &#39;priceToBook&#39;]]</pre><h4>Determine the optimal number of clusters</h4><p>Once the Average Annualized Return, Average Annualized Volatility and Price to Book data have been obtained, we can reapply the Elbow method to determine the optimal number of clusters</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uAvbHzRA6IzCdFNpjpqdZQ.png" /></figure><p>The optimal number of clusters is 3.</p><h4>K-means clustering</h4><p>Once the optimum number of clusters has been defined, we proceed to create them. In the first instance, the centroids are defined using the sklearn library. For the creation of groups of actions, the K-means algorithm iteratively assigns data points to the groups based on their similarity of characteristics, or “features”, in this case, Average Annualized Return, Average Annualized Volatility, and Price to Book.</p><pre># Create clusters<br>k_means_optimum = KMeans(n_clusters = 3, init = &#39;k-means++&#39;,  random_state=42)<br>y = k_means_optimum.fit_predict(X)<br><br># Plot 3D graph with plotly<br>clusters3d_df[&#39;cluster&#39;] = y <br><br>fig = px.scatter_3d(clusters3d_df, x=&#39;Returns&#39;, y=&#39;Volatility&#39;, z=&#39;priceToBook&#39;,<br>              color=&#39;cluster&#39;, hover_data=[&quot;Ticker&quot;])<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I-kzF7n3jJNJ7bSH2bLuwg.png" /></figure><h4>Outlier treatment</h4><p>Again, we note the presence of outliers. We remove them individually and repeat the steps to cluster them again</p><pre># Identify and remove the outliers stocks<br>clusters3d_df.drop(clusters3d_df[(clusters3d_df[&#39;Ticker&#39;] == &#39;HD&#39;)].index, inplace=True)<br>clusters3d_df.drop(clusters3d_df[(clusters3d_df[&#39;Ticker&#39;] == &#39;CL&#39;)].index, inplace=True)<br><br># Recreate data to feed into the algorithm<br>data3d = np.asarray([np.asarray(clusters3d_df[&#39;Returns&#39;]), np.asarray(clusters3d_df[&#39;Volatility&#39;]), np.asarray(clusters3d_df[&#39;priceToBook&#39;])]).T<br>X = data3d<br><br>#elbow method<br>distorsions = []<br>for i in range(1,20):<br>    k_means = KMeans(n_clusters=i,init=&#39;k-means++&#39;, random_state=42)<br>    k_means.fit(X)<br>    distorsions.append(k_means.inertia_)<br>    <br>#plot elbow curve<br>fig = plt.figure(figsize=(15, 5))<br>plt.plot(np.arange(1,20),distorsions)<br>plt.xlabel(&#39;Clusters&#39;)<br>plt.ylabel(&#39;SSE&#39;)<br>plt.title(&#39;Elbow curve&#39;)<br>plt.grid(True)<br><br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3faIqQn4jlMVu-HTOfroag.png" /></figure><p>Once the pertinent modifications have been made, it is possible to obtain 3 clusters generated by the K-means++ algorithm according to the t Average Annualized Return, Average Annualized Volatility, and Price to Book of each share.</p><pre># Initialize KMeans model with 3 clusters<br>k_means_optimum = KMeans(n_clusters = 3, init = &#39;k-means++&#39;,  random_state=42)<br><br># Cluster data using KMeans and store labels in &#39;y&#39;<br>y = k_means_optimum.fit_predict(X)<br><br># Add &#39;cluster&#39; column to dataframe with cluster labels from &#39;y&#39;<br>clusters3d_df[&#39;cluster&#39;] = y <br><br># Create 3D scatter plot with plotly express, color-coded by cluster label and with &#39;Ticker&#39; tooltip<br>fig = px.scatter_3d(clusters3d_df, x=&#39;priceToBook&#39;, y=&#39;Returns&#39;, z=&#39;Volatility&#39;, color=&#39;cluster&#39;, hover_data=[&quot;Ticker&quot;])<br><br># Display plot<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*YJbGJ1GkAlJfSCLvPDUdoA.png" /></figure><p>Finally, through clustering by the K-means++ algorithm, 3 sets of actions grouped by the 3 variables under study (Annualized Average Return, Annualized Average Volatility, and Price to Book) are obtained. The presence of shares with high Price to Book indices is striking. This information can be useful when creating an investment portfolio.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8441f75363de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sell in May and Go Away? A study of major stock market index’s historical monthly returns]]></title>
            <link>https://medium.com/@facujallia/sell-in-may-and-go-away-a-study-of-major-stock-market-indexs-historical-monthly-returns-96cf8ed3df1?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/96cf8ed3df1</guid>
            <category><![CDATA[index]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[stock-market]]></category>
            <category><![CDATA[equity]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Mon, 24 Apr 2023 17:34:48 GMT</pubDate>
            <atom:updated>2023-04-24T17:34:48.171Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*BtJX9g7_9-AcgNeHzpsgSA.jpeg" /></figure><p>This strategy suggests that investors should sell their holdings in the stock market at the beginning of May and wait until November to reinvest, based on the idea that stock market returns are weaker during the summer months.</p><p>In this article, we’ll take a deep dive into the historical monthly returns of major stock market indexes, including the S&amp;P 500, Dow Jones Industrial Average, NASDAQ, and Russell 2000. By examining the performance of these indexes over the past several decades, we’ll seek to answer the question: is there any truth to the “Sell in May and Go Away” strategy?</p><h3>Tracing the roots of Sell in May and Go Away</h3><p>The origins of the “Sell in May and Go Away” strategy can be traced back to the old adage “Sell in May and go away, come back on St. Leger’s Day,” which refers to the traditional English horse race that takes place in mid-September. The idea behind the strategy is that investors should avoid holding stocks during the summer months when trading volumes tend to be lower, liquidity can dry up, and volatility can increase. As a result, investors may be better off exiting their positions in May and waiting until November to reinvest.</p><p>While the origins of the strategy may be rooted in folklore, there is some empirical evidence to support the idea that stock market returns tend to be weaker during the summer months. Several studies have found that stock market returns are lower during the May-October period compared to the November-April period. For example, a study by Hirsch and Lander (1998) found that from 1950 to 1995, the S&amp;P 500 index produced an average return of 7.5% during the November-April period, compared to only 1.3% during the May-October period.</p><p>However, it’s important to note that this seasonal pattern in stock market returns is not always consistent from year to year, and there are many other factors that can influence market performance, such as macroeconomic conditions, company earnings, and geopolitical events. As a result, some investors may argue that the “Sell in May and Go Away” strategy is nothing more than a myth or a self-fulfilling prophecy.</p><p>Despite the debate surrounding the strategy’s efficacy, “Sell in May and Go Away” remains a popular investing adage, and many investors continue to implement the strategy in their portfolios. In the next section of this article, we’ll take a closer look at the historical monthly returns of major stock market indexes to see if there is any empirical evidence to support this investing strategy.</p><h3>Analyzing Historical Stock Index Trends step by step</h3><p>1. import the libraries that we are going to use:</p><pre>import pandas as pd <br>import numpy as np<br>import datetime as dt<br>import matplotlib.pyplot as plt<br>import seaborn as sb</pre><p>2. Define the dataframes from the .csv files with historical quotes</p><pre>sp = pd.read_csv(&quot;spy.csv&quot;)<br>nq = pd.read_csv(&quot;nq.csv&quot;)<br>dj = pd.read_csv(&quot;dia.csv&quot;)<br>rs = pd.read_csv(&quot;rusell.csv&quot;)</pre><p>3. Add a column with the number of each month with which we will group later</p><pre>sp[&#39;month_num&#39;] = pd.DatetimeIndex(sp[&quot;Date&quot;]).month<br><br>nq[&#39;month_num&#39;] = pd.DatetimeIndex(nq[&quot;Date&quot;]).month<br><br>dj[&#39;month_num&#39;] = pd.DatetimeIndex(dj[&quot;Date&quot;]).month<br><br>rs[&#39;month_num&#39;] = pd.DatetimeIndex(rs[&quot;Date&quot;]).month</pre><p>4. Create a column with the continuous monthly performance of each index</p><p>r = ln (Pi / Pj)</p><pre>sp[&quot;S&amp;P500&quot;] = np.log(sp[&quot;Close&quot;]/(sp[&quot;Close&quot;].shift(1)))<br><br>nq[&quot;NASDAQ&quot;] = np.log(nq[&quot;Close&quot;]/(nq[&quot;Close&quot;].shift(1)))<br><br>dj[&quot;DOW_JONES&quot;] = np.log(dj[&quot;Close&quot;]/(dj[&quot;Close&quot;].shift(1)))<br><br>rs[&quot;RUSELL2000&quot;] = np.log(rs[&quot;Close&quot;]/(rs[&quot;Close&quot;].shift(1)))</pre><p>5. Obtain the monthly average grouped by month of each index</p><pre>mr_sp = pd.DataFrame(sp.groupby(&#39;month_num&#39;)[&#39;S&amp;P500&#39;].mean())<br>mr_nq = pd.DataFrame(nq.groupby(&#39;month_num&#39;)[&#39;NASDAQ&#39;].mean())<br>mr_dj = pd.DataFrame(dj.groupby(&#39;month_num&#39;)[&#39;DOW_JONES&#39;].mean())<br>mr_rs = pd.DataFrame(rs.groupby(&#39;month_num&#39;)[&#39;RUSELL2000&#39;].mean())</pre><p>6. Format and round to two decimal places</p><pre>mr_sp[&#39;S&amp;P500&#39;] = round(mr_sp[&#39;S&amp;P500&#39;]*100,2)<br>mr_nq[&#39;NASDAQ&#39;] = round(mr_nq[&#39;NASDAQ&#39;]*100,2)<br>mr_dj[&#39;DOW_JONES&#39;] = round(mr_dj[&#39;DOW_JONES&#39;]*100,2)<br>mr_rs[&#39;RUSELL2000&#39;] = round(mr_rs[&#39;RUSELL2000&#39;]*100,2)</pre><p>7. We join the data in a single dataframe and transpose it for a better visualization</p><pre>dff = mr_sp.merge(mr_nq,on=&#39;month_num&#39;).merge(mr_dj,on=&#39;month_num&#39;).merge(mr_rs,on=&#39;month_num&#39;)<br><br>dfft = dff.transpose()</pre><p>8. We graphed in seaborn heatmap</p><pre># figure<br>fig, ax = plt.subplots(figsize=(11, 9))<br># plot heatmap<br>sb.heatmap(dfft, cmap=sb.diverging_palette(10, 130, n=11), vmin= -1.2, vmax=2.6, square=True,<br>           linewidth=0.3, annot=True , cbar_kws={&quot;shrink&quot;: .8})<br># xticks<br>ax.xaxis.tick_top()<br>xticks_labels = [&#39;Jan&#39;, &#39;Feb&#39;, &#39;Mar&#39;, &#39;Apr&#39;, &#39;May&#39;, &#39;Jun&#39;,<br>                &#39;Jul&#39;, &#39;Aug&#39;, &#39;Sep&#39;, &#39;Oct&#39;, &#39;Nov&#39;, &#39;Dec&#39;]<br>plt.xticks(np.arange(12) + .5, labels=xticks_labels)<br># axis labels<br>plt.xlabel(&#39;&#39;)<br>plt.ylabel(&#39;&#39;)<br># title<br>title = &#39;Historial monthly Average returns of main equity index \n&#39;<br>plt.title(title, loc=&#39;left&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*cjwQMGl0B5MmWKVTAbbw8g.png" /></figure><p>After analyzing the historical average monthly returns of the principal stock market indexes, we can draw some interesting conclusions about the “Sell in May and Go Away” strategy.</p><p>Firstly, we can see that the strategy does hold some merit for the S&amp;P 500 and Dow Jones Industrial Average indexes. According to the data, these indexes have weaker average monthly returns during the summer months (May-October) compared to the winter months (November-April). Specifically, the S&amp;P 500 produced an average return of only 0.66% during the May-October period, compared to an average return of 1.33% during the November-April period. Similarly, the Dow Jones Industrial Average produced an average return of only 0.31% during the May-October period, compared to an average return of 1.37% during the November-April period.</p><p>However, this trend is not as clear for the NASDAQ and Russell 2000 indexes. While the NASDAQ did produce slightly weaker average monthly returns during the summer months, the difference is not significant enough to support the “Sell in May and Go Away” strategy. The Russell 2000, on the other hand, actually produced stronger average monthly returns during the May-October period compared to the November-April period.</p><p>Overall, it’s important to note that the seasonal pattern in stock market returns is not always consistent from year to year, and there are many other factors that can influence market performance. As a result, investors should not rely solely on the “Sell in May and Go Away” strategy when making investment decisions.</p><p>In conclusion, while the historical data does provide some support for the “Sell in May and Go Away” strategy for the S&amp;P 500 and Dow Jones Industrial Average indexes, investors should consider other factors and strategies when making investment decisions. As with any investing strategy, it’s crucial to conduct thorough research and analysis to make informed decisions that align with your investment goals and risk tolerance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96cf8ed3df1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tweet, Trade, Profit: Exploring the Relationship between Elon Musk’s Tweets and Dogecoin Volatility…]]></title>
            <link>https://medium.com/@facujallia/tweet-trade-profit-exploring-the-relationship-between-elon-musks-tweets-and-dogecoin-volatility-7da28ee9aead?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/7da28ee9aead</guid>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[algorithmic-trading]]></category>
            <category><![CDATA[criptocurrency]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Thu, 20 Apr 2023 22:29:19 GMT</pubDate>
            <atom:updated>2023-04-20T23:43:52.673Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Tweet, Trade, Profit: Exploring the Relationship between Elon Musk’s Tweets and Dogecoin Volatility with NLP and Econometric Approach</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6q_5fkhQJE_D-luICL_uw.jpeg" /></figure><p>In recent years, social media platforms have become a powerful tool for investors seeking to gain an edge in the financial markets. Among the most influential social media figures in the investment community is Elon Musk, the billionaire entrepreneur and CEO of companies such as SpaceX and Tesla. In particular, Musk’s tweets about cryptocurrencies such as Dogecoin have been known to cause significant price movements in the market.</p><p>The hypothesis of this study is that Elon Musk’s tweets about Dogecoin can significantly impact the cryptocurrency’s volatility and price movements. By analyzing Musk’s tweet history and correlating it with market data on Dogecoin, we aim to identify trading opportunities that can benefit from the resulting increase in volatility.</p><p>The analysis is divided into different sections:</p><p>1. Downloading DOGE prices</p><p>2. Volatility has entered the chat</p><p>3. Elon Musk’s tweets: the ultimate source of memes and controversy</p><p>4. Natural Language Processing (NLP) Techniques</p><p>5. Statistical models</p><p>6. Possible trading strategies and final thoughts</p><p>First we need to import the libraries that we going to use for the analysis</p><pre># pandas and numpy for data manipulation and analysis<br>import pandas as pd<br>import numpy as np<br><br># re and json for regular expression operations and JSON parsing respectively<br>import re<br>import json<br><br># ccxt for cryptocurrency trading APIs<br>import ccxt<br><br># nltk for natural language processing tasks such as tokenization, stopword removal, and lemmatization<br>import nltk<br>from nltk.tokenize import word_tokenize<br>from nltk.corpus import stopwords<br>from nltk.stem import WordNetLemmatizer<br><br># statsmodels for statistical modeling and analysis<br>import statsmodels.api as sm<br>import statsmodels.formula.api as smf<br>from statsmodels.stats.anova import anova_lm<br><br># plotly and matplotlib for data visualization<br>import plotly.graph_objs as go<br>import plotly.express as px<br>import plotly.io as pio<br>from plotly.subplots import make_subplots<br>import matplotlib.pyplot as plt<br><br># wordcloud for generating word clouds from text data<br>from wordcloud import WordCloud</pre><h3><strong>1. Downloading DOGE prices</strong></h3><p>We use ccxt to create an instance of the Binance exchange API and fetch OHLCV data for the DOGE/USDT trading pair at a 1-hour timeframe since January 1st, 2021. We then retrieve the data in batches and append it to a Pandas DataFrame for further analysis.</p><p><strong>ccxt</strong> is a popular open-source library that provides a uniform interface for accessing various cryptocurrency exchange APIs. A while loop is used to download the historical OHLCV data in batches of 500 observations each. The reason for this is that many cryptocurrency exchange APIs have rate limits on data requests to prevent excessive traffic and potential overload on their servers. By limiting the number of observations in each request, we can reduce the chances of hitting these rate limits and ensure that our data is downloaded efficiently.</p><pre>#Create an instance of the Binance exchange API<br>exchange = ccxt.binance()<br><br>#Define the trading pair symbol to fetch data for<br>symbol = &#39;DOGE/USDT&#39;<br><br>#Define the timeframe to fetch data for<br>timeframe = &#39;1h&#39;<br><br>#Define the starting date to fetch data from<br>from_date = &#39;2021-01-01T00:00:00Z&#39;<br><br>#Set the number of candles to retrieve in each request<br>limit = 500<br><br>#Initialize an empty list to store the fetched OHLCV data<br>ohlcv = []<br><br>#Loop through the data requests until all data is fetched<br>while True:<br>    data = exchange.fetch_ohlcv(symbol, timeframe, since=exchange.parse8601(from_date), limit=limit)<br>    ## Append the fetched data to the ohlcv list<br>    ohlcv += data<br>    if len(data) &lt; limit:<br>        break<br>    from_date = exchange.iso8601(data[-1][0]+1)  # Set the new from_date to avoid duplicates<br><br>#Create a Pandas DataFrame from the fetched OHLCV data<br>df_doge = pd.DataFrame(ohlcv, columns=[&#39;timestamp&#39;, &#39;open&#39;, &#39;high&#39;, &#39;low&#39;, &#39;close&#39;, &#39;volume&#39;])<br><br>#Convert the timestamp column to a datetime format<br>df_doge[&#39;timestamp&#39;] = pd.to_datetime(df_doge[&#39;timestamp&#39;], unit=&#39;ms&#39;)</pre><p>We can visualize the data in a Plotly chart:</p><pre>pio.templates.default = &quot;plotly_dark&quot; # set dark background<br><br>fig = go.Figure()<br>fig.add_trace(go.Scatter(x=df_doge[&#39;timestamp&#39;], y=df_doge[&#39;close&#39;], name=&#39;DOGE-USD&#39;,<br>                         line=dict(color=&#39;#e1b303&#39;, width=2)))<br><br>fig.update_layout(title=(&#39;Hitorical closing price of DOGE/USDT in Binance&#39;),<br>                  xaxis_title=&#39;Date&#39;,<br>                  yaxis_title=&#39;Cumulative performance&#39;,<br>                  font=dict(color=&#39;white&#39;),<br>                  paper_bgcolor=&#39;black&#39;,<br>                  plot_bgcolor=&#39;black&#39;,<br>                  legend=dict(x=0, y=1.2, bgcolor=&#39;rgba(0,0,0,0)&#39;),<br>                  yaxis=dict(gridcolor=&#39;rgba(255,255,255,0.2)&#39;),<br>                  xaxis=dict(gridcolor=&#39;rgba(255,255,255,0.2)&#39;))<br>fig.update_xaxes(showgrid=True, ticklabelmode=&quot;period&quot;)<br>fig.update_yaxes(showgrid=True)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9oi15acWNSt90SqAJWcCvw.png" /></figure><h3><strong>2. Volatility has entered the chat</strong></h3><p>Volatility is a commonly used measure of risk in financial markets. It refers to the degree of variation in the price or value of an asset over a certain period of time. Higher volatility is generally associated with higher risk, as it implies a greater degree of uncertainty and unpredictability in the asset’s value.</p><p>In the context of cryptocurrency trading, volatility is a particularly important metric, as many cryptocurrencies are known for their high levels of price volatility (DOGE and SHIBA are extreme examples). In order to calculate volatility, we can use the standard deviation of the logarithmic returns of the asset’s price over a certain time period.</p><p>Measuring the impact of news or important opinions (such as Elon Musk’s tweets) in volatility rather than in price is a good idea because it allows us to more accurately capture the underlying sentiment and market reaction to the news. In many cases, the impact of news on an asset’s price may be short-lived or may not fully reflect the true sentiment of the market. By measuring the impact of volatility, we can more effectively capture the level of uncertainty and risk associated with the news, which can be valuable information for traders and investors.</p><p>Furthermore, volatility-based trading strategies are often more effective than price-based strategies in high-volatility markets, as they can help to mitigate risk and take advantage of sudden price movements. By analyzing the relationship between news and volatility in cryptocurrency markets, we can develop more effective trading strategies that are better suited to the unique characteristics of these markets.</p><pre># Calculate the log returns of the closing prices<br>log_returns = np.log(df_doge[&#39;close&#39;]) - np.log(df_doge[&#39;close&#39;].shift(1))<br><br># Calculate the volatility using a rolling window of 24 hours<br>df_doge[&#39;volatility&#39;] = log_returns.rolling(window=24).std() * 100</pre><p>We can visualize the data in a Plotly chart:</p><pre># Create a bar chart with the close price and volatility<br>fig = go.Figure()<br><br>fig.add_trace(go.Scatter(x=df_doge[&#39;timestamp&#39;], y=df_doge[&#39;close&#39;], name=&#39;Close&#39;, line_color=&#39;#e1b303&#39;))<br>fig.add_trace(go.Bar(x=df_doge[&#39;timestamp&#39;], y=df_doge[&#39;volatility&#39;], name=&#39;Volatility&#39;, <br>                     marker_color=&#39;#eeeeee&#39;, yaxis=&#39;y2&#39;, width=0.8, opacity=0.4,<br>                     marker_line_width=0))<br><br># Add title and axis labels<br>fig.update_layout(title=&#39;DOGE/USDT Close Price and Volatility&#39;,<br>                  xaxis_title=&#39;Date&#39;,<br>                  yaxis_title=&#39;Price (USDT)&#39;,<br>                  yaxis2=dict(title=&#39;Volatility&#39;, overlaying=&#39;y&#39;, side=&#39;right&#39;),<br>                  plot_bgcolor=&#39;#111111&#39;,<br>                  paper_bgcolor=&#39;#111111&#39;,<br>                  yaxis=dict(gridcolor=&#39;white&#39;))<br><br># Show the chart<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAu3BY1E-cl2Ndk0zJM__g.png" /></figure><p><strong>3. Elon Musk’s tweets: the ultimate source of memes and controversy</strong></p><p>Elon Musk, the co-founder and CEO of SpaceX and Tesla, is one of the most popular figures in the tech industry today. Musk has a massive following on Twitter, where he frequently shares his thoughts and updates on his various companies. As a result, analyzing his tweets has become a popular way to gain insight into his business strategies and thought process.</p><p>In this article, we will be analyzing a dataset of tweets posted by Elon Musk. The dataset was downloaded from <a href="https://data.world/barbaramaseda/elon-musk-tweets">Data World</a> and consists of tweets that were collected automatically using an IFTTT applet. The dataset includes a variety of information such as the tweet text, username, link to the tweet, and the date it was posted.</p><pre># Open the JSONL file and read each line as a separate JSON object<br>with open(&#39;C:/Trading algoritmico/DOGE Musk/user-tweets.jsonl&#39;) as f:<br>    data = [json.loads(line) for line in f]<br><br># Convert the list of JSON objects to a Pandas DataFrame<br>df_musk = pd.DataFrame.from_records(data)<br><br># Drop the columns UserName and TweetEmbedCode using the drop method from pandas:<br>df_musk = df_musk.drop([&#39;UserName&#39;, &#39;TweetEmbedCode&#39;], axis=1)<br><br># Convert the &#39;CreatedAt&#39; column to datetime format<br>df_musk[&#39;CreatedAt&#39;] = pd.to_datetime(df_musk[&#39;CreatedAt&#39;])<br><br># Subtract one hour from the CreatedAt column to adjust to UTC-Time<br>df_musk[&#39;CreatedAt&#39;] = df_musk[&#39;CreatedAt&#39;] - pd.Timedelta(hours=1)<br><br># Filter the data starting 1-1-2021<br>df_musk = df_musk[df_musk[&#39;CreatedAt&#39;] &gt;= &#39;2021-01-01&#39;]</pre><p>The result is a dataframe with 20981 Elon Musk tweets. We can visualize the data in a Plotly chart:</p><pre># Group the tweets by month and count the number of tweets per month<br>tweet_counts = df_musk.groupby(df_musk[&#39;CreatedAt&#39;].dt.strftime(&#39;%B %Y&#39;))[&#39;Text&#39;].count()<br><br># Convert the month name to a numerical value and sort the index<br>months = pd.to_datetime(tweet_counts.index, format=&#39;%B %Y&#39;).strftime(&#39;%Y-%m&#39;)<br>tweet_counts = tweet_counts.sort_index(key=lambda x: months)<br><br># Create the Plotly bar plot<br>fig = px.bar(x=tweet_counts.index, y=tweet_counts.values,<br>             labels={&#39;x&#39;: &#39;Month&#39;, &#39;y&#39;: &#39;Number of tweets&#39;},<br>             title=&#39;Quantity of tweets per month&#39;)<br>fig.update_layout(plot_bgcolor=&#39;black&#39;, paper_bgcolor=&#39;black&#39;,<br>                  font_color=&#39;white&#39;, title_font_color=&#39;white&#39;,<br>                  xaxis_tickangle=-45, xaxis_tickfont=dict(size=14),<br>                  yaxis_tickfont=dict(size=14), showlegend=False)<br>fig.update_traces(marker_color=&#39;#e1b303&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YuWUw2sxf_YtALZTwy9vBA.png" /></figure><h3><strong>4. Natural Language Processing (NLP) Techniques</strong></h3><p>Natural Language Processing (NLP) techniques are a set of tools and algorithms used to analyze and understand human language. With the explosion of data and information available on the internet, NLP has become increasingly important in fields such as machine learning, artificial intelligence, and data science.</p><p>When analyzing Elon Musk’s tweets related to Dogecoin, applying these NLP techniques can help filter out irrelevant information and focus on the most important tweets. By tokenizing, removing stopwords, and lemmatizing the text, we can better understand the frequency and context of words related to Dogecoin, and potentially gain insights into Musk’s thoughts and actions regarding this cryptocurrency.</p><h4><strong>Tokenization</strong></h4><p>Tokenization is the process of breaking down a text into individual words or tokens. This is necessary because computers don’t understand text as humans do — they need data in a structured format. By tokenizing a text, we can easily count the frequency of individual words and analyze their relationships.</p><pre># Download the punkt tokenizer if not already downloaded<br>nltk.download(&#39;punkt&#39;)<br><br># Tokenize the text in the &#39;Text&#39; column<br>df_musk[&#39;Text_Tokenized&#39;] = df_musk[&#39;Text&#39;].apply(lambda x: word_tokenize(x.lower()))</pre><h4><strong>Stopwords</strong></h4><p>Stopword removal is the process of removing commonly used words such as “the”, “a”, and “an”. These words don’t add much meaning to a text, and removing them can improve the accuracy of NLP models by reducing noise in the data.</p><pre># Download the stopwords if not already downloaded<br>nltk.download(&#39;stopwords&#39;)<br><br># Remove stopwords from the tokenized tweets<br>stop_words = set(stopwords.words(&#39;english&#39;))<br>df_musk[&#39;Filtered&#39;] = df_musk[&#39;Text_Tokenized&#39;].apply(lambda tokens: [token for token in tokens if token.lower() not in stop_words])</pre><h4><strong>Lemmatization</strong></h4><p>Lemmatization is the process of reducing words to their base or root form. This is important because words in different forms (e.g. “run”, “running”, “ran”) may have the same meaning, and treating them as separate words can lead to inaccurate analysis. By reducing words to their base form, we can more accurately count their frequency and understand their relationships.</p><pre># Download the stopwords if not already downloaded<br>nltk.download(&#39;wordnet&#39;)<br><br># Create an instance of WordNetLemmatizer<br>lemmatizer = WordNetLemmatizer()<br><br># Define a function to perform lemmatization on a list of tokens<br>def lemmatize_tokens(tokens):<br>    lemmatized_tokens = []<br>    for token in tokens:<br>        lemmatized_token = lemmatizer.lemmatize(token)<br>        lemmatized_tokens.append(lemmatized_token)<br>    return lemmatized_tokens<br><br># Apply the lemmatization function to the &#39;TokenizedText&#39; column<br>df_musk[&#39;LemmatizedText&#39;] = df_musk[&#39;Filtered&#39;].apply(lemmatize_tokens)<br><br># Join the pre-processed tokens back into strings<br>df_musk[&#39;processed_text&#39;] = df_musk[&#39;LemmatizedText&#39;].apply(lambda tokens: &#39; &#39;.join(tokens))</pre><h4><strong>Keywords &amp; regex filter</strong></h4><p>Not all of Musk’s tweets are created equal when it comes to their impact on DOGE/USDT. Some of his tweets may be unrelated to cryptocurrency. Therefore, it is important to filter Musk’s tweets in order to identify the ones that are most likely to affect the price of DOGE/USDT.</p><p>One way to filter Musk’s tweets is to focus on those that contain certain keywords related to cryptocurrency and Dogecoin. We can create a list of keywords that are commonly associated with Dogecoin and cryptocurrency.</p><p>The code then compiles these keywords into a regular expression pattern using the re.compile() function. This pattern is designed to match any tweet that contains one or more of the specified keywords, regardless of the capitalization of the letters. By using this regular expression pattern to filter Musk’s tweets, we can identify tweets relevant to DOGE.</p><pre># Create a list of keywords that are commonly associated with Dogecoin and cryptocurrency<br>keyword_doge = [&#39;dogecoin&#39;, &#39;dogefather&#39;, &#39;dogecointweet&#39;, &#39;dogecoinxx&#39;, &#39;crypto&#39;, &#39;hodl&#39;,<br>                 &#39;hodling&#39;, &#39;buy&#39;, &#39;btc&#39;, &#39;shiba&#39;, &#39;eth&#39;, &#39;ethereum&#39;, &#39;merge&#39;, &#39;hold&#39;, <br>                 &#39;memeinvestors&#39;, &#39;bitcoin&#39;, &#39;web3&#39;, &#39;hedge&#39;, &#39;shrubbery&#39;, &#39;itsdogecoin&#39;, &#39;doge&#39;]<br><br># Create a regular expression pattern from the list of keywords<br>pattern = re.compile(r&#39;\b(&#39; + &#39;|&#39;.join(keyword_doge) + r&#39;)\b&#39;, flags=re.IGNORECASE)<br><br># Apply the pattern to the &#39;processed_text&#39; column of the filtered dataframe<br>mask = df_musk[&#39;processed_text&#39;].str.contains(pattern)<br><br># Apply the mask to the filtered dataframe<br>df_musk_doge = df_musk[mask]<br><br># create a function to check if the keyword is only in mentions<br>def check_keyword_only_in_mentions(text, keyword):<br>    mentions = re.findall(r&#39;@\w+&#39;, text) # find all mentions in the tweet<br>    if keyword in mentions:<br>        # if the keyword is only in mentions, return True<br>        if text.count(keyword) == len(mentions):<br>            return True<br>    return False<br><br># apply the function to create a mask<br>mask = df_musk_doge[&#39;processed_text&#39;].apply(lambda x: check_keyword_only_in_mentions(x, &#39;dogecoin&#39;))<br><br># filter the dataframe with the mask<br>df_musk_doge = df_musk_doge[~mask]</pre><p>After applying the filter, we end up with a smaller subset of tweets that are more likely to have an impact on DOGE/USDT price. The resulting dataset consists of 161 tweets that mention specific keywords. By filtering the tweets in this way, we are able to reduce the noise in the data and focus on the tweets that are most likely to have an impact on the price of DOGE/USDT.</p><p>To further explore the relationship between DOGE/USDT and Elon Musk’s tweets, we can create a word cloud of the most common words in the filtered tweets. This visualization allows us to quickly identify the most frequently mentioned words related to DOGE/USDT and Elon Musk’s tweets.The word cloud for the filtered tweets shows that the most common words include “doge”, “crypto”, “coin”, and “buy”, among others.</p><pre># Concatenate all the tweets into a single string<br>text = &#39; &#39;.join(df_musk_doge[&#39;Text&#39;].tolist())<br><br># Generate the wordcloud<br>wordcloud = WordCloud(width=800, height=400, max_words=200, background_color=&#39;black&#39;, colormap=&#39;Wistia&#39;).generate(text)<br><br># Configure the plot<br>plt.figure(figsize=(10,10), facecolor=&#39;k&#39;, dpi=100)<br>plt.imshow(wordcloud, interpolation=&#39;bilinear&#39;)<br>plt.axis(&#39;off&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*tDecV7EMZrXCh--QStHMMQ.png" /></figure><h3><strong>5. Statistical models</strong></h3><p>In order to analyze the relationship between Elon Musk’s tweets and DOGE price and volatility we need to make a series of steps before implementing a statistic model. After doing some format transformation, we merge df_musk_doge and df_doge to put all the data in one dataframe.</p><p>Then, we create a column named <strong>Text_Polarity</strong> to indicate the presence of an Elon tweet relative to DOGE, and two columns <strong>close_pct_change</strong> and <strong>vol_pct_change </strong>to contain the variation of price and volatility in percentage.</p><pre># Rename column as timestamp<br>df_musk_doge = df_musk_doge.rename(columns={&#39;CreatedAt&#39;: &#39;timestamp&#39;})<br><br># Assuming &#39;timestamp&#39; is already in datetime format<br>df_musk_doge[&#39;timestamp&#39;] = df_musk_doge[&#39;timestamp&#39;].dt.strftime(&#39;%Y-%m-%d %H:00:00&#39;)<br><br># Convert timestamp column to datetime format<br>df_musk_doge[&#39;timestamp&#39;] = pd.to_datetime(df_musk_doge[&#39;timestamp&#39;])<br><br># Group the data frame by timestamp and concatenate the processed_text column<br>df_musk_doge[&#39;processed_text&#39;] = df_musk_doge[&#39;processed_text&#39;].fillna(&#39;&#39;)<br>df_musk_doge_gr = df_musk_doge.groupby(&#39;timestamp&#39;).agg({&#39;processed_text&#39;: lambda x: &#39; &#39;.join(x)})<br><br># Merge the data frames<br>df_model = pd.merge(df_musk_doge_gr, df_doge, on=&#39;timestamp&#39;, how=&#39;right&#39;)<br><br># Create a text polarity column<br>df_model[&#39;Text_Polarity&#39;] = df_model[&#39;processed_text&#39;].apply(lambda x: 0 if pd.isna(x) else 1)<br><br># Calculate percent change in closing price<br>df_model[&#39;close_pct_change&#39;] = df_model[&#39;close&#39;].pct_change() * 100<br>df_model[&#39;close_pct_change&#39;] = df_model[&#39;close_pct_change&#39;].round(4)<br><br># Calculate percent change in volatility<br>df_model[&#39;vol_pct_change&#39;] = df_model[&#39;volatility&#39;].pct_change() * 100<br>df_model[&#39;vol_pct_change&#39;] = df_model[&#39;vol_pct_change&#39;].round(4)</pre><h4><strong>Price analysis</strong></h4><p>We can visualize the percent change of DOGE after Elon Musk tweets in a Plotly chart:</p><pre># Filter the data to include only Elon Musk tweets<br>doge_on_elon_tweets = df_model[df_model[&quot;Text_Polarity&quot;] == 1]<br><br># Create a trace for the percent change<br>trace1 = go.Bar(x=df_model[&#39;timestamp&#39;], y=df_model[&#39;close_pct_change&#39;], name=&#39;DOGE/USDT % chg.&#39;, marker=dict(color=&#39;#e1b303&#39;), marker_line_width=0, opacity=0.4)<br>trace2 = go.Bar(x=doge_on_elon_tweets[&#39;timestamp&#39;], y=doge_on_elon_tweets[&#39;close_pct_change&#39;], name=&#39;DOGE/USDT % chg. after Elon tweet&#39;, marker=dict(color=&#39;#00acee&#39;), marker_line_width=0)<br><br># Create a list of vertical line positions<br>vertical_lines = df_model[df_model[&#39;processed_text&#39;].notna()][&#39;timestamp&#39;]<br><br># Configure the layout<br>layout = go.Layout(title=&#39;DogeCoin Percent of Change After Elon Musk Tweets&#39;, xaxis=dict(title=&#39;Date&#39;), yaxis=dict(title=&#39;DogeCoin Percent of Change&#39;))<br>layout.legend=dict(<br>    orientation=&quot;h&quot;,<br>    yanchor=&quot;top&quot;,<br>    y=1.2,<br>    xanchor=&quot;right&quot;,<br>    x=1<br>)<br># Combine the traces into a single plot<br>data = [trace1, trace2]<br><br># Create the figure<br>fig = go.Figure(data=data, layout=layout)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tJr2wfx3oS4k6mFQ7Wkd2Q.png" /></figure><p>As we can see in the graph above, the tweets led to important fluctuations in price, but not all the tweets produced positive impacts. In order to analyze the strong of the correlation, a first approximation is to compare the mean price change of DOGE/USDT with the mean price change after Elon Musk posts a DOGE-related tweet.</p><pre># Total mean<br>total_mean_price_change = df_model[&quot;close_pct_change&quot;].mean()<br><br># Compute the mean price change and volatility change after each Elon Musk tweet<br>mean_price_change = doge_on_elon_tweets[&quot;close_pct_change&quot;].mean()<br><br># Print the results<br>print(&quot;Mean price change: {:.2f}%&quot;.format(total_mean_price_change))<br>print(&quot;Mean price change after Elon Musk&#39;s tweet:: {:.2f}%&quot;.format(mean_price_change))</pre><blockquote>Mean price change: 0.03%</blockquote><blockquote>Mean price change after Elon Musk’s tweet:: 1.71%</blockquote><p>The mean price change before the tweet was found to be 0.03%, whereas after the tweets, the mean price change was found to be 1.71%. This suggests that Elon Musk’s tweets had a significant impact on the price of the asset, causing a much larger price change than the average price change before the tweets. We may conclude that Elon Musk’s tweets can be important signals for predicting future price movements of the asset, but for a deeper analysis we need to apply a stadistic model.</p><p>ANOVA is a statistical method used to analyze the differences among means and to test the hypothesis of whether these differences are statistically significant. In this case, we are using it to examine whether there is a significant relationship between the sentiment of Elon Musk’s tweets (measured by Text_Polarity) and the percentage change in the price of DOGE (close_pct_change). The summary of the ANOVA model provides information about the model fit, including the R-squared value, F-statistic, and p-value, which can help us to determine the strength of the relationship between the two variables.</p><pre># Fit the ANOVA model<br>model = smf.ols(&#39;close_pct_change ~ Text_Polarity&#39;, data=df_model).fit()<br><br># Print the model summary<br>print(model.summary())</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*oEkjmNiBwc6QVg1e1FWYHA.png" /></figure><p>The ANOVA test results with a dependent variable of close_pct_change and an independent variable of Text_Polarity show that the model has an R-squared value of 0.006, indicating that only 0.6% of the variance in close_pct_change is explained by Text_Polarity. The F-statistic of 118.9 suggests that the model is statistically significant, and the P-value of less than 0.05 for Text_Polarity indicates that it is a significant predictor of close_pct_change.</p><p>Overall, the model suggests that Text_Polarity may have some predictive power over close_pct_change, but the low R-squared value indicates that it is not a strong predictor. The high kurtosis value and potential for outliers suggest that the model may not be entirely reliable, and further analysis may be necessary before using it as a base for a trading strategy.</p><h4><strong>Volatility analysis</strong></h4><p>We can visualize the percent change of DOGE volatility after Elon Musk tweets in a Plotly chart:</p><pre># Create a trace for the percent change<br>trace1 = go.Bar(x=df_model[&#39;timestamp&#39;], y=df_model[&#39;vol_pct_change&#39;], name=&#39;DOGE/USDT % chg.&#39;, marker=dict(color=&#39;red&#39;), marker_line_width=0, opacity=0.4)<br>trace2 = go.Bar(x=doge_on_elon_tweets[&#39;timestamp&#39;], y=doge_on_elon_tweets[&#39;vol_pct_change&#39;], name=&#39;DOGE/USDT % chg. after Elon tweet&#39;, marker=dict(color=&#39;#00acee&#39;), marker_line_width=0)<br><br># Create a list of vertical line positions<br>vertical_lines = df_model[df_model[&#39;processed_text&#39;].notna()][&#39;timestamp&#39;]<br><br># Configure the layout<br>layout = go.Layout(title=&#39;DogeCoin Percent of vol After Elon Musk Tweets&#39;, xaxis=dict(title=&#39;Date&#39;), yaxis=dict(title=&#39;DogeCoin Percent of Change&#39;))<br>layout.legend=dict(<br>    orientation=&quot;h&quot;,<br>    yanchor=&quot;top&quot;,<br>    y=1.2,<br>    xanchor=&quot;right&quot;,<br>    x=1<br>)<br><br># Combine the traces into a single plot<br>data = [trace1, trace2]<br><br># Create the figure<br>fig = go.Figure(data=data, layout=layout)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AOWr-eKFsnchZkvHeGIYrg.png" /></figure><p>The chart reveals the presence of several volatility spikes that, in some cases, coincide with tweets by Elon Musk posted at the same time. Again, a first approximation is to compare the mean volatility change of DOGE/USDT with the mean volatility change after Elon Musk posts a DOGE-related tweet.</p><pre># Total metrics<br>total_mean_volatility_change = df_model[&quot;vol_pct_change&quot;].mean()<br><br># Compute the mean price change and volatility change after each Elon Musk tweet<br>mean_volatility_change = doge_on_elon_tweets[&quot;vol_pct_change&quot;].mean()<br><br># Print the results<br>print(&quot;Mean volatility change: {:.2f}%&quot;.format(total_mean_volatility_change ))<br>print(&quot;Mean volatility change after Elon Musk&#39;s tweet: {:.2f}%&quot;.format(mean_volatility_change))</pre><blockquote>Mean volatility change: 0.44%</blockquote><blockquote>Mean volatility change after Elon Musk’s tweet: 10.43%</blockquote><p>This indicates that there was a significant change in volatility after Elon Musk tweeted, much more notorious than the change produced in the price. The mean volatility change was 0.4397%, while the mean volatility change after Elon Musk tweeted was 10.4296%. This suggests that Elon Musk’s tweets had a considerable impact on the volatility of the market, but for a deeper analysis we need to apply a stadistic model.</p><pre># Fit the ANOVA model<br>model = smf.ols(&#39;vol_pct_change ~ Text_Polarity&#39;, data=df_model).fit()<br><br># Print the model summary<br>print(model.summary())</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*7Ea_e93Fts9vzz8LvGjjIQ.png" /></figure><p>The model resulted in an R-squared value of 0.005, which indicates that only 0.5% of the variability in the vol_pct_change can be explained by Text_Polarity. On the other hand, the coefficient of Text_Polarity is 10.0714, which indicates that a one-unit increase in Text_Polarity is associated with increases in vol_pct_change. Also, the F-statistic of 98.14 indicates that the model is significant, meaning that there is strong evidence to suggest that there is a relationship between Text_Polarity and vol_pct_change. In conclusion, the analysis shows that Elon Musk tweets related to DOGE have a statistically significant relationship with DOGE volatility. However, the R-squared value suggests that other factors might have a more significant impact on DOGE volatility.</p><p>To go one step further, we can analyze the ANOVA table. The ANOVA table shows the degrees of freedom, the sum of squares, the mean square, the F-statistic, and the associated p-value for each of the variables in the model. The ANOVA table can be used to test the significance of the variables in the model and to compare the fit of different models.</p><pre># Perform the ANOVA test<br>anova_table = anova_lm(model)<br><br># Print the ANOVA table<br>print(anova_table)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*kigSUOl8kP6Kyr56FVmfKw.png" /></figure><p>As per the ANOVA table output, the p-value for the variable “Text_Polarity” is less than the significance level of 0.05, which indicates that there is a significant relationship between “Text_Polarity” and “vol_pct_change” in the model. It means that Elon Musk tweets related to DOGE have a statistically significant impact on the volatility of the cryptocurrency.</p><p>Based on this analysis, it could be a good idea to use Elon Musk tweets related to DOGE as a signal of a potential increment in the volatility. Traders could use this signal to buy options or enter into volatility trades to profit from the expected increase in volatility. However, it is important to note that other factors should also be considered to avoid risks and maximize profitability.</p><h3><strong>6. Possible trading strategies and final thoughts</strong></h3><p>As we could see in the statistical analysis of the correlation with prices and volatility, a trading strategy that seeks to predict jumps in volatility based on Elon Musk’s tweets referring to Dogecoin would have more strength than a strategy that seeks to benefit from price changes.</p><p>We can attribute these results to the fact that the applied Natural Language Processing procedures filtered the tweets that were related to the topic, without distinguishing between those that had a positive or negative charge. Another possible approach to obtain a model with a higher correlation with positive price changes could include a sentiment analysis.</p><p>As for the possible strategies that take advantage of increases in volatility, they would involve the use of derivatives on DOGE/USDT. Exchanges such as Binance, ByBit and DeriBit offer the possibility of trading options on crypto assets.</p><p>Options trading strategies such as the straddles, strangles, straps, and strips can be powerful tools for traders looking to take advantage of potential volatility caused by Elon Musk’s tweets related to DOGE. These strategies can help traders limit their exposition to price movement while profiting from the increased volatility caused by Musk’s tweets:</p><p><strong>Straddle</strong>: A straddle involves buying a call and a put option at the same strike price and expiration date. This strategy profits when the underlying asset’s price moves significantly in either direction.</p><p><strong>Strangle</strong>: A strangle is similar to a straddle, but instead of buying options at the same strike price, the call and put options have different strike prices. This strategy profits when the underlying asset’s price moves significantly but is more flexible than the straddle as the options’ strike prices can be adjusted to suit the trader’s expectations.</p><p><strong>Strap</strong>: A strap is a bullish options trading strategy that involves buying two call options and one put option at the same strike price and expiration date. This strategy is profitable if the underlying asset’s price increases significantly.</p><p><strong>Strip</strong>: A strip is a bearish options trading strategy that involves buying two put options and one call option at the same strike price and expiration date. This strategy is profitable if the underlying asset’s price decreases significantly.</p><pre># Define the option contract details<br>strike_price = 0.30<br>call_price = 0.02<br>put_price = 0.01<br><br># Define the underlying asset price range<br>underlying_price = np.arange(0.20, 0.40, 0.01)<br><br># Define the payoff and profit functions for each strategy<br>def straddle_payoff(underlying_price):<br>    return (np.maximum(underlying_price - strike_price, 0) - call_price + np.maximum(strike_price - underlying_price, 0) - put_price) * 100<br><br>def strangle_payoff(underlying_price):<br>    return (np.where(underlying_price &lt; strike_price - 0.02, 0, underlying_price - strike_price + 0.02)<br>            - call_price + np.where(underlying_price &gt; strike_price + 0.02, 0, strike_price - underlying_price + 0.02)<br>            - put_price) * 100<br><br>def strap_payoff(underlying_price):<br>    return (np.maximum(underlying_price - strike_price, 0) - 2*call_price) * 100<br><br>def strip_payoff(underlying_price):<br>    return (np.maximum(strike_price - underlying_price, 0) - 2*put_price) * 100<br><br># Create the subplots<br>fig = make_subplots(rows=2, cols=2,<br>                    subplot_titles=(&#39;Straddle&#39;, &#39;Strangle&#39;, &#39;Strap&#39;, &#39;Strip&#39;),    <br>                    vertical_spacing=0.3,<br>                    horizontal_spacing=0.1)<br><br># Add the payoff and profit lines for each strategy<br>fig.add_trace(go.Scatter(x=underlying_price, y=straddle_payoff(underlying_price), name=&#39;Straddle Payoff&#39;), row=1, col=1)<br>fig.add_trace(go.Scatter(x=underlying_price, y=strangle_payoff(underlying_price), name=&#39;Strangle Payoff&#39;), row=1, col=2)<br>fig.add_trace(go.Scatter(x=underlying_price, y=strap_payoff(underlying_price), name=&#39;Strap Payoff&#39;), row=2, col=1)<br>fig.add_trace(go.Scatter(x=underlying_price, y=strip_payoff(underlying_price), name=&#39;Strip Payoff&#39;), row=2, col=2)<br><br># Define the strike prices<br>strike_price_strangle = 0.32<br>strike_price_strap = 0.28<br>strike_price_strip_call = 0.32<br>strike_price_strip_put = 0.28<br><br># Calculate the payoffs and profits for the strangle option strategy<br>payoff_strangle = ((np.maximum(underlying_price - strike_price_strangle, 0) - call_price) +<br>(np.maximum(strike_price_strangle - underlying_price, 0) - put_price)) * 100<br><br># Calculate the payoffs and profits for the strap option strategy<br>payoff_strap = (np.minimum(strike_price_strap - underlying_price, 0) - call_price +<br>np.minimum(underlying_price - strike_price_strap, 0) - put_price) * 100<br><br># Calculate the payoffs and profits for the strip option strategy<br>payoff_strip_call = (np.minimum(strike_price_strip_call - underlying_price, 0) - call_price) * -100<br>payoff_strip_put = (np.minimum(underlying_price - strike_price_strip_put, 0) - put_price) * -100<br>payoff_strip = payoff_strip_call + payoff_strip_put<br><br># Add the subplots<br>fig.add_trace(go.Scatter(x=underlying_price, y=payoff_strap, name=&#39;Strap&#39;, visible=False))<br>fig.add_trace(go.Scatter(x=underlying_price, y=payoff_strip, name=&#39;Strip&#39;, visible=False))<br><br># Set the axis labels and titles<br>fig.update_xaxes(title_text=&#39;Underlying Asset Price&#39;, row=1, col=1)<br>fig.update_yaxes(title_text=&#39;Returns&#39;, row=1, col=1)<br>fig.update_xaxes(title_text=&#39;Underlying Asset Price&#39;, row=1, col=2)<br>fig.update_yaxes(title_text=&#39;Returns&#39;, row=1, col=2)<br>fig.update_xaxes(title_text=&#39;Underlying Asset Price&#39;, row=2, col=1)<br>fig.update_yaxes(title_text=&#39;Returns&#39;, row=2, col=1)<br>fig.update_xaxes(title_text=&#39;Underlying Asset Price&#39;, row=2, col=2)<br>fig.update_yaxes(title_text=&#39;Returns&#39;, row=2, col=2)<br>fig.update_layout(title=&#39;Example Option Strategies on DOGE/USDT&#39;)<br>fig.update_layout(legend=dict(<br>    orientation=&quot;h&quot;,<br>    yanchor=&quot;top&quot;,<br>    y=1.2,<br>    xanchor=&quot;right&quot;,<br>    x=1<br>))<br><br># Show the plot<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FTmhLH9qBC_sZhbgNPcf0w.png" /></figure><p>To see more projects like this go to my GitHub profile: <a href="https://github.com/facundoallia">https://github.com/facundoallia</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7da28ee9aead" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Maximizing Profitability of Moving Average Crossover Trading Strategy: What moving averages to use?]]></title>
            <link>https://medium.com/@facujallia/maximizing-profitability-of-moving-average-crossover-trading-strategy-what-moving-averages-to-use-e5cd70085690?source=rss-b77229e22a72------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5cd70085690</guid>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[moving-average]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[trading]]></category>
            <dc:creator><![CDATA[Facundo Joel Allia Fernandez]]></dc:creator>
            <pubDate>Wed, 12 Apr 2023 23:42:14 GMT</pubDate>
            <atom:updated>2023-04-12T23:42:14.752Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tLHh3hBfDvGocDNFsVScoQ.jpeg" /></figure><p>As traders, we are always searching for the best trading strategies that will maximize profitability while minimizing risks. The dual moving average crossover trading strategy is a popular and straightforward technique that uses two moving averages to predict market trends and entry/exit points.</p><p>To implement this strategy, we first calculate the short-term and long-term moving averages of the price of a security. These moving averages can be of different lengths and are mathematically designed to have different variances and rates of direction. When the two moving averages intersect or “cross over,” it is a signal to buy or sell the security.</p><p>The technical approach to this strategy suggests that when the short-term moving average (STMA) crosses above the long-term moving average (LTMA), it is a signal to buy or go long. Conversely, when the STMA crosses below the LTMA, it is a signal to sell or go short. This approach is based on the principle of momentum, which states that a price moving up or down in period t is likely to continue moving in the same direction in period t+1.</p><p>In this article, we will discuss how to maximize profitability with the dual moving average crossover trading strategy by exploring the different moving average combinations that traders can use. By choosing the right moving average lengths, traders can optimize their strategy and improve their chances of success.</p><h4>The basic strategy</h4><p>First, we import the libraries that we going to use for the analysis:</p><pre>import numpy as np<br>import pandas as pd<br>import yfinance as yf<br><br>import matplotlib.pyplot as plt<br>import plotly.graph_objs as go<br>import seaborn as sns</pre><p>We useyfinance (imported as yf), a library allowing easy access to Yahoo Finance’s financial data. With yfinance, you can download historical market data for stocks, options, and currencies.</p><pre>#Download ticker price data from yfinance<br>tick = &#39;ARKK&#39;<br>ticker = yf.Ticker(tick)<br>ticker_history = ticker.history(period=&#39;10y&#39;)</pre><p>For this example, we analyze 10 years of the daily price history of <strong>ARKK, </strong>the<strong> </strong>ticker of<strong> </strong><em>ARK Innovation ETF</em> but feel free to use other securities.</p><pre>#Calculate 10 and 20 days moving averages<br>ticker_history[&#39;ma10&#39;] = ticker_history[&#39;Close&#39;].rolling(window=10).mean()<br>ticker_history[&#39;ma20&#39;] = ticker_history[&#39;Close&#39;].rolling(window=20).mean()<br><br>#Create a column with buy and sell signals<br>ticker_history[&#39;signal&#39;] = 0.0<br>ticker_history[&#39;signal&#39;] = np.where(ticker_history[&#39;ma10&#39;] &gt; ticker_history[&#39;ma20&#39;], 1.0, 0.0)<br><br>#Calculate daily returns for the ticker<br>ticker_history[&#39;returns&#39;] = ticker_history[&#39;Close&#39;].pct_change()<br><br>#Calculate strategy returns<br>ticker_history[&#39;strategy_returns&#39;] = ticker_history[&#39;signal&#39;].shift(1) * ticker_history[&#39;returns&#39;]<br><br>#Calculate cumulative returns for the ticker and the strategy<br>ticker_history[&#39;cumulative_returns&#39;] = (1 + ticker_history[&#39;returns&#39;]).cumprod()<br>ticker_history[&#39;cumulative_strategy_returns&#39;] = (1 + ticker_history[&#39;strategy_returns&#39;]).cumprod()<br><br>#Print the cumulative strategy returns at the last date<br>ticker_history[&#39;cumulative_strategy_returns&#39;][-1]</pre><blockquote>2.690754730303438</blockquote><p>This indicates that the result of the strategy in the period analyzed is 269.075%. In the same period, a Buy and Hold strategy would have had a return of 203.12%</p><p>We can create a chart with the cumulative returns for the ticker and the strategy applied with Plotly.</p><pre>fig = go.Figure()<br>fig.add_trace(go.Scatter(x=ticker_history.index, y=ticker_history[&#39;cumulative_returns&#39;], name=tick,<br>                         line=dict(color=&#39;white&#39;, width=2)))<br>fig.add_trace(go.Scatter(x=ticker_history.index, y=ticker_history[&#39;cumulative_strategy_returns&#39;], name=&#39;Estrategy&#39;,<br>                         line=dict(color=&#39;#22a7f0&#39;, width=2)))<br>fig.update_layout(title=(&#39;Moving average crossover strategy backtest in &#39; + tick),<br>                  xaxis_title=&#39;Date&#39;,<br>                  yaxis_title=&#39;Cumulative performance&#39;,<br>                  font=dict(color=&#39;white&#39;),<br>                  paper_bgcolor=&#39;black&#39;,<br>                  plot_bgcolor=&#39;black&#39;,<br>                  legend=dict(x=0, y=1.2, bgcolor=&#39;rgba(0,0,0,0)&#39;),<br>                  yaxis=dict(gridcolor=&#39;rgba(255,255,255,0.2)&#39;),<br>                  xaxis=dict(gridcolor=&#39;rgba(255,255,255,0.2)&#39;))<br>fig.update_xaxes(showgrid=True, ticklabelmode=&quot;period&quot;)<br>fig.update_yaxes(showgrid=True)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ynie-ZrL0zIOZltyhoCeeg.png" /></figure><p>But performance is not the only important thing, we can also analyze the drawdown of the strategy.</p><p>Drawdown is a financial term that refers to the maximum loss an investment portfolio or an asset has experienced from its peak value. It is the difference between the highest point of the investment’s value and the lowest point it reaches before it recovers.</p><p>Drawdowns are important because they provide insight into the risk and volatility of an investment. High drawdowns indicate that an investment is more volatile and risky, while low drawdowns suggest a more stable investment. Therefore, understanding drawdowns can help investors manage their portfolios and assess the potential risk and rewards of their investments.</p><p>We can study the drawdown of the strategy with the following code:</p><pre>#Calculate the maximum cumulative returns of the strategy so far<br>ticker_history[&#39;cumulative_strategy_max&#39;] = ticker_history[&#39;cumulative_strategy_returns&#39;].cummax()<br><br>#Calculate the current drawdown of the strategy returns in relation to the maximum cumulative returns<br>ticker_history[&#39;cumulative_strategy_drawdown&#39;] = (ticker_history[&#39;cumulative_strategy_returns&#39;] / ticker_history[&#39;cumulative_strategy_max&#39;]) - 1<br><br>#Print the current drawdown<br>print(&#39;The current drawdown of the strategy is:&#39;, ticker_history[&#39;cumulative_strategy_drawdown&#39;][-1])</pre><blockquote>The current drawdown of the strategy is: -0.5364383150776705</blockquote><p>But what if we change <em>STMA</em> to 20 and <em>STMA</em> to 40? Or <em>STMA</em> to 9 and <em>STMA</em> to 21? Taking into count that the most common <em>MA</em> are 5, 7, 9, 10, 20, 21, 30, 40, 50, 100, and 200 we can analyze the returns and drawdown of different combination <em>MA</em>.</p><h3>Returns matrix with different moving averages</h3><p>In order to create the matrix, first we develop a similar model of <em>MA</em> crossover</p><pre>#Define base slow and fast moving averages<br>fast_ma = 10<br>slow_ma = 20<br><br>#Calculate the moving averages for the strategy<br>ticker_history[&#39;fast_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=fast_ma).mean()<br>ticker_history[&#39;slow_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=slow_ma).mean()<br><br>#Create a column with buy and sell signals<br>ticker_history[&#39;signal&#39;] = np.where(ticker_history[&#39;fast_ma&#39;] &gt; ticker_history[&#39;slow_ma&#39;], 1.0, 0.0)<br><br># Calculate daily ticker and strategy returns<br>ticker_history[&#39;returns&#39;] = ticker_history[&#39;Close&#39;].pct_change()<br>ticker_history[&#39;strategy_returns&#39;] = ticker_history[&#39;signal&#39;].shift(1) * ticker_history[&#39;returns&#39;]</pre><p>Then, we create a null matrix with different combinations of moving averages using a Pandas DataFrame:</p><pre># Define two lists of fast and slow moving average values to be used as index and columns for a returns matrix<br>fast_ma_range = [5, 7, 9, 10, 20, 21, 30, 40, 50, 100]<br>slow_ma_range = [7, 9, 10, 20, 21, 30, 40, 50, 100, 200]<br><br># Create a DataFrame with the index and columns as the fast and slow moving average values, respectively<br># The DataFrame will be used to store returns data for different combinations of the moving averages<br>returns_matrix = pd.DataFrame(index=fast_ma_range, columns=slow_ma_range)</pre><p>As a result, we obtain a null 10x10 matrix. To fill it with the hypothetical returns of strategies with different combinations of MA we use 2 for loops:</p><pre># Iterate through all combinations of fast and slow moving average values<br>for fast_ma in fast_ma_range:<br>    for slow_ma in slow_ma_range:<br>        <br>        # Calculate the fast and slow moving averages of the stock&#39;s closing price<br>        ticker_history[&#39;fast_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=fast_ma).mean()<br>        ticker_history[&#39;slow_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=slow_ma).mean()<br>        <br>        # Generate a signal to buy (1.0) or sell (0.0) based on the relative positions of the two moving averages<br>        ticker_history[&#39;signal&#39;] = np.where(ticker_history[&#39;fast_ma&#39;] &gt; ticker_history[&#39;slow_ma&#39;], 1.0, 0.0)<br>        <br>        # Calculate the daily returns of the stock and the strategy<br>        ticker_history[&#39;strategy_returns&#39;] = ticker_history[&#39;signal&#39;].shift(1) * ticker_history[&#39;returns&#39;]<br>        <br>        # Calculate the cumulative returns of the strategy<br>        cumulative_strategy_returns = (1 + ticker_history[&#39;strategy_returns&#39;]).cumprod()<br>        <br>        # Add the cumulative returns of the strategy to the returns matrix<br>        returns_matrix.loc[fast_ma, slow_ma] = cumulative_strategy_returns[-1]</pre><p>Now we have a complete matrix with returns of strategies with different combinations of MA. To visualize we can create a heat map using Seaborn library:</p><pre># Make a copy of the returns_matrix and convert index and columns to string type<br>values = returns_matrix.copy()<br>values.index = values.index.astype(str)<br>values.columns = values.columns.astype(str)<br><br># Convert values to float type<br>values = values.astype(float)<br><br># Change the style of the plot and set the default color scheme<br>plt.style.use(&#39;dark_background&#39;)<br>sns.set_palette(&quot;bright&quot;)<br><br># Create a heatmap with the provided values, including annotations, using the RdYlGn color map<br>sns.heatmap(values, annot=True, cmap=&#39;RdYlGn&#39;)<br><br># Add labels to the axes<br>plt.xlabel(&#39;Slow Moving Average&#39;, fontsize=12, color=&#39;white&#39;)<br>plt.ylabel(&#39;Fast Moving Average&#39;, fontsize=12, color=&#39;white&#39;)<br><br># Add a title to the plot<br>plt.title(&#39;Heatmap of Returns&#39;, fontsize=16, color=&#39;white&#39;)<br><br># Increase the font size to 0.8 and change the color to white for all text elements<br>sns.set(font_scale=0.8, rc={&#39;text.color&#39;:&#39;white&#39;, &#39;axes.labelcolor&#39;:&#39;white&#39;, &#39;xtick.color&#39;:&#39;white&#39;, &#39;ytick.color&#39;:&#39;white&#39;})<br><br># Change the background color of the plot to black<br>fig = plt.gcf()<br>fig.set_facecolor(&#39;#000000&#39;)<br><br># Display the plot<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*N1M0pogiApG41sDggSwn5w.png" /><figcaption>Returns matrix</figcaption></figure><p>As we can see in the graph above, strategies with different combinations of moving averages can achieve uneven returns. The best 2 combinations for the analyzed period are:</p><ol><li>STMA:50, LTMA:200 with a cumulative return of 4.5 (451.48%)</li><li>STMA:7, LTMA:50 with a cumulative return of 4.2 (421.29%)</li></ol><p>200, 50, and 40 seem to be good options to choose as LTMA for this security but is not too clear to choose a good candidate for STMA.</p><p>Complementary, we can analyze drawdowns for different combinations of moving averages as we did with returns.</p><h3>Drawdowns matrix with different moving averages</h3><p>Create another null matrix with different combinations of moving averages using a Pandas DataFrame:</p><pre># Define two lists of fast and slow moving average values to be used as index and columns for a returns matrix<br>fast_ma_range = [5, 7, 9, 10, 20, 21, 30, 40, 50, 100]<br>slow_ma_range = [7, 9, 10, 20, 21, 30, 40, 50, 100, 200]<br><br># Create a DataFrame with the index and columns as the fast and slow moving average values, respectively<br># The DataFrame will be used to store returns data for different combinations of the moving averages<br>dd_matrix = pd.DataFrame(index=fast_ma_range, columns=slow_ma_range)</pre><p>As a result, we obtain a null 10x10 matrix. To fill it with the hypothetical drawdowns of strategies with different combinations of <em>MA</em> we use 2 for loops:</p><pre># Loop through different values for fast and slow moving averages<br>for fast_ma in fast_ma_range:<br>    for slow_ma in slow_ma_range:<br>        <br>        # Calculate the moving averages for the strategy<br>        ticker_history[&#39;fast_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=fast_ma).mean()<br>        ticker_history[&#39;slow_ma&#39;] = ticker_history[&#39;Close&#39;].rolling(window=slow_ma).mean()<br><br>        # Create a binary signal column for buy/sell signals<br>        ticker_history[&#39;signal&#39;] = np.where(ticker_history[&#39;fast_ma&#39;] &gt; ticker_history[&#39;slow_ma&#39;], 1.0, 0.0)<br><br>        # Calculate daily returns for the ticker and the strategy<br>        ticker_history[&#39;strategy_returns&#39;] = ticker_history[&#39;signal&#39;].shift(1) * ticker_history[&#39;returns&#39;]<br><br>        # Calculate cumulative returns for the strategy<br>        ticker_history[&#39;cumulative_strategy_returns&#39;] = (1 + ticker_history[&#39;strategy_returns&#39;]).cumprod()<br><br>        # Calculate maximum cumulative returns for the strategy<br>        ticker_history[&#39;cumulative_strategy_max&#39;] = ticker_history[&#39;cumulative_strategy_returns&#39;].cummax()<br><br>        # Calculate the drawdown for the strategy<br>        ticker_history[&#39;cumulative_strategy_drawdown&#39;] = (ticker_history[&#39;cumulative_strategy_returns&#39;] / ticker_history[&#39;cumulative_strategy_max&#39;]) - 1<br><br>        # Add drawdown to the matrix of drawdowns<br>        dd_matrix.loc[fast_ma, slow_ma] = ticker_history[&#39;cumulative_strategy_drawdown&#39;][-1]</pre><p>Now we have a complete matrix with drawdowns of strategies with different combinations of MA. To visualize we can create a heat map using the Seaborn library:</p><pre># Make a copy of the drawdown matrix and convert the index and columns to string type<br>values_dd = dd_matrix.copy()<br>values_dd.index = values_dd.index.astype(str)<br>values_dd.columns = values_dd.columns.astype(str)<br><br># Convert values to float type<br>values_dd = values_dd.astype(float)<br><br># Change the style of the plot and set the default color scheme<br>plt.style.use(&#39;dark_background&#39;)<br>sns.set_palette(&quot;bright&quot;)<br><br># Create a heatmap with the provided values, including annotations, using the RdYlGn color map<br>sns.heatmap(values_dd, annot=True, cmap=&#39;RdYlGn&#39;)<br><br># Add labels to the axes<br>plt.xlabel(&#39;Slow Moving Average&#39;, fontsize=12, color=&#39;white&#39;)<br>plt.ylabel(&#39;Fast Moving Average&#39;, fontsize=12, color=&#39;white&#39;)<br><br># Add a title to the plot<br>plt.title(&#39;Heatmap of Drawdowns&#39;, fontsize=16, color=&#39;white&#39;)<br><br># Increase the font size to 0.8 and change the color to white for all text elements<br>sns.set(font_scale=0.8, rc={&#39;text.color&#39;:&#39;white&#39;, &#39;axes.labelcolor&#39;:&#39;white&#39;, &#39;xtick.color&#39;:&#39;white&#39;, &#39;ytick.color&#39;:&#39;white&#39;})<br><br># Change the background color of the plot to black<br>fig = plt.gcf()<br>fig.set_facecolor(&#39;#000000&#39;)<br><br># Display the plot<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*lyyTr4J1wHp__-nF4YTOZQ.png" /><figcaption>Drawdowns matrix</figcaption></figure><p>Again, we see positive metrics in strategies that work with 40 and 200 LTMA for this asset. On the other hand, the matrix presents high drawdowns for fast numbers like STMA:5//LTMA:10 or STMA:7//LTMA:9.</p><h3>Final thoughts</h3><p>Through the study carried out, it can be observed how the choice of different moving averages can affect the results of the strategy. However, great care must be taken when parameterizing possible strategies since the study shows results on past quotes (see <a href="https://www.quantifiedstrategies.com/out-of-sample-trading-tests-explained/#:~:text=The%20in%2Dsample%20test%20is,of%2Dsample%20in%20your%20backtesting.">in-sample vs out-of-sample</a> for more info).</p><p>In addition, it is very important to take into account that to apply this strategy in a real environment, commissions should be added to the analysis, as well as the effect of slippage. This probably reduces the performance of the strategies, but it would be possible to add other rules such as investing at a risk-free rate when not trading, shorting when a sell signal is observed, etc.</p><p>See the complete code in <a href="https://github.com/facundoallia/Maximizing-Profitability-of-Moving-Average-Crossover-Trading-Strategy">GitHub</a> (Spanish version)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5cd70085690" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>