<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marie Imokoyende on Medium]]></title>
        <description><![CDATA[Stories by Marie Imokoyende on Medium]]></description>
        <link>https://medium.com/@marie.imokoyende?source=rss-79dc1431a34c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*B5fRe4-gLqpgz_uot2YVWQ.jpeg</url>
            <title>Stories by Marie Imokoyende on Medium</title>
            <link>https://medium.com/@marie.imokoyende?source=rss-79dc1431a34c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:50:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@marie.imokoyende/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cointegration in Practice: Implementing a Pairs-Trading Reality Check]]></title>
            <link>https://medium.com/@marie.imokoyende/cointegration-in-practice-implementing-a-pairs-trading-reality-check-736d585818c7?source=rss-79dc1431a34c------2</link>
            <guid isPermaLink="false">https://medium.com/p/736d585818c7</guid>
            <dc:creator><![CDATA[Marie Imokoyende]]></dc:creator>
            <pubDate>Sun, 04 Oct 2026 22:36:32 GMT</pubDate>
            <atom:updated>2026-10-05T03:45:26.500Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Using Claude to build pairs validation and cointegration modules in my quant trading app.</em></p><p>Summer is over. As an Ivorian-born Canadian, I’ve learned to treasure the warm months. Drinking an iced flat-white on a sweltering Toronto day is a simple pleasure. I admit I didn&#39;t read as much as I wanted to because I was too often outside. But I did manage to read the first 7 chapters of <a href="https://a.co/d/0cFLCZpr"><em>Quantitative Trading</em> (2nd Edition)</a> by Ernest P. Chan, and implemented a few of his examples along the way.</p><p>One result surprised me enough to write this up and share my findings. When I built a new panel in my quant trading app to test Chan’s textbook pairs against the last two years of market data, my tool rejected almost all of them. Using Claude, I’ve added a Pairs Validation tab and a Cointegration panel to my <a href="https://quant-trading-app.streamlit.app/">quant trading app</a>.</p><h4>The appeal of Pairs Trading</h4><p>A pairs trade doesn’t bet on direction. Instead of guessing whether the market rises or falls, you pick two related instruments whose prices have drifted apart and bet the gap closes: short the one that moves up and buy the cheaper one. The appeal is a strategy that can work in any regime, for any pair whose spread is stationary (mean-reverting).</p><p>To read more about correlation, stationarity, and cointegration, I recommend this well-written article by Hudson &amp; Thames: <a href="https://hudsonthames.org/an-introduction-to-cointegration/">An Introduction to Cointegration for Pairs Trading</a>.</p><h4>What I’ve Implemented and were it came from</h4><p>The two features I added — a Pair Validation tab and a Cointegration panel — aren’t my ideas. They’re Chan’s. Chapter 7 works through cointegration with two worked examples: Example 7.2 forms a cointegrating pair from GLD and GDX (gold bullion versus gold miners), and Example 7.3 shows that Coca-Cola and Pepsi are strongly correlated yet fail to cointegrate. I wanted to implement a toy example on 2024–2026 data. Observations in the book no longer hold. The cointegration of GLD/GLX is no longer observed.</p><h3>The Pair Validation Tab</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vB2TtE2l0y_m7FT32s68jQ.png" /><figcaption>Pair Validation Tab — Input</figcaption></figure><p>The input section is simple. Pick one of the built-in pairs from the dropdown, or type any two tickers of your own. Ticker Y is the leg you hold long; Ticker X is the hedge leg the spread is measured against. <br>The entry threshold (z) controls how far the spread must stretch from its mean before the strategy trades. The default of 2 means two standard deviations. <br>The transaction-cost slider (basis points per side) lets you match the backtest to your own broker; it defaults to roughly what Interactive Brokers charges on a liquid ETF.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jhO1eFXCDH1kk6FCcAEtAQ.png" /><figcaption>Pair Validation Tab — Cointegration Check</figcaption></figure><p>The preflight runs the Engle-Granger cointegration test on the formation window. For GLD/GDX the verdict is a clean fail, a p-value of 0.88, nowhere near the 0.05 it would need to pass, and a half-life of “n/a” because the spread isn’t mean-reverting at all. <br>The number that should give you pause is the last one: GLD and GDX still have a return correlation of 0.80, yet they don’t cointegrate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MXBl9psHw5Py6DOXXMGGDw.png" /><figcaption>Pair Validation Tab — Spread and z-score</figcaption></figure><p>The top chart shows why GLD/GDX is no longer a valid pair.</p><p>The spread (fixed on the shaded formation window) drifts steadily out of sample, from near zero to below −$150, and never returns to its mean. A cointegrating pair would keep crossing the black line on both sides.</p><p>The z-score below does pierce the ±2 band a few times, so you’d expect the strategy to trade, yet the backtest reports none.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4M8IJucw08Gzk8YHcFFklA.png" /><figcaption>Pair Validation Tab — Rolling ADF p-value</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PNBlnJmh9X85d7wQS6hrww.png" /><figcaption>Pair Validation Tab — Backtest Results</figcaption></figure><p>The reason there are no trades is a second, ongoing check suggested by Claude: the cointegration-breakdown filter. The one-time preflight only tests cointegration on the formation window; this filter keeps re-testing as the strategy runs. On a trailing window of the spread (120 days, re-checked roughly monthly), it periodically runs an Augmented Dickey-Fuller (ADF) test to check whether a series is stationary (mean-reverting) or just wandering. If the spread fails that test (p-value stays above 0.10), the strategy marks the relationship as broken and refuses to open a position. GLD/GDX’s spread fails on almost every out-of-sample bar (92% in this run), so nearly every entry is vetoed.</p><p>In plain terms, GDX (gold miners) outran GLD (bullion) by more than the fitted 5.852 ratio allowed. The spread falling past −$150 shows a real divergence.</p><p>Readers are encouraged to test different entry signals, and not to skip a stretch of paper trading if a backtest looks good.</p><h3>The Correlation vs Cointegration Panel</h3><h4>What to do</h4><p>Start on the Correlations tab. Enter the tickers you want to compare (here, KO and PEP like example 7.3), leave the basis on daily returns, choose a 2-year window, and click Calculate Correlations. The app loads the price history once (501 trading days for this pair) and reuses it for every chart below, so no extra data is fetched.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CJJOAGRhvZs8gpwwXWP_lw.png" /><figcaption>Correlation Tab — Input</figcaption></figure><h4>The Results</h4><p>The panel puts the two ideas side by side. KO and PEP correlate at 0.589, but they fail the cointegration test with a p-value of 0.7885, nowhere near the 0.05 needed to pass, and the half-life is blank because the spread never mean-reverts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PULQTurtOzWDmID54LxouQ.png" /><figcaption>Correlation, Cointegration p-value and Half-life</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kG9-oRQ280yj1to7eSFWXQ.png" /><figcaption>Correlation vs Cointegration — Scatter Plot</figcaption></figure><p>The scatter plot makes the same point visually. Its horizontal axis is return correlation and its vertical axis is the cointegration p-value, with a dashed line marking the 0.05 threshold near the bottom. KO/PEP sits to the right of center (positive correlation around 0.59) but high up (p around 0.79), so it lands in the red, well above the line. If correlation implied cointegration, the dot would sit low, near the threshold. It clearly does not.</p><p><strong>Note:</strong> App users can rerun the test with more than two tickers (for example GLD, GDX, SLV).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sSNQMXLU1Q4-9RPS_pKU0Q.png" /><figcaption>Rolling Correlation</figcaption></figure><p>Across two years, the rolling-correlation chart shows that KO and PEP stay positively correlated, mostly between 0.45 and 0.75. They genuinely move together day to day, which is what tempts a correlation-based screen. Cointegration asks a different question, whether the gap between them stays bounded and reverts, and on that question the pair fails.</p><p>I’ve often been doubtful of traders selling book describing their strategies. I didn’t expect Mr Chan to share so many actionable ideas. We now have the ability to make learning even more fun and interactive than before. With AI, we can test ideas with different timeframes, keep the good ones and quickly dismiss the ones that are no longer valid or incorrect before backtesting.</p><p>I wish Claude existed when I was completing M.Sc. in Mathematical Finance. Learning theorems and writing proofs was fun too but there is something exhilarating about practicing and failing (though frustrating at times). I hope today’s graduate students can take advantage of the technology at their disposal and explore beyond theorems and proofs.</p><p>If there’s a single takeaway, it’s to stay curious and keep testing. Buy <a href="https://a.co/d/0cFLCZpr"><em>Quantitative Trading</em> (2nd Edition)</a> or any other quant trading book you wish, but do not let it collect dust on your bookshelf.</p><p><em>For the record: I have no affiliation with Ernest Chan or Anthropic, and nothing here is sponsored.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=736d585818c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning By Doing: Building My First Quant Trading Platform]]></title>
            <link>https://medium.com/@marie.imokoyende/learning-by-doing-building-my-first-quant-trading-platform-9316fba8ecb0?source=rss-79dc1431a34c------2</link>
            <guid isPermaLink="false">https://medium.com/p/9316fba8ecb0</guid>
            <category><![CDATA[stock-trading]]></category>
            <category><![CDATA[quantitative-trading]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[Marie Imokoyende]]></dc:creator>
            <pubDate>Sun, 05 Jul 2026 22:52:43 GMT</pubDate>
            <atom:updated>2026-07-05T22:52:43.380Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Creating a Streamlit app for US stocks and ETFs, powered by Claude, Massive/Polygon, and the Wall Street Journal.</em></p><p>Every summer, I read a book on trading or on machine learning and AI applied to finance.</p><p>This year, I chose <a href="https://a.co/d/0cFLCZpr"><em>Quantitative Trading</em> (2nd Edition)</a> by Ernest P. Chan. I first read the 1st edition back in 2014 — borrowed from the Concordia University library — and it flipped a switch in my head. That’s where my passion for researching and understanding quantitative trading really began.</p><p>This time, I wanted to learn differently. Instead of working through concepts in Jupyter notebooks and then archiving them, I decided to build an app that implements the ideas I read about over the summer — something interactive, and more playful.</p><p>I’m a convinced Claude user, and I wanted an app I could run locally on my own machine — one that would grow and evolve with how I use it, taking advantage of all my subscriptions.</p><p>As the project grows, I plan to add machine learning and deep learning components. Being a gamer, I already have an NVIDIA GPU on hand, so I might as well put it to work.</p><p>There was another motivation, too. Instead of constantly subscribing and unsubscribing to various online tools, I wondered whether I could make better use of the subscriptions I already pay for — and build something tailored to my own needs.</p><h4>What I set out to build</h4><p>At this initial stage, my goal was a focused one: an app for US stocks and ETFs that leverages three things I already have — Claude, my Massive (Polygon) market-data subscription, and my Wall Street Journal subscription.</p><p>I wanted to build:</p><ul><li><strong>A Dashboard</strong> for any given stock displaying price and volume charts, price and return distributions, and key statistics.</li><li><strong>A Market Intelligence page</strong> that pulls news from the WSJ first, then from other web sources, generates an AI market assessment, and allows the user to ask follow-up questions about a stock.</li><li><strong>An Asset Correlation page</strong> that computes correlations, plots rolling correlations and cumulative returns, and generates an AI correlation analysis written by Claude.</li></ul><h4>Why I made it public</h4><p>I initially built a local app and was happy with the results, but I decided to publish a public version — along with the GitHub code — so anyone could try it. I also think it’s valuable to build a portfolio of projects showcasing how I use generative-AI tools on subjects I’m interested in.</p><h3>Using the public app</h3><p>To use the online version fully, you’ll need <strong>your own Claude API key</strong> for the AI-powered parts of the app — the market assessment, the chatbot, and the correlation analysis. It’s not required for everything else (charts, data, news).</p><p>It is also possible to run the app in <strong>Demo mode</strong> without entering any Claude API key. You’ll see sample AI output, so you can get a feel for each feature at no cost.</p><p><strong>App link</strong>: <a href="https://quant-trading-app.streamlit.app/">https://quant-trading-app.streamlit.app/</a></p><h4>Running it locally</h4><p>If you’d rather run it yourself, you can clone the code from my <a href="https://github.com/MarieImokoyende/quant-trading-app/tree/main">GitHub repo</a> and create a .env file with your API keys:</p><ul><li>POLYGON_API_KEY — your Massive/Polygon market-data key</li><li>FINNHUB_API_KEY — for company news</li><li>ANTHROPIC_API_KEY — your Claude key</li><li>BRAVE_API_KEY — for web search (see the note below)</li></ul><p>For web search, I used the <strong>Brave Search API</strong> in the online version, because I kept running into timeout errors with DuckDuckGo on Streamlit. Running locally, DuckDuckGo works fine. As an exercise, users can modify the load_wsj_news and web_search functions to use DuckDuckGo instead — and skip the Brave key entirely. <em>(Hint: ask Claude</em> 😉)</p><h4>How to use the app</h4><p>If you’ve entered your Claude API key:</p><ol><li>Choose the <strong>Claude model</strong> and the <strong>Max Tokens</strong> value in the sidebar.</li><li>Enter a <strong>US ticker</strong></li><li>A <strong>cost counter</strong> keeps a running tally of the amount spent during the session.</li></ol><p>With the ticker entered, you’re ready to explore the app’s three pages: <strong>Dashboard</strong>, <strong>Market Intelligence</strong>, and <strong>Correlations</strong>.</p><h4>Dashboard</h4><p>The Dashboard is where every ticker analysis begins. You get:</p><ol><li><strong>A full snapshot</strong>: current price and the day’s move, 20- and 50-day moving averages, and a <em>historical percentile</em>.</li><li><strong>An interactive candlestick chart</strong> (selectable from one month to two years, with moving-average overlays) and a volume chart.</li><li><strong>A mean reversion signals section</strong> that turns a Z-score against the 20-day average into a plain-language buy/sell/neutral read.</li><li><strong>Histograms of the price and daily-return distributions.</strong></li><li><strong>Two Key Statistics tables </strong>covering everything from annualized volatility to skewness and kurtosis.</li><li><strong>The last 20 days of raw data</strong> tucked into an expander at the bottom.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*14qOxiiPYzf5M5zGelaz-Q.png" /><figcaption>Dahboard — Candlestick Chart</figcaption></figure><h4>Market Intelligence</h4><p>The Market Intelligence page is where the app gathers the news and lets Claude reason over it. It contains:</p><ol><li><strong>A combined news feed</strong> from three sources in a deliberate order: the Wall Street Journal first — the subscription that started this whole idea — then Finnhub’s market news, then live web results from Brave Search, with a search box you can point at any query rather than the default “{ticker} stock market news today.”</li><li><strong>The AI Market Assessment</strong> that feeds the quantitative market data plus all three news sources to Claude and returns a current assessment, a news and intelligence summary and trend analysis. <strong>One practical tip:</strong> the assessment is only as good as the material you give it, so before generating one, make sure the news feed is actually populated, ideally with results showing in at least two of the three sections (WSJ, Finnhub, and Web Search). The WSJ and Finnhub feeds load on their own, but coverage varies by ticker, so if one comes up empty, run the Web Search (or refresh it with your own query) to make sure Claude has enough current material to work with.</li><li><strong>The Ask the AI Analyst box</strong> where you can ask anything about the stock, the markets, or world events. It runs a fresh web search on your question, shows you the sources it used, and has Claude answer from those live results grounded in the stock’s market context. A question like “how does this week’s news affect the stock?” comes back tied to recent news headlines rather than just the stock price data.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PpamuIn9020lLw-rAktQKg.png" /><figcaption>AI Market Assessment</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WRm7PooGMJPG5kPDZWpO0g.png" /><figcaption>Ask the AI Analyst</figcaption></figure><h4>Correlations</h4><p>The Correlations page steps back from a single stock to look at how a basket of assets moves together. You can enter up to ten tickers, choose whether to measure correlation on daily, weekly, or monthly returns, pick a lookback from six months to two years, and hit Calculate. It will generate:</p><ol><li><strong>An annotated correlation heatmap</strong>, with a collapsible table of the exact figures.</li><li><strong>A Key Insights panel</strong> showing the three lowest-correlation pairs and the three highest.</li><li><strong>A rolling-correlation chart</strong> that tracks how each asset’s correlation to a chosen base ticker drifts over time (for spotting correlations that tighten under stress).</li><li><strong>A cumulative-returns comparison chart</strong> to visualize how the assets’ returns stack up against one another over time.</li><li><strong>A return scatter plot</strong> between any two tickers that fits a trend line and reports the pair’s correlation, beta, and R².</li><li><strong>A return summary table</strong> that rounds out the numbers with annualized return, volatility, Sharpe ratio, skewness, and kurtosis for each ticker.</li><li><strong>An AI Correlation Analysis </strong>button that hands the whole picture (the matrix, the ranked pairs, and the return summary) to Claude, which comes back with a written read on diversification, portfolio construction, and the risks worth watching.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2QIWhoxWF66F2Rf5zKFr8Q.png" /><figcaption>Asset Correlation Analysis</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FA_WDV_dWJh-_Z_niAS3hQ.png" /><figcaption>Correlation Matrix</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gXG3ukgkZ-bkYsWncXlTiA.png" /><figcaption>Return Scatter Plot</figcaption></figure><h4>Conclusion</h4><p>Building this app taught me that I could put together my own custom tools, and do it quickly. Most of the work happened over the Canada Day/Staycation weekend.</p><p>I picked up most of my coding during my maths and statistics studies at university rather than through any formal computer-science training, so I expected the process to be more difficult. I was pleasantly surprised by how smoothly the app came together.</p><p>This experience has changed how I plan to learn going forward. Rather than getting stuck on implementation, I can focus first on the concepts and the mathematics, then lean on generative AI to turn those ideas into working code, fast. For someone whose instinct is to learn by doing, that’s a real shift.</p><p>As for where this app goes next: I’d like to test other models, such as Grok and Gemini, alongside Claude, and start adding machine learning/deep learning tools of my own. This first version was about getting the foundation right. The next ones are where it gets interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9316fba8ecb0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Variational Autoencoder In Finance]]></title>
            <link>https://medium.com/data-science/variational-autoencoder-in-finance-53ee5eb9ed98?source=rss-79dc1431a34c------2</link>
            <guid isPermaLink="false">https://medium.com/p/53ee5eb9ed98</guid>
            <category><![CDATA[index]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[variational-autoencoder]]></category>
            <category><![CDATA[dimensionality-reduction]]></category>
            <category><![CDATA[exchange-traded-funds]]></category>
            <dc:creator><![CDATA[Marie Imokoyende]]></dc:creator>
            <pubDate>Mon, 15 Apr 2019 01:13:22 GMT</pubDate>
            <atom:updated>2019-04-25T12:34:33.116Z</atom:updated>
            <content:encoded><![CDATA[<h4>Dimensionality Reduction of Financial Time Series and Index Construction</h4><p>This article explores the use of a variational autoencoder to reduce the dimensions of financial time series with Keras and Python. We will further detect similarities between financial instruments in different markets and will use the results obtained to construct a custom index.</p><p><strong>Disclaimer:</strong> The research presented in this article comes from our Winter 2019 Term Project for the <a href="https://learn.utoronto.ca/programs-courses/courses/3546-deep-learning">Deep Learning course</a> at the University of Toronto School of Continuing Studies. It was done in collaboration with <a href="https://www.linkedin.com/in/humbertors">Humberto Ribeiro de Souza</a>. The concepts and ideas are our own. We are in no way representing our current or previous employers.</p><h4>Part 1: Dimensionality Reduction Using a Variational Autoencoder</h4><p>In this section, we will discuss:</p><ul><li>Creating the geometric moving average dataset</li><li>Augmenting the data with stochastic simulation</li><li>Building the variational autoencoder model</li><li>Obtaining the predictions.</li></ul><p><strong>Creating The Geometric Moving Average Dataset</strong></p><p>In order to compare time series of various price ranges, we have chosen to compute geometric moving average time series of returns defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*ySf1S92Gz5_9fejH1Qdl7A.png" /></figure><p>We chose <strong>d=5</strong>, as it represents a typical trading week of 5 business days.</p><p>The dataset used in this article contains 423 geometric moving average time series for a period going from January 4th, 2016 to March 1st, 2019.</p><p>Readers can follow the steps described in the <a href="https://github.com/MarieImokoyende/vae-dimRedFinance/blob/master/3546%20Deep%20Learning%20Project%20-%20Data%20Treatment.ipynb">data treatment notebook</a> to build their own dataset. It should be similar to this one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*flzgTrvQtJiuD-cfpyBbIQ.png" /></figure><p>Results can be verified by plotting some sample stock price time series and their geometric moving average curves:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*_ULtacWLxbysYlxekokA8A.png" /></figure><p>Then, the dataframe just built can be divided in two time periods of equal length, transposing the one for the first period only. Period 1 goes from January 12th, 2016 to August 4th, 2017. Period 2, goes from August 7th, 2017 to March 1st, 2019.</p><p>We will only use the period 1 data to obtain predictions.</p><pre># Divide in two<br>geoMA_5d_stocks_p1 = geoMA_5d_stocks.head(int(len(geoMA_5d_stocks)/2))<br>geoMA_5d_stocks_p2 = geoMA_5d_stocks.tail(int(len(geoMA_5d_stocks)/2))</pre><pre># Transpose the dataframe for period 1<br>geoMA_5d_stocks_p1_T = geoMA_5d_stocks_p1.T</pre><p>We transpose the dataframe so that each row will represent a time series for a given stock:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*Nxie3BBk7Rp1srTahyKOZg.png" /></figure><p><strong>Augmenting the data with stochastic simulation</strong></p><p>We will use stochastic simulation to generate synthetic geometric moving average curves. The objective is not to precisely model returns but to obtain curves with a behavior similar to real data. By training the model with only simulated curves we can keep the real data to obtain the predictions.</p><p>The synthetic curves are generated using Geometric Brownian Motion. We followed the steps below:</p><ul><li>Using the first period dataframe, select 100 tickers randomly</li><li>For each ticker selected, calculate a vector of log returns such that:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/1*gMXbZsEthCynJR4uv5fPzQ.png" /></figure><ul><li>Then for each ticker selected, we will generate 100 paths such that:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*yCOOkI-UDeuF_ed8irxH2A.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97669a2b3a7a711270ed880ecb689b3e/href">https://medium.com/media/97669a2b3a7a711270ed880ecb689b3e/href</a></iframe><p>Here is a sample of a simulated curve and a real curve:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*vCdx_BQPcLt4dzpo2WlZ-Q.png" /></figure><p>We have expanded a dataset of 423 time series to 100*100 = 10,000 new time series similar (but not equal) to the stock dataset.</p><p>This will allow us to keep the actual stock dataset universe for predictions and not even have to use it for the validation.</p><p>Before building the VAE model, create the training and test sets (using a 80%-20% ratio):</p><pre># Shuffle the generated curves<br>shuffled_array = np.random.permutation(sim_paths_matrix)</pre><pre># Split the simulated time series into a training and test set<br>x_train = shuffled_array[0:8000]<br>x_test = shuffled_array[8000:]</pre><p>Readers should also note that there is no need to remove the seasonality and trend of the time series before training the model.</p><p><strong>Building the Variational Autoencoder (VAE) Model</strong></p><p>We will use a variational autoencoder to reduce the dimensions of a time series vector with 388 items to a two-dimensional point.</p><p>Autoencoders are unsupervised algorithms used to compress data. They are built with an encoder, a decoder and a loss function to measure the information loss between the compressed and decompressed data representations.</p><p>Our goal is not to write yet another autoencoder article. Readers who are not familiar with autoencoders can read more on the <a href="https://blog.keras.io/building-autoencoders-in-keras.html">Keras Blog</a> and the <a href="https://arxiv.org/abs/1312.6114"><em>Auto-Encoding Variational Bayes</em> paper</a> by Diederik Kingma and Max Welling.</p><p>We will use a simple VAE architecture similar to the one described in the <a href="https://blog.keras.io/building-autoencoders-in-keras.html">Keras blog</a>.</p><p>The encoder model has:</p><ol><li>One input vector of length 388</li><li>One intermediate layer of length 300 with a rectified linear unit (ReLu) activation function</li><li>One encoder with two dimensions.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*owBoHTwS4hf6ppnLnKxFYQ.png" /><figcaption>Encoder Model Summary</figcaption></figure><p>The decoded model has:</p><ol><li>One input vector of two dimensions (sampled from the latent variables)</li><li>One intermediate layer of length 300 with a rectified linear unit (ReLu) activation function</li><li>The decoded vector of length 388 with a sigmoid activation function.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*q-vfTpx2I0I3ZXwcV6BbsA.png" /><figcaption>Decoder Model Summary</figcaption></figure><p>The code below is adapted from <a href="https://github.com/keras-team/keras/blob/master/examples/variational_autoencoder.py">variational_autoencoder.py</a> on the Keras team Github. It is used to build and train the VAE model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf8a2a9eccbbe5e468f1fe3509eaa05b/href">https://medium.com/media/bf8a2a9eccbbe5e468f1fe3509eaa05b/href</a></iframe><p>After training, we plot the training and validation loss curves:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*2uCti-QB2mRHPimS9qk_3Q.png" /></figure><p><strong>Obtaining the Predictions</strong></p><p>We will only use the encoder to obtain the predictions. We will use a matrix of real values including both the stock dataset and one or multiple time series of interest.</p><p>In our project, we tested a stock dataset against a front month futures contract listed in another country and in a different currency.</p><pre># Obtaining the predictions:<br>encoded_p1 = encoder.predict(matrix_to_test_p1, batch_size=batch_size)</pre><pre># Convert the predictions into a dataframe<br>encoded_p1_df = pd.DataFrame(data = encoded_p1, columns = [&#39;x&#39;,&#39;y&#39;], index = dataframe_to_test_p1.T.index)</pre><p>We obtained the following results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*TzKN3rvmQUw8C7nCzo61mA.png" /></figure><p>Before plotting the results, we have to:</p><ol><li>Calculate the distance between the futures contract point and all the other stocks in the dataframe</li><li>Select the 50 pints closest to the futures contract</li></ol><pre># Calculate the distances between the futures contract point and all other points in the stocks dataset</pre><pre>ref_point = encoded_p1_df.loc[&#39;Futures&#39;].values<br>encoded_p1_df[&#39;Distance&#39;] = scipy.spatial.distance.cdist([ref_point], encoded_p1_df, metric=&#39;euclidean&#39;)[0]</pre><pre># Get the 50 closest points:<br>closest_points = encoded_p1_df.sort_values(&#39;Distance&#39;, ascending = True)<br>closest_points_top50 = closest_points.head(51)[1:] #We take head(51), because the Futures reference point is the first entry<br>closest_points_top50[&#39;Ticker&#39;] = closest_points_top50.index</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/1*iU3IJ7kTURownFNRkvPslQ.png" /></figure><p>We can now plot the results obtained to visualize the closest 50 stocks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YxBrWNfsPnJy23SoX0DpEg.png" /></figure><p>We’ve done our analysis for a futures contract listed in another country. However it is possible to follow the same steps in Part 1 for stocks from the same exchange.</p><h4>Part 2: Index Construction</h4><p>Let’s use the results obtained in Part 1 to create an index.</p><p>Due to the randomness of the VAE model, we will not obtain the same exact list of top 50 stocks on each run. To get a fair representation of the closest 50 points, we will run the VAE model 10 times (re-initializing and retraining it on each run). Then we will take the 50 closest points found on each run to create a dataframe <strong><em>closest_points_df</em></strong> dataframe of length 500.</p><p>Once the <strong><em>closest_points_df</em></strong> dataframe is built:</p><ol><li>Sort the points by distance</li><li>Drop the duplicate tickers, keeping only the first occurrence</li></ol><pre>sorted_by_dist = results_df.sort_values(&#39;Distance&#39;, ascending = True)</pre><pre>sorted_by_dist.drop_duplicates(subset=&#39;Ticker&#39;, keep=&#39;first&#39;, inplace = True)</pre><p>After dropping the duplicates, we will only keep the 50 closest points.</p><p><strong>Compute the weights of each stock</strong></p><p>In index construction, stock weights are calculated by using different methodologies such as market capitalization or stock prices.</p><p>Instead, we will calculate the weight of each stock such that the points closest to the futures contract point will get a higher weight than the ones further from it.</p><p><strong>With non-anonymized stock data, it is important to filter the results obtained</strong> <strong>before computing the stock weights. </strong>Outliers should be removed and the market capitalization range should be refined.</p><pre># Calculate the weights<br>top50 = sorted_by_dist.head(50).copy() # Keep the closest 50 points<br>top50[&#39;Weight&#39;] = (1/top50[&#39;Distance&#39;])/np.sum(1/top50[&#39;Distance&#39;])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*0j-zPrnignsoOuAehssdCw.png" /><figcaption>Sample of weights calculated</figcaption></figure><p><strong>Compute the number of shares of each stock</strong></p><p>After computing the weights, we calculate the number of shares of each stock in our custom index. We need to:</p><ul><li>Get the price of each stock on January 4th, 2016 (the first day of period 1)</li><li>Define the net assets amount</li><li>Compute the number of shares</li></ul><pre>#Get the stock prices on January 4th 2016</pre><pre>jan4_2016_stockPrice = np.zeros(len(stock_data_top50.columns))<br>for i in range(len(jan4_2016_stockPrice)):<br>  if stock_data_top50.columns[i] == top50[&#39;Ticker&#39;].iloc[i]:<br>    jan4_2016_stockPrice[i] = stock_data_top50[stock_data_top50.columns[i]].iloc[0]</pre><pre>top50[&#39;Price Jan4_2016&#39;] = jan4_2016_stockPrice</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*1F4nMyZXu_8qjDqFdPrrhg.png" /><figcaption>We add a column for the stock prices on January 4th, 2016</figcaption></figure><pre># We compute the number of shares</pre><pre>net_assets = 10000000 # We chose net assets = 10 million (in the currency of the stock market)</pre><pre>numShares = np.zeros(len(stock_data_top50.columns))<br>for i in range(len(jan4_2016_stockPrice)):<br>  if stock_data_top50.columns[i] == top50[&#39;Ticker&#39;].iloc[i]:<br>    numShares[i] = int(net_assets*top50[&#39;Weight&#39;].iloc[i]/top50[&#39;Price Jan4_2016&#39;].iloc[i])<br>    <br>top50[&#39;numShares&#39;] = numShares</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*CMEUiZrfDP1EKYjHyRyzcg.png" /><figcaption>We add a column for the number of shares</figcaption></figure><p><strong>Construct the index</strong></p><p>To build the index, we will use the <a href="https://en.wikipedia.org/wiki/Price_index#Paasche_and_Laspeyres_price_indices">Laspeyres index</a> computed as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*w3InDrjDWlhbFDvIuRzH9w.png" /></figure><pre>stock_index = np.zeros(len(stock_data_top50))</pre><pre>for i in range(len(stock_data_top50)):<br>  sum_num = 0<br>  sum_denom = 0<br>  for j in range(len(stock_data_top50.columns)):<br>    sum_num = sum_num + stock_data_top50[stock_data_top50.columns[j]].iloc[i]*top50[&#39;numShares&#39;].iloc[j]<br>    sum_denom = sum_denom + stock_data_top50[stock_data_top50.columns[j]].iloc[0]*top50[&#39;numShares&#39;].iloc[j]<br>  stock_index[i] = sum_num /sum_denom</pre><pre># We arbitrarily start the index at 100<br>stock_index_df = pd.DataFrame(stock_index*100, columns =  [&#39;stock_index&#39;], index = stock_data_top50.index)</pre><p>We plot the custom index obtained:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*LwgjE-ekUf_GZ40jaRF14w.png" /></figure><p><strong>Compare our custom index with the futures time series</strong></p><p>We have to scale the futures price data in order to plot it in the same graph as our custom index. To do so we have to:</p><ul><li>Calculate the daily percentage change of the futures price data</li><li>Set S_0 = 100</li></ul><pre># Calculate the percentage change<br>futures_data_stock_data_pct_change = futures_data_stock_data.pct_change()<br>futures_data_stock_data_pct_change.dropna(inplace = True)</pre><pre># Scale the time series<br>futures_theoretical = np.zeros(len(stock_index_df))<br>futures_theoretical[0] = stock_index_df.iloc[0]<br>for i in range(len(futures_theoretical)-1):<br>  futures_theoretical[i+1] = (1+futures_data_stock_data_pct_change.iloc[i])*futures_theoretical[i]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*e_dJY5MNVrB_vPfHS8sajQ.png" /></figure><p>We now plot both curves in the same graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*alKWthhVU6oN5-UKDPxZaw.png" /></figure><p>Our index has mostly the same trend as the reference futures time series except for the second half of 2018. Because we use anonymized data, we did not filter the stocks for outliers and market capitalization limits. Furthermore there was no re-balancing throughout the two time periods observed and we ignored distributions.</p><p>It is absolutely possible for the custom index to beat the futures index if tickers are identified and outliers are removed.</p><p>We encourage our readers to take advantage of the free GPU instances available online to create their own indices. It was a fun experiment for us and we discovered some interesting stock patterns.</p><p>Feel free to download the two notebooks available on GitHub:</p><ol><li><a href="https://github.com/MarieImokoyende/vae-dimRedFinance/blob/master/3546%20Deep%20Learning%20Project%20-%20Data%20Treatment.ipynb">3546 Deep Learning Project — Data Treatment.ipynb</a></li><li><a href="https://github.com/MarieImokoyende/vae-dimRedFinance/blob/master/3546%20Deep%20Learning%20Project%20-%20VAE%20_%20Index%20Construction.ipynb">3546 Deep Learning Project — VAE &amp; Index Construction.ipynb</a></li></ol><p><strong>Conclusion</strong></p><p>The use of variational autoencoders can speed up the development of new indices in foreign stock markets, even if analysts are unfamiliar with them. Furthermore, niche indices or portfolios could be created to match customers interests.</p><p>While this method can be used to create ETFs, we believe that it can also create new investment possibilities for Direct Indexing and Robo Advisors firms worldwide.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53ee5eb9ed98" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/variational-autoencoder-in-finance-53ee5eb9ed98">Variational Autoencoder In Finance</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>