<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Roshan Adusumilli on Medium]]></title>
        <description><![CDATA[Stories by Roshan Adusumilli on Medium]]></description>
        <link>https://medium.com/@roshan.adusumilli?source=rss-4b91022e00a0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*15Ip7Mr5WwjL3ST1eaXhog.png</url>
            <title>Stories by Roshan Adusumilli on Medium</title>
            <link>https://medium.com/@roshan.adusumilli?source=rss-4b91022e00a0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:25:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@roshan.adusumilli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Penny Saved is A Penny Earned]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@roshan.adusumilli/a-penny-saved-is-a-penny-earned-72ec2145b6d0?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*skINHS-mNCH55sOiu8KBPA.jpeg" width="4990"></a></p><p class="medium-feed-snippet">The comprehensive guide to financial inclusion</p><p class="medium-feed-link"><a href="https://medium.com/@roshan.adusumilli/a-penny-saved-is-a-penny-earned-72ec2145b6d0?source=rss-4b91022e00a0------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@roshan.adusumilli/a-penny-saved-is-a-penny-earned-72ec2145b6d0?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/72ec2145b6d0</guid>
            <category><![CDATA[financial-inclusion]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[social-issues]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[banking]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Fri, 15 Jan 2021 23:29:54 GMT</pubDate>
            <atom:updated>2021-01-15T23:29:54.270Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to become an Übermensch]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@roshan.adusumilli/how-to-become-an-%C3%BCbermensch-311ab8281276?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*gsC_gNhCqbsVK-hclCxixg.jpeg" width="3872"></a></p><p class="medium-feed-snippet">(German philosopher Nietzsche&#x2019;s concept of the superman and how you can apply it to your life)</p><p class="medium-feed-link"><a href="https://medium.com/@roshan.adusumilli/how-to-become-an-%C3%BCbermensch-311ab8281276?source=rss-4b91022e00a0------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@roshan.adusumilli/how-to-become-an-%C3%BCbermensch-311ab8281276?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/311ab8281276</guid>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[self-awareness]]></category>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[nietzsche]]></category>
            <category><![CDATA[life-lessons]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Wed, 07 Oct 2020 12:02:41 GMT</pubDate>
            <atom:updated>2020-10-07T12:02:41.382Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How This Startup Raised $20 million in Series B Funding to Scale Its Location Data Infrastructure]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://ehandbook.com/how-this-startup-raised-20-million-in-series-b-funding-to-scale-its-location-data-infrastructure-5340c2761330?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/1180/1*Aq_w7zawjAwHKAevrgi3fw.png" width="1180"></a></p><p class="medium-feed-snippet">Interview with Nick Patrick, CEO of Radar</p><p class="medium-feed-link"><a href="https://ehandbook.com/how-this-startup-raised-20-million-in-series-b-funding-to-scale-its-location-data-infrastructure-5340c2761330?source=rss-4b91022e00a0------2">Continue reading on Entrepreneurship Handbook »</a></p></div>]]></description>
            <link>https://ehandbook.com/how-this-startup-raised-20-million-in-series-b-funding-to-scale-its-location-data-infrastructure-5340c2761330?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/5340c2761330</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Tue, 30 Jun 2020 18:08:21 GMT</pubDate>
            <atom:updated>2020-06-30T19:29:15.664Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Systematic Bias in Artificial Intelligence]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/systematic-bias-in-artificial-intelligence-8698ffa20d57?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*yPZap7X4nhqiLrkQZe-6wQ.jpeg" width="4032"></a></p><p class="medium-feed-snippet">Shining a light on one of the biggest problems of tomorrow</p><p class="medium-feed-link"><a href="https://medium.com/data-science/systematic-bias-in-artificial-intelligence-8698ffa20d57?source=rss-4b91022e00a0------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/systematic-bias-in-artificial-intelligence-8698ffa20d57?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/8698ffa20d57</guid>
            <category><![CDATA[bias]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Fri, 19 Jun 2020 14:08:27 GMT</pubDate>
            <atom:updated>2020-06-19T14:12:20.001Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Artificial Intelligence in the Healthcare Industry]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/artificial-intelligence-in-the-healthcare-industry-3aea245bce93?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*QJiMhpEHfanohWBr8DlJPQ.jpeg" width="3000"></a></p><p class="medium-feed-snippet">How AI can lead the future of diagnosis and drug discovery</p><p class="medium-feed-link"><a href="https://medium.com/data-science/artificial-intelligence-in-the-healthcare-industry-3aea245bce93?source=rss-4b91022e00a0------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/artificial-intelligence-in-the-healthcare-industry-3aea245bce93?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3aea245bce93</guid>
            <category><![CDATA[research]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[innovation]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Mon, 08 Jun 2020 00:43:51 GMT</pubDate>
            <atom:updated>2020-06-16T00:07:05.489Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How are Americans reacting to Covid-19?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/how-are-americans-reacting-to-covid-19-700eb4d5b597?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/620/1*G7hhNbwwhrbLlfTw5IPSSg.jpeg" width="620"></a></p><p class="medium-feed-snippet">Using Twitter and sentiment analysis to  answer the question</p><p class="medium-feed-link"><a href="https://medium.com/data-science/how-are-americans-reacting-to-covid-19-700eb4d5b597?source=rss-4b91022e00a0------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/how-are-americans-reacting-to-covid-19-700eb4d5b597?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/700eb4d5b597</guid>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Sun, 24 May 2020 16:54:28 GMT</pubDate>
            <atom:updated>2020-05-24T16:54:28.358Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Automation’s Impact on Banking]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.datadriveninvestor.com/automations-impact-on-banking-ceeb238d08f9?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*94dohr09iYQn9BtdDMLbxQ.jpeg" width="5184"></a></p><p class="medium-feed-snippet">The death or future of jobs in the industry?</p><p class="medium-feed-link"><a href="https://medium.datadriveninvestor.com/automations-impact-on-banking-ceeb238d08f9?source=rss-4b91022e00a0------2">Continue reading on DataDrivenInvestor »</a></p></div>]]></description>
            <link>https://medium.datadriveninvestor.com/automations-impact-on-banking-ceeb238d08f9?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/ceeb238d08f9</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[banking]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Wed, 06 May 2020 13:11:01 GMT</pubDate>
            <atom:updated>2020-05-06T13:11:01.247Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The (Artificially) Intelligent Investor]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/the-artificially-intelligent-investor-379a180e199f?source=rss-4b91022e00a0------2"><img src="https://cdn-images-1.medium.com/max/2600/1*Zrnn3q4j7GyjMuZBZXTJrw.jpeg" width="5760"></a></p><p class="medium-feed-snippet">Using RNNs to generate investing advice</p><p class="medium-feed-link"><a href="https://medium.com/data-science/the-artificially-intelligent-investor-379a180e199f?source=rss-4b91022e00a0------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/the-artificially-intelligent-investor-379a180e199f?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/379a180e199f</guid>
            <category><![CDATA[ai-in-finance]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[investing]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Mon, 20 Apr 2020 14:08:07 GMT</pubDate>
            <atom:updated>2020-04-20T14:08:07.823Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Diseases: A Thing of the Past?]]></title>
            <link>https://medium.com/@roshan.adusumilli/diseases-a-thing-of-the-past-bdbd3b0fba60?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdbd3b0fba60</guid>
            <category><![CDATA[crispr]]></category>
            <category><![CDATA[medicine]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[genetics]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Fri, 13 Mar 2020 20:59:00 GMT</pubDate>
            <atom:updated>2020-03-13T20:59:29.463Z</atom:updated>
            <content:encoded><![CDATA[<h4>A quick introduction to how CRISPR-Cas9 can revolutionize medicine</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*68O338AEsXy1O9BN" /></figure><p>CRISPR-Cas9 has been hailed as the biggest biotech invention of the past decade. Simply put, it’s a technique that enables scientists to cut DNA by inserting or reordering bits of genetic code with remarkable accuracy that’s orders of magnitude greater than previous methods.</p><h3>The Technology</h3><p>Jennifer Duodena discovered the use of CRISPR-Cas9 in gene editing when she investigated the question of how bacteria fights off viruses. When enzymes produced by the bacteria fight off invading viruses and eventually kill them, remnants of the virus’s genetic code are picked up by other enzymes and stored in CRISPR (Clustered Regularly Interspaced Short Palindromic Repeats) spaces. Then, the bacteria utilizes the stored genetic information to combat future attacks from the same virus. Once another viral infection happens, bacteria produce Cas9, which are special enzymes that carry the genetic code from the previous virus. As a result, Cas9 starts cutting the virus’ DNA if the virus’ RNA matches the previously stored genetic code, preventing the virus from causing any harm. (This is similar to how vaccines work on people. They introduce small bits of the disease to help the body later combat it.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PJc_ZaG1V8HX_se1.png" /></figure><p>The real breakthrough came when Duodena’s team found that feeding artificial RNA to Cas9 would make it start cutting anything with the same genetic code. Essentially, scientists could use CRISPR/Cas9 to cut any genome however they wanted.</p><h3>Application to Fighting Disease</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hkVFBFx53pMEd1Pd.jpg" /><figcaption>Image: Javier Zarracina/Vox</figcaption></figure><p>There are several diseases that CRISPR-Cas9 technology may be able to solve through gene editing; here are two examples.</p><h4>AIDS/HIV</h4><p>Therapies cannot get completely get rid of the HIV virus since it can remain in the body in a dormant, inactive form. But, CRISPR can target the virus to prevent the replication of T cells that can’t be treated with existing treatments. Although this seems promising, scientists need to figure out how to introduce a CRISPR-Cas9 system that targets HIV-1 genes into the infected cells of patients.</p><h4>Cystic Fibrosis</h4><p>Cystic Fibrosis is a genetic disease caused by mutations in the CFTR gene. It causes severe respiratory problems, limiting the average lifespan of someone with the condition to only 40 years. CRISPR has been tested by scientists on human lung cells derived from patients with cystic fibrosis to fix the most common mutation behind the disease. Multiple different mutations can cause the condition so different CRISPR therapies must be developed. Before it can be effectively implemented, testing on humans must begin.</p><h4>Side Effects</h4><p>Unfortunately, there are serious consequences of using CRISPR technology that we must consider. Gene edits can result in changes in DNA at a farther distance from the targeted location than previously thought. Even worse, these damages aren’t detected by standard DNA tests. These changes can cause insertions/deletions that switch unwanted DNA on and off, resulting in unknown mutations and health problems for patients.</p><p>In conclusion, CRISPR is an exponential technology that can improve the quality of life for billions of people, yet its side effects must be considered before implementing it on a wide scale.</p><p><strong>Don’t leave yet</strong>!</p><p>I’m Roshan, a 16 year old passionate about emerging technologies who’s currently focusing on the intersection of artificial intelligence and finance. For a broad view of AI in finance, check out this article: <a href="https://towardsdatascience.com/artificial-intelligence-and-its-application-in-finance-9f1e0588e777">https://towardsdatascience.com/artificial-intelligence-and-its-application-in-finance-9f1e0588e777</a>.</p><p>Reach out to me on Linkedin: <a href="https://www.linkedin.com/in/roshan-adusumilli-96b104194/">https://www.linkedin.com/in/roshan-adusumilli-96b104194/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdbd3b0fba60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning for Options Trading]]></title>
            <link>https://medium.datadriveninvestor.com/reinforcement-learning-for-options-trading-765c84d0d97d?source=rss-4b91022e00a0------2</link>
            <guid isPermaLink="false">https://medium.com/p/765c84d0d97d</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-in-finance]]></category>
            <category><![CDATA[finance]]></category>
            <dc:creator><![CDATA[Roshan Adusumilli]]></dc:creator>
            <pubDate>Fri, 13 Mar 2020 20:48:34 GMT</pubDate>
            <atom:updated>2021-04-27T13:14:13.162Z</atom:updated>
            <content:encoded><![CDATA[<h4>Artificial Intelligence in Finance</h4><h4>Q Learning + Black-Scholes = Optimal Option Price</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*eGjYY9NZfT7Yy2VY3ZRt5w.png" /><figcaption>Image Credit: Financial Times</figcaption></figure><p>Artificial intelligence is making its impact on many areas of finance, particularly trading. A diverse range of artificial intelligence subfields such as deep learning, reinforcement learning, and natural language processing are currently being utilized to predict stock movements. A reinforcement learning trading agent attempts to learn stock prices through trial and error. By combining Q learning, a type of reinforcement learning algorithm, with the Black-Scholes model, a traditional model for option pricing, we can create a Q Learning Black Scholes (QLBS) model to determine optimal option prices.</p><p>In this article, I’ll go over options, the Black-Scholes model, and Q learning before showing the implementation of a Q Learning Black-Scholes model for an European put option.</p><p><em>Note: click </em><a href="#aced"><em>here</em></a><em> if you want to go straight to the implementation of the QLBS model (link doesn’t work in mobile app)</em></p><h3>Options Explained</h3><p>Options are a type of derivative security, meaning their value depends on the price of some other asset such as stock or commodities. For example, an option contract for a stock usually represents 100 shares of the underlying stock. The price of the option contract is known as the premium. Essentially, the contract allows the bearer to either buy or sell an amount of the underlying asset at a pre-determined price (referred to as the strike price) at or before the contract expires. Additionally, the bearer does not have an obligation to buy or sell, so they can also let the contract expire.</p><p>The two major types of options are call options and put options; the former allows the bearer to buy the asset at a stated price and the latter allows the bearer to sell the asset at a stated price. Buyers can use call options for speculation and put options for hedging purposes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*Xm8LNLLmuPFS3lE2ACev9w.png" /><figcaption>Image credit: Gatsby</figcaption></figure><p>To understand this better, we’ll look at a real-world example. Suppose that Apple shares are trading at $200 per share. You think that shares may rise above $210 over the next month and buy a $210 call trading at $0.67 per contract. If the price rises over $210 before or on the expiration date of the contract, you buy the shares at $210 and if the shares fall or don’t rise over $210, you only lose money from the $67 premium ($0.67 x 100 shares). Conversely, let’s say you already own Apple shares and think that the price may fall, leading you to buy a $190 put trading at $0.63 per contract. If the price falls below $190 before or on the expiration date of contract, you profit by selling the shares at $190 and if the shares don’t fall, you only lose money from the $63 premium ($0.63 x 100 shares). Now that we understand options, let’s move on to the Black-Scholes Model.</p><h3>Black-Scholes Model</h3><p>The Black-Scholes equation, which is probably the most famous equation in finance, provided the first widely used model for option pricing. The current stock prices, the option’s strike price, expected interest rates, time to expiration and volatility (measure of fluctuations in price) are all used as inputs to calculate the theoretical value of options. Introduced in 1973 by economists Fischer Black, Myron Scholes and Robert Merton, the equation was so influential that it won Scholes and Merton the 1997 Nobel Prize in Economics (Black unfortunately died before he could receive the honor).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E0TRiL1bK9MIJzU_xzLxwQ.png" /><figcaption>Image Credit: KhanAcademy</figcaption></figure><p>In the image above, C is the call option price, N(d1) is the normal distribution corresponding to the call option’s delta (ratio comparing change in the price of an asset to the corresponding change in the price of its derivative), and N(d2) is the normal distribution corresponding to the probability that the call option will be exercised at expiration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*1_DIpW0DFsvnLTaqvBCF0g.png" /><figcaption>Black-Scholes for put option</figcaption></figure><p>For the sake of brevity, I’ll focus on the assumptions the Black-Scholes equation makes as well as its limitations rather than the actual math behind it. If you want a deeper understanding of Black-Scholes, watch <a href="https://www.khanacademy.org/economics-finance-domain/core-finance/derivative-securities/black-scholes/v/introduction-to-the-black-scholes-formula">this</a>.</p><h4>Assumptions</h4><ul><li>The option is European (can only be exercised at expiration, not before)</li><li>No dividends are paid out during the life of the option.</li><li>Markets are efficient (market movements cannot be predicted).</li><li>No transaction costs in buying the option.</li><li>Known and constant risk-free interest rate and volatility of underlying asset.</li><li>Normally distributed returns on the underlying asset.</li></ul><h4>Limitations</h4><ul><li>Doesn’t work for US options (can be exercised before expiration)</li><li>Volatility fluctuates in real life</li><li>Transaction costs exist</li><li>Risk free interest rate is not always constant in real life</li></ul><p>Great, now that we have an overview of Black-Scholes, we’ll go over Q learning before jumping into the implementation of the QLBS model.</p><h3>Q-Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*2rmKGjZOv5pGkLLVt-EuMA.png" /><figcaption>Image Credit: <a href="http://incompleteideas.net/book/bookdraft2017nov5.pdf">Reinforcement Learning:An Introduction</a></figcaption></figure><p>In reinforcement learning, the goal is to maximize rewards. The agent performs an action to transition from one state to the next and the action taken in each state gives the agent an reward. For example, think of a scenario with a dog and its human master. In the home (environment), the dog (agent) runs around (action) when the master commands it to sit down (state) and receives no treats (reward). In the next state if the master commands the dog to sit down, it sits down (because running did not give treats) and receives treats. Essentially, the dog learns through trial and error.</p><p><a href="https://www.datadriveninvestor.com/2019/02/19/artificial-intelligence-trends-to-watch-this-year/">5 Artificial Intelligence Trends To Watch This Year | Data Driven Investor</a></p><p>Q-learning is a reinforcement learning algorithm where the goal is to learn the optimal policy (the policy tells an agent what action to take under what circumstances). A Q-Table of dimensions states x actions has values initialized to zero. Then, the agent chooses an action, observes a reward, and enters a new state, updating Q, the “quality” of the action taken in a state at each time t. Here’s the algorithm below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nOkIED37J38hIvuc1MMihw.png" /><figcaption>Image Credit: <a href="https://en.wikipedia.org/wiki/Q-learning">Wikipedia</a></figcaption></figure><p>The learning rate, which is usually constant for all time t, determines to what extent from 0 (agent learns nothing new) to 1 (agent only considers recent information) new information overrides old information. Furthermore, the discount factor, which determines the importance of future rewards, ranges from 0 (only current rewards matter) to 1 (long term reward prioritized).</p><p>The agent can interact with the environment in two ways. One way is exploitation, where the agent uses the Q-table as reference and chooses the action that has the highest value. However, the Q-table begins with all zeros so actions sometimes have to chosen randomly. This is exploring, when the agent selects an action at random instead of choosing based on max future reward. The epsilon value sets the percent of time you want your agent to explore instead of exploit.</p><p>Finally, its time to move on to the intersection of Q Learning and Black -Scholes!</p><h3>Q-Learning + Black-Scholes</h3><p>When Q-learning and Black-Scholes are combined, our QLBS model uses trading data to autonomously learn both the optimal option price and optimal hedge. For our implementation of the model, we’ll be working with a European put option. Before implementing the QLBS model, we’ll also implement the classic Black-Scholes formula to compare the results of the two. I’ll be leaving out the code for graphs and show the graphs directly instead to avoid making this article unnecessarily long; you can still click <a href="https://colab.research.google.com/drive/1wa2puOkPuUt5LHdQBbyLJtQi6qYTYEC0">here</a> to view the full code (the project is an exercise from Coursera’s <a href="https://www.coursera.org/learn/reinforcement-learning-in-finance">Reinforcement Learning in Finance</a> course).</p><h3>Imports</h3><p>First, we make the necessary imports.</p><pre>import numpy as np<br>import pandas as pd<br>from scipy.stats import norm<br>import random<br>import time<br>import matplotlib.pyplot as plt<br>import sys</pre><h3>Monte Carlo Simulation</h3><p>After making imports, we’ll set the parameters for a Monte Carlo simulation of prices. A Monte Carlo simulation is used to model the probability of different outcomes in a process (such as stock price movement) which is unpredictable due to the presence of random variables.</p><pre>S0 = 100      # initial stock price<br>mu = 0.05     # drift<br>sigma = 0.15  # volatility<br>r = 0.03      # risk-free rate<br>M = 1         # maturity</pre><pre>T = 24        # number of time steps<br>N_MC = 10000  # number of paths</pre><pre>delta_t = M / T                # time interval<br>gamma = np.exp(- r * delta_t)  # discount factor</pre><h3>Black-Scholes Simulation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XA73NQTp_sw0q2fvTn3MCg.png" /><figcaption>Images from Coursera: Reinforcement Learning in Finance</figcaption></figure><pre>np.random.seed(42)</pre><pre># stock price<br>S = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>S.loc[:,0] = S0</pre><pre># standard normal random numbers<br>RN = pd.DataFrame(np.random.randn(N_MC,T), index=range(1, N_MC+1), columns=range(1, T+1))</pre><pre>for t in range(1, T+1):<br>    S.loc[:,t] = S.loc[:,t-1] * np.exp((mu - 1/2 * sigma**2) * delta_t + sigma * np.sqrt(delta_t) * RN.loc[:,t])</pre><pre>delta_S = S.loc[:,1:T].values - np.exp(r * delta_t) * S.loc[:,0:T-1]<br>delta_S_hat = delta_S.apply(lambda x: x - np.mean(x), axis=0)</pre><pre># state variable<br>X = - (mu - 1/2 * sigma**2) * np.arange(T+1) * delta_t + np.log(S)   # delta_t here is due to their conventions</pre><p>Here’s what some stock price and state variable paths look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*MCAc9JEgNwFmkt87QwkvuQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*Jq5sTc2lYH1NnohNU0OzpQ.png" /></figure><h3>Terminal Payoff</h3><p>The terminal payoff is the dollar amount an investor receives at expiration from following the option strategy. We’ll define a function to compute the terminal payoff of an European put option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*as8SUZqkzGgUUUb74juzbw.png" /></figure><pre>def terminal_payoff(ST, K):<br>    # ST   final stock price<br>    # K    strike<br>    payoff = max(K - ST, 0)<br>    return payoff</pre><h3>Spline Basis Functions</h3><p>A spline function is a function that is constructed piece-wise from polynomial functions. The B-spline function is the maximally differentiable interpolative basis function, which we can use for the state variable X.</p><pre>!pip install bspline<br>import bspline<br>import bspline.splinelab as splinelab</pre><pre>X_min = np.min(np.min(X))<br>X_max = np.max(np.max(X))<br>print(&#39;X.shape = &#39;, X.shape)<br>print(&#39;X_min, X_max = &#39;, X_min, X_max)</pre><pre>p = 4            # order of spline (as-is; 3 = cubic, 4: B spline?)<br>ncolloc = 12</pre><pre>tau = np.linspace(X_min,X_max,ncolloc)  <br># These are the sites to which we would like to interpolate</pre><pre># k is a knot vector that adds endpoints repeats as appropriate for a spline of order p<br># To get meaninful results, one should have ncolloc &gt;= p+1<br>k = splinelab.aptknt(tau, p) <br>                             <br># Spline basis of order p on knots k<br>basis = bspline.Bspline(k, p)        <br>        <br>f = plt.figure()<br># B   = bspline.Bspline(k, p)     # Spline basis functions <br>print(&#39;Number of points k = &#39;, len(k))<br>basis.plot()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*BtuAnofAmAPlb5xFqCQTBQ.png" /></figure><h3>Data Matrices</h3><p>Now we make data matrices with feature values; the features here are the values of basis functions at data points and the outputs are 3D arrays with dimensions num_tSteps x num_MC x num_basis.</p><pre>num_t_steps = T + 1<br>num_basis =  ncolloc # len(k) #</pre><pre>data_mat_t = np.zeros((num_t_steps, N_MC,num_basis ))<br>print(&#39;num_basis = &#39;, num_basis)<br>print(&#39;dim data_mat_t = &#39;, data_mat_t.shape)</pre><pre># fill it <br>for i in np.arange(num_t_steps):<br>    x = X.values[:,i]<br>    data_mat_t[i,:,:] = np.array([ basis(el) for el in x ])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*sWqm87e7tCy0wQ7Tzzs4DQ.png" /></figure><h3>Dynamic Programming solution for QLBS</h3><p>Quickly explained, a Markov Decision Process (MDP) contains a set of possible world states S, a set of possible actions A, A real valued reward function R(s,a), and a description T of each action’s effects in each state. The Bellman optimality equation can provide an optimal policy to solve every MDP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5NhLYkZL1VK2un9BfXP5qA.png" /></figure><h3>Define the option strike and risk aversion parameter</h3><pre>risk_lambda = 0.001 <em># risk aversion</em><br>K = 100             <em># option stike</em></pre><h3>Calculate coefficients of the optimal action</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*GIsuVjdej13yCBJrKMJUJg.png" /></figure><pre># functions to compute optimal hedges<br>def function_A_vec(t, delta_S_hat, data_mat, reg_param):<br>    X_mat = data_mat[t, :, :]<br>    num_basis_funcs = X_mat.shape[1]<br>    this_dS = delta_S_hat.loc[:, t]<br>    hat_dS2 = (this_dS ** 2).values.reshape(-1, 1)<br>    A_mat = np.dot(X_mat.T, X_mat * hat_dS2) + reg_param * np.eye(num_basis_funcs)<br>    return A_mat<br>   <br>        <br>def function_B_vec(t, Pi_hat, delta_S_hat=delta_S_hat, S=S, data_mat=data_mat_t, gamma=gamma, risk_lambda=risk_lambda):<br>    tmp = Pi_hat.loc[:,t+1] * delta_S_hat.loc[:, t]<br>    X_mat = data_mat[t, :, :]  # matrix of dimension N_MC x num_basis<br>    B_vec = np.dot(X_mat.T, tmp)<br>    return B_vec</pre><h3>Compute optimal hedge and portfolio value</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3LCer78A35oXXUhn6Bh2Q.png" /></figure><pre># portfolio value<br>Pi = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>Pi.iloc[:,-1] = S.iloc[:,-1].apply(lambda x: terminal_payoff(x, K))</pre><pre>Pi_hat = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>Pi_hat.iloc[:,-1] = Pi.iloc[:,-1] - np.mean(Pi.iloc[:,-1])</pre><pre># optimal hedge<br>a = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>a.iloc[:,-1] = 0</pre><pre>reg_param = 1e-3 # free parameter<br>for t in range(T-1, -1, -1):<br>    A_mat = function_A_vec(t, delta_S_hat, data_mat_t, reg_param)<br>    B_vec = function_B_vec(t, Pi_hat, delta_S_hat, S, data_mat_t, gamma, risk_lambda)<br>    # print (&#39;t =  A_mat.shape = B_vec.shape = &#39;, t, A_mat.shape, B_vec.shape)<br>    <br>    # coefficients for expansions of the optimal action<br>    phi = np.dot(np.linalg.inv(A_mat), B_vec)<br>    <br>    a.loc[:,t] = np.dot(data_mat_t[t,:,:],phi)<br>    Pi.loc[:,t] = gamma * (Pi.loc[:,t+1] - a.loc[:,t] * delta_S.loc[:,t])<br>    Pi_hat.loc[:,t] = Pi.loc[:,t] - np.mean(Pi.loc[:,t])<br>    <br>a = a.astype(&#39;float&#39;)<br>Pi = Pi.astype(&#39;float&#39;)<br>Pi_hat = Pi_hat.astype(&#39;float&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*4QIZ_s6kdPFpCrcJmT2D3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*FTlyTLe0nb5adSQRkKrs9A.png" /></figure><h3>Compute rewards for all paths</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YaAvAzkKwfoAEi1t_NKyQg.png" /></figure><pre># Compute rewards for all paths<br># reward function<br>R = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>R.iloc[:,-1] = - risk_lambda * np.var(Pi.iloc[:,-1])</pre><pre>for t in range(T):<br>    R.loc[1:,t] = gamma * a.loc[1:,t] * delta_S.loc[1:,t] - risk_lambda * np.var(Pi.loc[1:,t])<br>  <br># plot 10 paths<br>plt.plot(R.T.iloc[:, idx_plot])<br>plt.xlabel(&#39;Time Steps&#39;)<br>plt.title(&#39;Reward Function&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*6vGtqJDLxucNHLOAZVQjtg.png" /></figure><h3>Compute the optimal Q-function</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v84Cwm3UV_vB41DjPh2YLg.png" /></figure><pre>def function_C_vec(t, data_mat, reg_param):<br>    X_mat = data_mat[t, :, :]<br>    num_basis_funcs = X_mat.shape[1]<br>    C_mat = np.dot(X_mat.T, X_mat) + reg_param * np.eye(num_basis_funcs)<br>    return C_mat<br>   <br>def function_D_vec(t, Q, R, data_mat, gamma=gamma):<br>    X_mat = data_mat[t, :, :]<br>    D_vec = np.dot(X_mat.T, R.loc[:,t] + gamma * Q.loc[:, t+1])<br>    return D_vec</pre><p>Now we can call the functions to get the optimal Q function</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*17QPpSr756Rhz4wQC28HmA.png" /></figure><pre># Q function<br>Q = pd.DataFrame([], index=range(1, N_MC+1), columns=range(T+1))<br>Q.iloc[:,-1] = - Pi.iloc[:,-1] - risk_lambda * np.var(Pi.iloc[:,-1])</pre><pre>reg_param = 1e-3<br>for t in range(T-1, -1, -1):<br>    C_mat = function_C_vec(t,data_mat_t,reg_param)<br>    D_vec = function_D_vec(t, Q,R,data_mat_t,gamma)<br>    omega = np.dot(np.linalg.inv(C_mat), D_vec)<br>    <br>    Q.loc[:,t] = np.dot(data_mat_t[t,:,:], omega)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*JKTDlsbCjBwRmeQIqwCSRg.png" /></figure><h3>Comparison</h3><p>Let’s compare the QLBS price to the European put price given by the Black-Scholes formula.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*cfghUzNS25gE2trG3n_AzQ.png" /></figure><pre># The Black-Scholes prices<br>def bs_put(t, S0=S0, K=K, r=r, sigma=sigma, T=M):<br>    d1 = (np.log(S0/K) + (r + 1/2 * sigma**2) * (T-t)) / sigma / np.sqrt(T-t)<br>    d2 = (np.log(S0/K) + (r - 1/2 * sigma**2) * (T-t)) / sigma / np.sqrt(T-t)<br>    price = K * np.exp(-r * (T-t)) * norm.cdf(-d2) - S0 * norm.cdf(-d1)<br>    return price</pre><pre>def bs_call(t, S0=S0, K=K, r=r, sigma=sigma, T=M):<br>    d1 = (np.log(S0/K) + (r + 1/2 * sigma**2) * (T-t)) / sigma / np.sqrt(T-t)<br>    d2 = (np.log(S0/K) + (r - 1/2 * sigma**2) * (T-t)) / sigma / np.sqrt(T-t)<br>    price = S0 * norm.cdf(d1) - K * np.exp(-r * (T-t)) * norm.cdf(d2)<br>    return price</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*bweU8ulPnaRwpf-9eWeRPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*DT2jV7nD1J5-oqw4-jHgMQ.png" /></figure><p>Here, we can see that the QLBS put price was 0.4 higher than the Black-Scholes put price. Given that the option price fell heavily in the majority of paths, we can see that the QLBS put price was clearly better than the Black-Scholes put price in this instance.</p><h3>Summary</h3><p>For reference, here are the various graphs we have plotted throughout the project. It would be interesting to test QLBS vs BS on a European call option, and try to figure out how Q learning can be applied to American call/put options. Once again, I’m linking the full code for the project <a href="https://colab.research.google.com/drive/1wa2puOkPuUt5LHdQBbyLJtQi6qYTYEC0">here</a> if you want to take a look at it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TX7yUPZC1WCS11ukLjcpAg.png" /></figure><p>Whew, that’s it for today!</p><h3>References</h3><p>[1] Coursera, <a href="https://www.coursera.org/learn/reinforcement-learning-in-finance">Reinforcement Learning in Finance</a></p><p>[2] Igor Halperin, <a href="https://arxiv.org/abs/1801.06077#clicktoread">The QLBS Q-Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios (2018)</a>, arXiv</p><p>[3] James Chen, <a href="https://www.investopedia.com/terms/o/option.asp">Options</a>, Investopedia</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fb2a0d6%3Fas_embed%3Dtrue&amp;dntp=1&amp;display_name=Upscribe&amp;url=https%3A%2F%2Fupscri.be%2Fb2a0d6%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/0707f5c806284d01a4a13c7b13a91ce3/href">https://medium.com/media/0707f5c806284d01a4a13c7b13a91ce3/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=765c84d0d97d" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/reinforcement-learning-for-options-trading-765c84d0d97d">Reinforcement Learning for Options Trading</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>