<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hyunsu Kim on Medium]]></title>
        <description><![CDATA[Stories by Hyunsu Kim on Medium]]></description>
        <link>https://medium.com/@hyunsukim-9320?source=rss-2b34b5c2101d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*1MUPzX1PmEW2rx28</url>
            <title>Stories by Hyunsu Kim on Medium</title>
            <link>https://medium.com/@hyunsukim-9320?source=rss-2b34b5c2101d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 13:05:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hyunsukim-9320/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deep Hedging and GANs]]></title>
            <link>https://hyunsukim-9320.medium.com/deep-hedging-and-gans-20c47a6e7b1c?source=rss-2b34b5c2101d------2</link>
            <guid isPermaLink="false">https://medium.com/p/20c47a6e7b1c</guid>
            <dc:creator><![CDATA[Hyunsu Kim]]></dc:creator>
            <pubDate>Fri, 04 Sep 2020 17:42:05 GMT</pubDate>
            <atom:updated>2021-03-11T01:47:47.848Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deep Hedging, GANs, and Beyond</h3><p>This brief research project introduces a potential application of deep learning/artificial intelligence in finance — particularly its applications in hedging.</p><p>For a more comprehensive paper, please refer to <a href="https://arxiv.org/abs/2103.03913">https://arxiv.org/abs/2103.03913</a></p><h3>Introduction</h3><p>In quantitative finance, traditionally two separate areas require quite advanced quantitative/mathematical techniques. As briefly mentioned by Attilio Meucci, on the one hand, the “Q” area of derivatives pricing deals with “extrapolating the present” by typically using Ito’s calculus and partial differential equations (PDEs) under some risk-neutral probability measures (Q). On the other hand, the “P’ area of quantitative risk/portfolio management deals with “modeling the future” by using multi-variate statistics and econometrics under some real-world probability measure (P).</p><p>Although those two areas seem to be distinctive and unique on each own, essentially two different sets of probabilistic weighting schemes are associated with the same possible outcomes given the same set of independent variables — understanding of risk premia makes a difference between them. The very gist of hedging is eventually protecting the future PnL (Profit&amp;Loss) of a given position from a set of risk factors. This sounds like a concept of the P-world. But, in calculating Greeks (as an example, delta — option’s sensitivity with respect to the underlying security), we classically use pricing models from Q-world, such as the Black-Scholes framework.</p><p>Through this brief research paper, we try to demonstrate a potential application of the classic direct policy search RLs in hedging. In doing so, we will show how recurrent neural networks directly learn an optimal hedging strategy by trial and errors (model-free) given the risk aversion of the investor — minimizing a convex risk measure like CVaR (Conditional Value at Risk). Also, we will briefly introduce how novel time-series GANs can be further explored in this hedging application — even beyond that, potentially bridging a gap between the P-world and the Q-world driven by artificial intelligence.</p><h3>Method</h3><p>As opposed to off-policy Q-learning and Monte Carlo Tree Search (MCTS), we will be using a policy search method where an agent directly learns how to map a given state to a particular action without learning action values to determine action selections. As detailed by Richard S. Sutton et al., this direct policy search method can entail many advantages over action value-based methods. A couple of them are:</p><p>- Asymptotically converging to deterministic policies with right levels of exploration</p><p>- Natively handling continuous action spaces well</p><p>- Being better suited for some problems to parametrize the policy rather than the value/action-value functions</p><p>The overall framework of the direct policy search method is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*YYcDszV7kkrSCGnTTNOnRQ.png" /><figcaption>General Description of Direct Policy Search</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*bPTR7dpCNfkHsbuLoXoUAg.jpeg" /></figure><p>With this general framework of the direct policy search in mind, the detailed framework of our hedging model is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*JwKYxc0Yp5pV8GxUo-s_JA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*lOFEar_bI8tes7bwblOhZQ.jpeg" /></figure><p>Then, connecting dots with the generic direct policy search, we can frame ours as the following reinforcement learning problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*4H4pcZXLIWY8Pmc3MQPkxA.jpeg" /></figure><p>Throughout the experiments in this paper, the followings are the initial parameter setting for the Black-Scholes framework and the shared hyper-parameter setting for the neural net framework:</p><ul><li>Initial underlying value = 100</li><li>K (strike value) = 100 (at the money)</li><li>r (risk-free rate) = 0</li><li>𝜎 (volatility) = 0.15</li><li>𝑇 (time-to-maturity) = 1/12 (annual basis — one month)</li><li>Time steps in a given month = 30 (30 days calendar)</li><li>RNN nodes at each layer = [128, 64, 64, 1]</li><li>Training epochs = 50</li><li>Batch size = 1000</li><li>Alpha (risk aversion parameter for CVaR) = 0.99</li><li>Optimizer = Adam</li></ul><h3>Results</h3><p>The overall hedge PnL distributions are the following for Black-Scholes, Simple RNN, LSTM, and GRU.</p><ul><li>Black-Scholes (Baseline)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*eWOHk9QjY1puPd_FZNXSbw.png" /></figure><ul><li>Simple RNN</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*f2jImMo1CPgFzXEFauVS-Q.png" /></figure><ul><li>LSTM</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*1sgo1vbIRNbucFDLNyKHcQ.png" /></figure><ul><li>GRU</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*E8a1q7SVCZGG5m3nePZJAw.png" /></figure><p>Both mean and CVaR(alpha = 0.99) show fairly close values to the Black-Scholes framework — particularly LSTM. But, the excess kurtosis of the Black-Scholes indicates rather higher values in comparison to all of the three nets. One of many potential explanations would be that all of the three agents were trained with a loss function whose objective is to minimize the left tail exposure, namely the hedge loss. In specific, in our experiment, the objective function was set to minimize the worst 1 in 100 hedge loss events — say 10 worst hedge losses out of 1000 events (definition of CVaR with the alpha of 0.99). As a result, the overall distribution of PnLs from all of the three agents is expected to show rather thinner left tails compared to the Black-Scholes framework. This, in turn, lowers the corresponding fourth moments for RL agents. And, this can be seen in the following bar chart looking at the left tail side of PnL distributions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*p3zNyOLtmD0ErlYdFmwKJQ.png" /><figcaption>PnL Distribution Comparison</figcaption></figure><p>Also, from the following scatter plots, we can confirm that the delta values of a call option were captured well across different time horizons — with the convexity nature in mind — although both Simple RNN and GRU showed some odd delta values between spot 90 and spot 100.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*dv1-yjyVQ1fYIugD4bhHHw.png" /><figcaption>Evolution of Deltas across Time Horizon</figcaption></figure><h3>Conclusions</h3><p>In this brief research report, we have investigated an application of reinforcement learning in hedging. We have shown that our RNN-based direct policy search RL agents are capable of achieving promising delta hedging results, minimizing tail risk exposures (consideration of risk appetite of investors), and still properly capturing the expected temporal behavior of deltas. Particularly, we have observed great performance from the LSTM-based RL agent in terms of its CVaR(0.99), overall PnL distribution, and the temporal behavior of deltas.</p><p>For those who want to extend this work for actual production usage, we recommend having three-way data splits; training/validation/test. That way, a set of hyper-parameters of the trained model was not chosen based on the test data — which is supposed to be unseen data from a modeling perspective.</p><h3>Directions for Future Work</h3><p>Bridging a gap between the Q-world and the P-world has been explored by many researchers in the Q-world. With a “better” understanding of the risk premia, one can shift from one probabilistic measure to the other via advanced PDE concepts (i.e. Girsanov’s theorem).</p><p>In this section, we introduce a novel time-series generative adversarial networks (GANs) pioneered by Jinsung Yoon et al. and how it can be used to generate synthetic data given historical multi-variate information available. This led us to believe that this general framework (or, beyond this framework such as time-series VAE-GAN expressed in rather continuous latent space with the Bayesian framework overlay) can be served as:</p><ul><li>A rather realistic time-series “simulator” generating potential market paths without imposing parametric process constraints</li><li>A multi-variate time-series forecasting model taking into account both temporal dependencies and correlation structures across different independent variables</li></ul><p>We will walk through an overview of generic GANs and the time-series GANs framework and present an experiment result with historical S&amp;P500 data set (multi-variate — six dimensions; open, high, low, close, adj_close, and volume).</p><p>The GANs were inspired by the very notion of the Turning test (Turing learning) where an artificial intelligence agent keeps trying to beat a given system (critic). In doing so, the system is eventually becoming convinced that the agent is a near-human for achieving a given goal. With the mathematical framework below in mind, the optimization of GANs essentially keeps carrying out until it reaches to the so-called “Nash equilibrium” where both parties in this game (generator and discriminator) have reached to their peak abilities so that the optimization does not improve any further — which is somehow opposite of the modal collapse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/959/1*ISzwxSswpILnC8M4U69W0Q.png" /><figcaption>Generic Framework of GANs</figcaption></figure><p>On top of its very nature of difficult convergence, dealing with multi-variate time-series poses additional challenges in terms of properly constructing a generative model. The reason being is that the model not only has to capture the distributions of independent variables within each time step but also should be able to accurately capture the serial dynamics across different time steps.</p><p>The time-series GANs proposed by Jinsung Yoon et al. took into account these aspects of consideration. By imposing the auto-encoder structure on top of the GANs, the adversarial nets can more efficiently play their game (between discriminator and generator) at a lower dimension latent space. Besides, it overlays a stepwise supervised loss based on the real data so that it facilitates the model to better preserve the stepwise conditional distributions in the data. And, minimizing this supervised loss jointly carries out with the reconstruction loss (from the auto-encoder part) and the unsupervised loss (from the adversarial networks). In doing so, the embedded latent space preserves both temporal and multi-variate dynamics concurrently in a lower-dimensional space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*MegnVj02ihbknBd74kGTFw.png" /><figcaption>Building Blocks of Time-series GANs</figcaption></figure><p>With the proposed time-series GANs architecture, we generated a synthetic dataset for the S&amp;P500 index. The S&amp;P500 dataset we used in this experiment consists of 17770 data points with six different features (open, high, low, close, adj_close, and volume).</p><p>The following two 2-D scatter plots show differences between the original data and the generated synthetic data — one visualized by PCA and the other presented by t-SNE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*1Rh3lego3DRSS4Knp1L9bQ.png" /><figcaption>Visualization of Original/Synthetic S&amp;P500 Data</figcaption></figure><p>The general pattern of the original data set was captured reasonably well. But, both two plots indicated there are some data points in certain sub-space that the synthetic data could not properly capture well. Further work is needed to investigate what causes the network’s inability to properly generate rather accurate synthetic data in certain sub-space (i.e., tail events — imbalance data in nature). Also, for future work reference, the classical auto-encoder embedding structure is known to not lead to particularly useful or nicely structured latent spaces without much good compression. By overlaying Bayesian inference, variational auto-encoder (VAE) can force the model to learn rather continuous and highly structured latent space. We believe that exploring time-series VAE-GANs will be a promising research area on this front, taking into account the advantages of both VAE and GANs in generating highly accurate synthetic data and even predicting sequential data in multivariate space.</p><h3>References</h3><p>Jinsung Yoon et al., “Time-series Generative Adversarial Networks,” Advances in Neural Information Processing Systems 32 (2019)</p><p>Attilio Meucci, “’P’ Versus ‘Q’: Differences and Commonalities between the Two Areas of Quantitative Finance,” GARP Risk Professional (2011)</p><p>Hans Buehler et al., “Deep Hedging,” arXiv preprint arXiv: 1802.03042 (2018)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20c47a6e7b1c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>