<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by ML Blogger on Medium]]></title>
        <description><![CDATA[Stories by ML Blogger on Medium]]></description>
        <link>https://medium.com/@mlblogging.k?source=rss-566f32fb7dde------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*PjVqoexC4PIFWBzp</url>
            <title>Stories by ML Blogger on Medium</title>
            <link>https://medium.com/@mlblogging.k?source=rss-566f32fb7dde------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:01:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mlblogging.k/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Awesome list of ‘nano’ repositories to learn from and build]]></title>
            <link>https://medium.com/@mlblogging.k/awesome-list-of-nano-repositories-to-learn-from-and-build-f8bedce04ed8?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8bedce04ed8</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tutorial]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sat, 18 Oct 2025 08:51:48 GMT</pubDate>
            <atom:updated>2025-10-18T08:51:48.655Z</atom:updated>
            <content:encoded><![CDATA[<p>Andrej Karpathy recently released “<a href="https://github.com/karpathy/nanochat">nanochat</a>,” a minimalistic implementation for training and fine-tuning GPT-type models end-to-end. It’s fully functional yet easy to understand and modify. Builders, hackers, and students will benefit massively they can learn from it and build on it to train their own models.</p><p>Here are similar projects which help you do the same</p><h3><a href="https://github.com/wolfecameron/nanoMoE">nanoMoE</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vbzJOB1-R3mClOROv19Ptw.png" /></figure><p>MoE stands for Mixture of Experts. This is a key transformer architecture innovation used by many leading opensource models like DeepSeek, Qwen, Mistral and many more. MoE is an efficient architecture replacing the dense FFN with many skinny experts; a cheap gating router activates only the top-k (often just 2) per token, so parameter count explodes while FLOPs per inference stay nearly flat. The model is sparsely conditioned: each expert learns a niche, the gate learns when to call it, and load-balancing losses keep the experts from collapsing into a single favorite. Result: trillion-scale models that train like giants but infer like dwarfs, scaling knowledge capacity faster than compute. MoE is used by many open source models like DeepSeek, Qwen, Mistral etc..</p><h3><a href="https://github.com/rjb7731/nanoMamba">nanoMamba</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*PFfKBDOzreBMkLFxg1b5oA.png" /></figure><p>Mamba removes attention entirely and uses a selective, input-dependent state space model (SSM) instead. The hidden state evolves through structured convolution, but the kernel is recomputed dynamically at each step — allowing the model to forget, copy, or focus as needed. A hardware-aware scan converts the recurrence into a single associative loop that runs in O(N) time and O(1) memory, with no KV-cache required. Stack these selective SSM blocks with SiLU gating and you get a sub-quadratic backbone that matches Transformer quality while scaling linearly on long contexts.</p><h3><a href="https://github.com/changgyhub/nanoDiffusion">nanoDiffusion</a> &amp; <a href="https://github.com/charles214chen/nanoDiffusion">nanoDDPM</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nyRPPZllLKnLDjYv3BkTvg.png" /></figure><p>Diffusion models learn to reverse a gradual noising process: a neural network (usually a U-Net) predicts the noise added at each step, guiding pure Gaussian chaos back to clean data. Training is simply denoising with squared error; sampling chains many small denoising steps, trading compute for quality. The same framework generates pixels, waveforms, or molecules — and with classifier-free guidance, it steers generation without needing extra reward models.</p><h3><a href="https://github.com/karpathy/micrograd">micrograd &amp;</a> <a href="https://github.com/xames3/nanotorch">nanotorch</a></h3><p>Autograd powers the current deep learning revolution. It computes gradients for arbitrary neural network architectures, enabling us to train large and complex models. Without it, even training a simple two-layer neural network becomes complex due to manual gradient computation. Autograd builds a dynamic DAG while you write plain Python every tensor operation becomes a node that caches its inputs and a backward function. During loss.backward(), the graph is traversed in reverse, applying the chain rule to emit exact gradients with memory usage proportional to the graph depth. It turns imperative loops, branches, and even in-place operations into differentiable programs.</p><h3><a href="https://github.com/huggingface/nanoVLM">nanoVLM</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P5YzeWEZobodyRCMpNaY8g.png" /></figure><p>A VLM connects a vision encoder (ViT, CNN, or hybrid) to a frozen LLM through a lightweight projection layer, mapping pixels into the same embedding space as tokens. After large-scale contrastive pre-training and instruction tuning, the model can answer questions about images, ground text in bounding boxes, and reason over interleaved photo-paragraph sequences all without task-specific heads. Because both modalities share one autoregressive decoder, the same attention layers that predict the next word also “predict” what they see, enabling unified reasoning that cites visual evidence in natural language.</p><h3><a href="https://github.com/huggingface/picotron">picotron &amp;</a> <a href="https://github.com/huggingface/nanotron">nanotron</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YnNbkSVC1GO0DwnPh7WKLA.png" /></figure><p>The minimalist &amp; most-hackable repository for pre-training Llama-like models with 4D Parallelism (Data, Tensor, Pipeline, Context parallel). It is designed with simplicity and educational purposes in mind, making it an excellent tool for learning and experimentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8bedce04ed8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Linear Attention]]></title>
            <link>https://medium.com/@mlblogging.k/understanding-linear-attention-74a0945b0155?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/74a0945b0155</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sun, 28 Sep 2025 16:12:13 GMT</pubDate>
            <atom:updated>2025-09-28T16:12:13.273Z</atom:updated>
            <content:encoded><![CDATA[<h3>Self Attention</h3><p>Attention mechanism in transformers is defined as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*peciTsS5Bbh8hBnAia_45A.png" /></figure><p>here <em>Q, K, V</em> are <em>( N, d )</em> which results in a complexity of <em>O(N²)</em> for the attention mechanism.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*w71kOYiZlJBhGTB8v5aJvA.png" /></figure><h4>Complexity</h4><p><em>Q, K, V</em> are (<em>N, d) </em>matrices, given this the attention multiplication sequence is ( ignoring the <em>softmax</em> for now )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/103/1*feuuHv-LoGtE7E8MlKqn7w.png" /><figcaption>Figure 1</figcaption></figure><p>this gives us a complexity of <em>O(N²d)</em> which ~ <em>O(N²) </em>as <em>d</em> is assumed a constant</p><h3>Linear Attention</h3><p>To get to linear attention we need to get to lower complexity required to compute attention from <em>O(N²)</em> to <em>O(N)</em>. One way to simplify this to revisit the matrix multiplication involved in attention. Rearranging the sequence of matrix multiplication in attention as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/103/1*s6kNTzoRJ4_PsVd9kWK6zQ.png" /><figcaption>Figure 2</figcaption></figure><p>gives us a complexity of <em>O(Nd²). </em>As you can see that it differs from <em>Figure 1. </em>So<em> </em>stops us from doing this for self attention ?</p><p>The answer is <em>softmax</em>. <em>Softmax</em> is a non linear operation which means</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*Im4vyTP2ot5gmb4Z8yLUzw.png" /><figcaption>Figure 3</figcaption></figure><p>If we can find a way to approximate the <em>softmax</em> operation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*IPk5z-xPrATqHU_ZxnYtbQ.png" /><figcaption>Figure 4</figcaption></figure><p>this will enable us to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*gSsi1g6oRab3zl-bKJ-KsQ.png" /><figcaption>Figure 5</figcaption></figure><p>and this leads us to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*iyw6QeBbzLtUU-KJm3XKGg.png" /><figcaption>Figure 6</figcaption></figure><h4>Softmax approximation using kernels</h4><p><a href="https://arxiv.org/pdf/1908.11775">Transformer Dissection</a> discusses the attention mechanism from a kernel perspective and shows softmax can be approximated using different types kernel functions. Later papers have takens this further and proposed multiple variations of Linear Attention using different kernel functions. Some of the examples are as follows</p><ul><li><a href="https://arxiv.org/abs/2006.16236">Transformers are RNNs</a> : uses a simple kernel using elu ( exponential linear unit ) as the non linearity.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/135/1*f9FXUR7KLC9wwWp5HStRrQ.png" /></figure><ul><li><a href="https://arxiv.org/abs/2202.08791">Cosformer</a> : Applies relu to the matrices to make the values positive and then applies a cos weighting scheme which mimics behaviour of softmax</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*lK4RDlcqyd11rVz4Xun7wQ.png" /></figure><ul><li><a href="https://arxiv.org/abs/2402.04347">HedgeDog</a> : uses a learnable weight matrix to transform the matrices.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/157/1*i70uxIOsL0w0L8hqWmuxGQ.png" /></figure><h4>Attention in autoregressive or causal setting</h4><p><strong>Self Attention</strong></p><p>In the autoregressive setting Self Attention will process an input <em>q,k,v </em>at time <em>t. </em>The challenges are obvious : the hidden state which is the <em>Q * Kᵀ </em>keeps growing in size as the input increases and it needs to stored to compute the attention with the input <em>q,k,v.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*zCPVrbmKC1mmmMeqZyTdBA.png" /><figcaption>Figure 7</figcaption></figure><p><strong>Linear Attention</strong></p><p>In the case of Linear Attention, the hidden state is of fixed size i.e (d, d). This is because the hidden state is computed as <em>Kᵀ*V </em>which is of (d, d).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*k0sZoL1dN3vK3lk65G1okg.png" /><figcaption>Figure 8</figcaption></figure><p>Its evident from <em>Figure 7 &amp; 8</em> that Linear Attention not only gives us linear complexity in processing inputs but also reduced memory to hold the hidden states of previous time steps. In Self Attention we have to save the hidden using a Key-Value cache to compute attention over incoming tokens where as in Linear attention the hidden state is always <em>(d, d).</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*cb9WbDq4OAbuvYMlK8TIMQ.png" /><figcaption>Figure 9</figcaption></figure><p>The Linear Attention in <em>figure 9</em> sets the stage for generic Linear Sequence modelling which uses a hidden state to process sequences as opposed to the Self Attention which requires expanding hidden state.</p><h3><strong>References</strong></h3><ul><li><a href="https://arxiv.org/abs/2508.09834">Speed Always Wins: A Survey on Efficient Architectures for Large Language Models</a></li><li><a href="https://arxiv.org/abs/2006.16236">Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention</a></li><li><a href="https://arxiv.org/abs/2202.08791">cosFormer: Rethinking Softmax in Attention</a></li><li><a href="https://arxiv.org/abs/2402.04347">The Hedgehog &amp; the Porcupine: Expressive Linear Attentions with Softmax Mimicry</a></li><li><a href="https://arxiv.org/pdf/1908.11775">Transformer Dissection: A Unified Understanding of Transformer’s Attention via the Lens of Kernel</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74a0945b0155" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding In-context Learning in Large Language Models ( like GPT3/GPT-J/GPTNeoX )]]></title>
            <link>https://medium.com/@mlblogging.k/understanding-in-context-learning-in-large-language-models-like-gpt3-gpt-j-gptneox-e0a71063a6db?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0a71063a6db</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sun, 12 Feb 2023 16:14:19 GMT</pubDate>
            <atom:updated>2023-02-12T16:16:05.429Z</atom:updated>
            <content:encoded><![CDATA[<h3>Large Language Models ( LLMs )</h3><p>The Large Language Model revolution started with the advent of transformers in 2017. Since then there has been an exponential growth in the models trained. Models with 100B+ parameters have been trained. These pre-trained models have changed the way NLP is done. It is much easier to pick a pre-trained model and fine-tune it for a downstream task ( sentiment, question answering, entity recognition etc.. ) than training a model from scratch. Fine-tuning can be done with a much smaller set of examples than training a model from scratch making the whole process of NLP much easier.</p><h3>Usage of LLMs</h3><p>Large Languages Models can be used in various settings for solving problems in NLP. Some of the well known approaches are as follows</p><ul><li><strong>Fine-tuning </strong>: This is probably the most well known way of applying pre-trained LLMs to any downstream task ( sentiment, entity recongnition, summarization etc.. ). This process usually involves taking the weights of the pre-trained LLM and fine-tuning on a smaller domain specific dataset. This is easy to do as the dataset for fine-tuning is usually smaller in size ( usually between 100s to 1000s of examples ). There has been recent advances in this method as the case of ChatGPT where Reinforcement Learning is used for fine-tuning ( <a href="https://medium.com/@mlblogging.k/reinforcement-learning-for-tuning-language-models-how-chatgpt-is-trained-9ecf23518302?source=friends_link&amp;sk=82d94fb2d3b6de273979c1991dcd8ea6">RLHF</a> ). You can read about this <a href="https://medium.com/@mlblogging.k/reinforcement-learning-for-tuning-language-models-how-chatgpt-is-trained-9ecf23518302?source=friends_link&amp;sk=82d94fb2d3b6de273979c1991dcd8ea6">here</a>.</li><li><strong>Prompting</strong> : This is a technique which came into light after the GPT model were released. Researchers and practitioners realized that you could get the model to give the right ‘result’ or ‘outcome’ by appending certain strings for keyword to indicate the task or problem you are trying to solve. This as evolved into in a niche field of its own given the success of Stable Diffusion type generative models.</li><li><strong>Incontext Learning (ICL) </strong>: This a few shot learning technique which again exclusively happens in Large Language Models of a certain size. These strong abilities have been widely verified as<a href="https://arxiv.org/abs/2206.07682"> emerging abilities for large language models</a>. Incontext learning (ICL) is the ability to learn from a few examples in the context.</li></ul><h3>Incontext Learning (ICL)</h3><p>large language models (LLMs) demonstrate an in-context learning (ICL) ability, that is, learning from a few examples in the context.The key idea of in-context learning is to learn from analogy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*Mj4UMqLgsgI7Kd4VbUGaVw.png" /></figure><p>The above image shows a summary of how incontext learning works. The basic process of incontext learning works as follows</p><ul><li>ICL requires a few examples to form a demonstration context. These examples are usually written in natural language templates</li><li>ICL concatenates a query question and a piece of demonstration context together to form a prompt is then fed into the language model for prediction</li></ul><p>This process is different from supervised learning which makes use of gradient updates to train the model. ICL doesn’t do parameter updates. This is a new paradigm and offers the following advantages</p><ul><li>It provides an interpretable interface to communicate with LLMs. This paradigm makes it much easier to incorporate human knowledge into LLMs by changing the demonstration and templates.</li><li>In-context learning is similar to the decision process of human beings by learning from analogy.</li><li>Compared with supervised training, ICL is a training-free learning framework</li></ul><h3>Examples</h3><p>Lets look at a few examples of how ICL works. We will be using GPT-NeoX for our examples ( though GPT3 or any other LLM will work as well ). You can use the GPT-NeoX playground <a href="https://20b.eleuther.ai/">here</a> or <a href="https://textsynth.com/playground.html">here</a> to test out the models. You can also use Huggingface library to test out ICL.</p><h4><strong>Entity Recognition</strong></h4><p><strong>Input :</strong></p><pre>[Text]: Elon Musk is the owner and CEO of Twitter, a social media company which he acquired in 2022.<br>[Name] : Elon Musk<br>[Position]: owner and CEO<br>[Company]: Twitter<br>###<br>[Text]: Microsoft was founded by Bill Gates on April 4, 1975. The current CEO of Microsoft is Satya Nadella.<br>[Name] : Satya Nadella<br>[Position]: CEO<br>[Company]: Microsoft<br>###<br>[Text]: Tim Cook became the CEO of Apple after Steve Jobs resigned as the CEO citing health reasons.<br>[Name]: Tim Cook<br>[Position]: CEO<br>[Company]: Apple<br>###<br>[Text]: Demis Hassabis is the CEO of DeepMind which was acquired by Google in 2014.</pre><p><strong>Output :</strong></p><pre>[Text]: Elon Musk is the owner and CEO of Twitter, a social media company which he acquired in 2022.<br>[Name] : Elon Musk<br>[Position]: owner and CEO<br>[Company]: Twitter<br>###<br>[Text]: Microsoft was founded by Bill Gates on April 4, 1975. The current CEO of Microsoft is Satya Nadella.<br>[Name] : Satya Nadella<br>[Position]: CEO<br>[Company]: Microsoft<br>###<br>[Text]: Tim Cook became the CEO of Apple after Steve Jobs resigned as the CEO citing health reasons.<br>[Name]: Tim Cook<br>[Position]: CEO<br>[Company]: Apple<br>###<br>[Text]: Demis Hassabis is the CEO of DeepMind which was acquired by Google in 2014.<br><strong>[Name] : Demis Hassabis<br>[Position]: CEO<br>[Company]: DeepMind<br>###</strong></pre><h4>SQL Code Completion</h4><p><strong>Input:</strong></p><pre>Question: Fetch the companies that have less than five people in it.<br>Answer: SELECT COMPANY, COUNT(EMPLOYEE_ID) FROM Employee GROUP BY COMPANY HAVING COUNT(EMPLOYEE_ID) &lt; 5;<br>###<br>Question: Show all companies along with the number of employees in each department<br>Answer: SELECT COMPANY, COUNT(COMPANY) FROM Employee GROUP BY COMPANY;<br>###<br>Question: Show the last record of the Employee table<br>Answer: SELECT * FROM Employee ORDER BY LAST_NAME DESC LIMIT 1;<br>###<br>Question: Fetch three employees from the Employee table</pre><p><strong>Output:</strong></p><pre>Question: Fetch the companies that have less than five people in it.<br>Answer: SELECT COMPANY, COUNT(EMPLOYEE_ID) FROM Employee GROUP BY COMPANY HAVING COUNT(EMPLOYEE_ID) &lt; 5;<br>###<br>Question: Show all companies along with the number of employees in each department<br>Answer: SELECT COMPANY, COUNT(COMPANY) FROM Employee GROUP BY COMPANY;<br>###<br>Question: Show the last record of the Employee table<br>Answer: SELECT * FROM Employee ORDER BY LAST_NAME DESC LIMIT 1;<br>###<br><strong>Question: Fetch three employees from the Employee table<br>Answer: SELECT * FROM Employee ORDER BY LAST_NAME ASC LIMIT 3;<br>###</strong></pre><h4>Python Code Completion</h4><p><strong>Input:</strong></p><pre># use pandas to read &#39;test.csv&#39; into dataframe<br>df = pd.read_csv(&#39;test.csv&#39;)<br># select columns &#39;time&#39; and &#39;data&#39; from df<br>df = df[[&#39;time&#39;, &#39;data&#39;]]<br># sort dataframe by &#39;time&#39;<br>df = df.sort_values(by=&#39;time&#39;)<br># parse &#39;time&#39; column to pandas datetime format DD-MM-YYYY</pre><p><strong>Output:</strong></p><pre># use pandas to read &#39;test.csv&#39; into dataframe<br>df = pd.read_csv(&#39;test.csv&#39;)<br># select columns &#39;time&#39; and &#39;data&#39; from df<br>df = df[[&#39;time&#39;, &#39;data&#39;]]<br># sort dataframe by &#39;time&#39;<br>df = df.sort_values(by=&#39;time&#39;)<br># parse &#39;time&#39; column to pandas datetime format DD-MM-YYYY<br><strong>df[&#39;date&#39;] = pd.to_datetime(df[&#39;time&#39;], format=&#39;%d-%m-%Y&#39;)</strong></pre><p>The section in bold is the model output generated based on the on the ‘context’ of examples. The output is limited to the first delimiter which is ‘###’ in the first two cases and is the new line starting with ‘#’ in the last case. We see that most of the model output seem right with minor errors. These outputs can be different depending on the model sampling parameters.</p><h3>Improving ICL</h3><p>As we have seen in the above examples ( and is pretty much evident ) that the performance of ICL depends on the quality of the examples chosen in the context. Few studies have shown that it also depends on the ordering of the examples as well.</p><h4><a href="https://arxiv.org/pdf/2302.00083.pdf">RALM based Incontext Learning</a></h4><p>RALM is Retrieval-Augmented Language Modelling. RALM involves choosing examples condition a language model (LM) on relevant documents from a grounding corpus during generation, have been shown to significantly improve language modelling while also providing a natural source attribution mechanism.</p><p>The core idea behind RALM for incontext learning can be summarized in the following steps</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*nIG8BG1PDxvqw7Ps0cTWTQ.png" /></figure><h3>How does it work ?</h3><p>While it is not known exactly how ICL works or functions there are few theories which try to explain the inner working of this phenomenon. Few papers studies which offer various explanations of this are as follows</p><ul><li><strong>Data Distributional Properties</strong> : This <a href="https://arxiv.org/abs/2205.05055">paper</a> tries to explain ICL by considering the distribution of data used to train the LLMs. The authors suggest that data which have a `skewed Zipfian distribution`, display non i.i.d behaviour &amp; burstiness tend to show emergent properties in LLMs.</li><li><strong>ICL as Implicit Bayesian Inference</strong> : This <a href="https://openreview.net/forum?id=R">paper</a> suggests that in-context learning can emerge when pretraining documents have long-range coherence. Here, the LM must infer a latent document-level concept to generate coherent next tokens during pretraining. At test time, in-context learning occurs when the LM also infers a shared latent concept between examples in a prompt. They prove when this occurs despite a distribution mismatch between prompts and pretraining data in a setting where the pretraining distribution is a mixture of HMMs.</li><li><strong>Implicit Fine-tuning</strong> : This <a href="https://arxiv.org/abs/2211.15661">paper</a> considers linear regression as a prototypical problem to show that Transformer-based in-context learners can implement standard finetuning algorithms implicitly.</li></ul><h3>References</h3><ul><li>A Survey on In-context Learning <a href="https://arxiv.org/pdf/2301.00234.pdf">https://arxiv.org/pdf/2301.00234.pdf</a></li><li>Emergent Abilities of Large Language Models <a href="https://arxiv.org/abs/2206.07682">https://arxiv.org/abs/2206.07682</a></li><li>In-Context Retrieval-Augmented Language Models <a href="https://arxiv.org/pdf/2302.00083.pdf">https://arxiv.org/pdf/2302.00083.pdf</a></li><li><a href="https://www.cs.princeton.edu/courses/archive/fall22/cos597G/lectures/lec07.pdf">https://www.cs.princeton.edu/courses/archive/fall22/cos597G/lectures/lec07.pdf</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0a71063a6db" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[14 Awesome Free Books for Machine Learning, Deep Learning & Reinforcement Learning]]></title>
            <link>https://medium.com/@mlblogging.k/14-awesome-free-books-for-machine-learning-deep-learning-reinforcement-learning-8f1d3a90a394?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f1d3a90a394</guid>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sat, 04 Feb 2023 18:46:15 GMT</pubDate>
            <atom:updated>2023-02-04T18:46:15.483Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re looking to have a career in machine learning or a data scientist who wants to transition into the machine learning world, below is a list of 14 free e-books to help you achieve this.These books will accelerate your learning and get you upto speed in Machine Learning and Deep Learning.</p><h4>1. <a href="https://hastie.su.domains/ISLR2/ISLRv2_website.pdf">An Introduction to Statistical Learning</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*tAz8vvd9eEYuqOhw2Z7WZA.jpeg" /></figure><p>This is a classic introductory book for Machine Learning concepts. The book talks about basic of Machine Learning from Linear Regression , Tree based models and also touches upon Neural networks. This book is good for understanding basic concepts of ML like bias-variance, overfitting, regularization etc..</p><h4>2. <a href="https://d2l.ai/d2l-en.pdf">Dive into Deep Learning</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*D0FKoHVW6Dv9Et_tGxooXQ.jpeg" /></figure><p>This book has a hands-on approach to Deep Learning. This is an interactive, deep learning book with code, math, and discussions with implementations with PyTorch, NumPy/MXNet, JAX, and TensorFlow. The authors of this book are academics &amp; industry professionals so the content has academic rigour and industry best practices.</p><h4>3. <a href="https://www.cs.cornell.edu/jeh/book.pdf">Foundations of Data Science</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*vSHnXpilaMKHiLdywOTp9A.jpeg" /></figure><p>This book is by John Hopcroft who is a well known Computer Scientist whose books on Automata Theory and Data structures are standard books. This book covers the basics from mathematical perspective.</p><h4>4. <a href="https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf">Understanding Machine Learning: From Theory to Algorithms</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*-cuSPrlAhGH7HWmz51pk6w.jpeg" /></figure><p>The book provides an extensive theoretical account of the fundamental ideas underlying machine learning and the mathematical derivations that transform these principles into practical algorithms.These include a discussion of the computational complexity of learning and the concepts of convexity and stability; important algorithmic paradigms including stochastic gradient descent, neural networks, and structured output learning; and emerging theoretical concepts such as the PAC-Bayes approach and compression-based bounds.</p><h4>5. <a href="https://algorithmsbook.com/files/dm.pdf">Algorithms For Decision Making</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*EsIqvx9aMxx8zNuw9igm5g.jpeg" /></figure><p>This book provides a broad introduction to algorithms for decision making under uncertainty. The book covers a wide variety of topics related to decision making, introducing the underlying mathematical problem formulations and the algorithms for solving them. Figures, examples, and exercises are provided to convey the intuition behind the various approaches. All the algorithms are implemented in the Julia Language.</p><h4>6. <a href="https://github.com/probml/pml-book/releases/latest/download/book1.pdf">Probabilistic Machine Learning: An Introduction</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*OlAEfjXLa-LzSLScigPdRQ.jpeg" /></figure><p>This book is by Kevin Murphy, who is the author of a previous well known <a href="https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020">book</a>. He is also a Senior Staff Research Scientist at Google. This book covers the basic of Machine Learning from Linear Algebra to Optimization.</p><h4>7. <a href="https://github.com/probml/pml2-book/releases/latest/download/book2.pdf">Probabilistic Machine Learning: Advanced Topics</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*G7l5pzAINUVqMdznH45FLg.jpeg" /></figure><p>This book is again by Kevin Murphy. This is part II to the above book <a href="https://github.com/probml/pml-book/releases/latest/download/book1.pdf"><em>Probabilistic Machine Learning: An Introduction</em></a><em>. </em>It takes the basics from the previous and builds on it. Various concepts and models like Inference, prediction &amp; generative models.</p><h4>8. <a href="https://arxiv.org/pdf/2301.04856">Multimodal Deep Learning</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*YIgF6eUWuzsWt57xFEWQDA.jpeg" /></figure><p>MultiModal Deep Learning is next big thing in Deep Learning. We now have models which have amazing performance in each individual modality of data i.e images, audio, text etc.. . The real world is more complex data is usually a combination of multi modal streams of data and building models to handle these datasets will bring the next break through.</p><h4>9. <a href="http://www.incompleteideas.net/book/RLbook2020.pdf">Reinforcement Learning: An Introduction</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*ZqbT7eELrrAPaS4IimSv1g.jpeg" /></figure><p>Sutton’s book is a classic. Richard Sutton is one of pioneers in the field of reinforcement learning. This book covers all the basics of reinforcement learning from scratch and no prior background is necessary apart from basic calculus and linear algebra.</p><h4>10. <a href="https://arxiv.org/pdf/2201.02135">Deep Reinforcement Learning, a textbook</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*-hadr_hnZM_HXyWP7uxXRw.jpeg" /></figure><p>This is an introductory book on Deep Reinforcement Learning covering all the recent basics and advances in Value based and Policy based Reinforcement Learning. The book also talks about advanced methods in Reinforcement Learning like Multi Agent RL, Hierarchical RL, Meta-Learning &amp; Self Play.</p><h4>11. <a href="https://www.cis.upenn.edu/~jean/math-deep.pdf">Algebra, Topology, Differential Calculus, and Optimization Theory</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*9Z-RV5YIRqhZxEyVsQ_ndA.jpeg" /></figure><p>This book is rigorous introduction to the math behind Computer Science and Machine Learning. This is book for advanced readers and this has in-depth coverage of mathematical concepts used in Machine Learning.</p><h4>12. <a href="http://www.mlebook.com/wiki/doku.php">Machine Learning Engineering</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*LUxtUEDutl-JBOTokz0uJw.jpeg" /></figure><p>This book takes a practical approach to things. It walks through the entire process a ML Engineer or a Data Scientist takes in the real world. The entire process of creating datasets to model training to model deployment is discussed.</p><h4>13. <a href="https://mml-book.github.io/book/mml-book.pdf">Mathematics for Machine Learning</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*AmmUUyAUCS_bU4nhzwzabQ.jpeg" /></figure><p>This book is intended to be a guidebook to the vast mathematical literature that forms the foundations of modern machine learning. The book motivates the need for mathematical concepts by directly pointing out their usefulness in the context of fundamental machine learning problems</p><h4>14. <a href="https://github.com/udlbook/udlbook/releases/download/v0.6.1/UnderstandingDeepLearning_02_02_23_C.pdf">Understanding Deep Learning</a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*oMsOV-VB_NCo3RFOrIrPig.jpeg" /></figure><p>This text is primarily about the ideas that underlie deep learning. The first part of the book introduces deep learning models and discusses how to train them, measure their performance, and improve this performance. The next part considers architectures that are specialized to images, text, and graph data. These chapters require only introductory linear algebra, calculus, and probability and should be accessible to any second-year undergraduate in a quantitative discipline. Subsequent parts of the book tackle generative models and reinforcement learning</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f1d3a90a394" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[14 Loss functions you can use for Regression]]></title>
            <link>https://medium.com/@mlblogging.k/14-loss-functions-you-can-use-for-regression-b24db8dff987?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/b24db8dff987</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sat, 21 Jan 2023 11:43:47 GMT</pubDate>
            <atom:updated>2023-01-21T11:43:47.504Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is a loss function ?</h3><p>In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some “cost” associated with the event. An optimization problem seeks to minimize a loss function. An objective function is either a loss function or its opposite (in specific domains, variously called a reward function, a profit function, a utility function, a fitness function, etc.), in which case it is to be maximized. The loss function could include terms from several levels of the hierarchy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*ZR5-Q2oItLdJhHX9SNtVxA.png" /></figure><h3>Regression vs Classification</h3><ul><li>Regression algorithms predict a continuous value based on the input variables. The main goal of regression problems is to estimate a mapping function based on the input and output variables. Regression algorithms are used to predict the continuous values like age, price etc..</li><li>Classification is a predictive model that approximates a mapping function from input variables to identify discrete output variables, which can be labels or categories.</li></ul><p>The kind of loss function you are going to use depends on the kind of problem you are working i.e Regression or Classification.</p><h3>Loss Functions for Regression</h3><p>Below you will find the loss functions you can use for solving a Regression problem</p><h4>1. Mean Absolute Error (MAE)</h4><p>This is also known as the L1 loss</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/243/1*U84o7PI5n7HkpvYA7vUwuQ.png" /></figure><p>This loss function is easy to compute and measures the absolute difference between the true and predicted value. It is not sensitive to outliers and it is also not differentiable at zero.</p><h4>2. Mean Squared Error (MSE)</h4><p>This is also known as the L2 loss</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*iaYUR1ewSIh82-rjfUBwBw.png" /></figure><p>This loss function handles outliers in an efficient manner as outliers are detected due to the quadratic loss. Convergence is also smooth as the gradient becomes smaller as the loss decreases.</p><h4>3. Mean Bias Error (MBE)</h4><p>This loss measures the ability of the model to over estimate or under estimate the bias</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/212/1*O-MvV_WDEIYcFjonu5v8OQ.png" /></figure><p>The above loss differs from MAE as it doesn’t use the absolute value. It helps in identifying the direction of the model bias i.e ( +ve or -ve ). The issue with this loss is that some times the errors may cancel out between individual samples resulting in a zero loss which may be misleading.</p><h4>4. Relative Absolute Error (RAE)</h4><p>The calculation of relative absolute error involves taking the total absolute error and dividing it by the absolute difference between the mean and the actual value.5. Relative Squared Error (RSE)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/301/1*qW924oAPHs4KF4exvp3klg.png" /></figure><p>RAE is a ratio-based metric that assesses the efficacy of a predictive model. RAE has a possible value between 0 and infinity. Values near zero, with zero being the best value, are characteristics of a good model.</p><h4>5. Relative Squared Error (RSE)</h4><p>The relative squared error is relative to what it would have been if a simple predictor had been used. More specifically, this simple predictor is just the average of the actual values. Thus, the relative squared error takes the total squared error and normalizes it by dividing by the total squared error of the simple predictor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*CPEIQGoIH170i6mMYwF42A.png" /></figure><p>For a perfect fit, the numerator is equal to 0 and loss = 0. So, the loss index ranges from 0 to infinity, with 0 corresponding to the ideal.</p><h4>6. Mean Absolute Percentage Error (MAPE)</h4><p>The mean absolute percentage error (MAPE), also known as the mean absolute percentage deviation (MAPD), is a metric used to assess the accuracy of a forecast system</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/258/1*1WkTibzGsG4y28c1A4rZ5A.png" /></figure><h4>7. Root Mean Squared Error (RMSE)</h4><p>The square root of MSE is used to calculate RMSE</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*g1cqPIwIFml90n_B68Zmvg.png" /></figure><h4>8. Mean Squared Logarithmic Error (MSLE)</h4><p>The mean squared logarithmic error (MSLE) measures the difference between actual and expected values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*B8f-qD3fREamJYtIwDzeKg.png" /></figure><p>The loss function is scale independent as it is a difference of two log values which is the same as log of the ratio of the values. Due to the loss being log it penalizes underestimates more than overestimates.</p><h4>9. Root Mean Squared Logarithmic Error (RMSLE)</h4><p>The root mean squared logarithmic error is determined by applying log to the actual and predicted numbers and then subtracting them. RMSLE is resistant to outliers when both minor and large errors are considered</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*J6fj4HAOgO50peUF-4WTfg.png" /></figure><p>The loss function is scale independent as it is a difference of two log values which is the same as log of the ratio of the values. Due to the loss being log it penalizes underestimates more than overestimates. Same as MSLE but the root mean square version of it.</p><h4>10. Normalized Root Mean Squared Error (NRMSE)</h4><p>The Normalized Root Mean Square Error (NRMSE) the RMSE facilitates the comparison between models with different scales. Here o is the average of the observation values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*YZU8JSUbbhxPy_Y7Crqdnw.png" /></figure><p>NRMSE overcomes the scale dependency and eases comparison between models of different scales or datasets.</p><h4>11. Relative Root Mean Squared Error (RRMSE)</h4><p>RRMSE is an RMSE variant without dimensions. Relative Root Mean Square Error (RRMSE) is a root mean square error measure that has been scaled against the actual value and then normalized by the root mean square value</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/276/1*n_cabLnGIn78DExqjzspsw.png" /></figure><h4>12. Huber Loss</h4><p>Huber loss is an ideal combination of quadratic and linear scoring algorithms</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*nxlOZlzmjm8sTSXoEPVysw.png" /></figure><p>Huber loss is a combination of two loss functions quadratic and linear. The behaviour of the loss is defined by the value of the threshold. for loss values beyond the threshold the loss is linear else quadratic.</p><h4>13. Log Cosh Loss</h4><p>Log cosh calculates the logarithm of the hyperbolic cosine of the error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*YTGdDM54Nqw87qzefHzt3Q.png" /></figure><p>This loss is very close to the Huber loss in behaviour as it has linear behaviour for very large values of error and quadratic for small loss values. The benefit here is you dont have to decide the value of the threshold as in the case of the Huber loss.</p><h4>14. Quantile Loss</h4><p>The Quantile regression loss function is used to forecast quantiles. The quantile is the value that indicates how many values in the group fall below or above a particular threshold</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/438/1*b6lyv014qCmy5lQWVl0Xog.png" /></figure><p>Here, γ represents the required quantile.The quantile values are chosen based on how we wish to balance the positive and the negative errors.</p><h3>References</h3><ul><li>A survey and taxonomy of loss functions in machine learning <a href="https://arxiv.org/abs/2301.05579">https://arxiv.org/abs/2301.05579</a></li><li>A Comprehensive Survey of Regression Based Loss Functions for Time Series Forecasting <a href="https://arxiv.org/abs/2211.02989">https://arxiv.org/abs/2211.02989</a></li><li>A survey of loss functions for semantic segmentation <a href="https://arxiv.org/abs/2006.14822">https://arxiv.org/abs/2006.14822</a></li><li><a href="https://en.wikipedia.org/wiki/Loss_function">https://en.wikipedia.org/wiki/Loss_function</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b24db8dff987" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[6 metrics to evaluate the performance of your portfolio]]></title>
            <link>https://medium.com/@mlblogging.k/6-metrics-to-evaluate-the-performance-of-your-portfolio-659b4b80890b?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/659b4b80890b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[finance]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sat, 21 Jan 2023 11:31:59 GMT</pubDate>
            <atom:updated>2023-01-21T11:31:59.937Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*Wl1WSAY92xperHsSAc_O3g.jpeg" /></figure><p>The following metrics can be used to measure the performance your portfolio or trading returns.</p><h4>1. Sharpe Ratio</h4><p>The Sharpe ratio (also known as the Sharpe index, the Sharpe measure, and the reward-to-variability ratio) measures the performance of an investment such as a security or portfolio compared to a risk-free asset, after adjusting for its risk. It was named after William F. Sharpe, who developed it in 1966.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/167/1*JEeLM_gj7Nqtt-PQitQHew.png" /></figure><pre> def annualized_sharpe_ratio(self):<br>     return round(<br>         ( (self.returns.mean() - self.risk_free_rate) / (self.returns.std()) )\<br>                    * ( np.sqrt(self.frequency) ), 2<br>         )</pre><h4>2. Calmar Ratio</h4><p>Calmar ratio (or Drawdown ratio) is a performance measurement used to evaluate Commodity Trading Advisors and hedge funds. It was created by Terry W. Young and first published in 1991 in the trade journal <em>Futures.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*T56zuRndWmQpQ8JiR9VC3w.png" /></figure><pre>def calmar_ratio(self):<br>    return round(self.annualized_returns()/abs(self.max_draw_down()), 2)</pre><h4>3. Sortino Ratio</h4><p>The Sortino ratio measures the risk-adjusted return of an investment asset, portfolio, or strategy. It is a modification of the Sharpe ratio but penalizes only those returns falling below a user-specified target or required rate of return, while the Sharpe ratio penalizes both upside and downside volatility equally.</p><pre>def sortino_ratio(self):<br>    average_annual_return = np.nanmean(self.returns, axis=0) * self.frequency<br>    return round(<br>        (average_annual_return - self.risk_free_rate)/self.downside_risk(), 2<br>    )</pre><h4>4. Max DrawDown</h4><p>A maximum drawdown (MDD) is the maximum observed loss from a peak to a trough of a portfolio, before a new peak is attained. Maximum drawdown is an indicator of downside risk over a specified time period</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*rNJfKqb6jzjcTOz1fXk3xA.png" /></figure><pre>def max_draw_down(self):<br>    cum_returns = self.returns.add(1).cumprod()<br>    draw_down = cum_returns.div(cum_returns.cummax()).sub(1)<br>    max_draw_down = draw_down.min()*100<br>    return round( max_draw_down, 2 )</pre><h4>5. Volatility</h4><p>Volatility often refers to the amount of uncertainty or risk related to the size of changes in a security’s value. A higher volatility means that a security’s value can potentially be spread out over a larger range of values</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*JDUS3tw5EZwvnAUGmt8YGQ.png" /></figure><pre>def annualized_volatility(self):<br>    return round(<br>          self.returns.std()*np.sqrt(self.frequency)*100, 2<br>    )</pre><h4>6. Beta</h4><p>A beta coefficient can measure the volatility of an individual stock compared to the systematic risk of the entire market.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*GKLInqkzF0DUAf2j9ONy5A.png" /></figure><pre>def beta(self,index):<br>    return round(self.returns.cov(index)/index.var(), 2)</pre><h4>Sample Metrics for Lockheed Martin ( LMT ) between 2022–01–01 &amp; 2023–01–01</h4><p>The entire code to run for the metrics is given below and can be run for any stock by providing the ticker.</p><pre>import pandas as pd<br>import pandas_datareader.data as web<br>import numpy as np<br><br>class Metrics():<br>    def __init__(self, returns, risk_free_rate=2, period=&#39;daily&#39;):<br>        self.returns = returns<br>        self.period = {<br>            &#39;monthly&#39; : 12,<br>            &#39;weekly&#39; : 52,<br>            &#39;daily&#39; : 252<br>        }<br>        self.frequency = self.period[period]<br>        self.risk_free_rate = np.power(1+risk_free_rate*0.01,1/self.frequency) - 1<br><br>    def annualized_sharpe_ratio(self):<br>        return round(<br>            ( (self.returns.mean() - self.risk_free_rate) / (self.returns.std()) )\<br>                      * ( np.sqrt(self.frequency) ), 2<br>        )<br><br>    def annualized_volatility(self):<br>        return round(<br>            self.returns.std()*np.sqrt(self.frequency)*100, 2<br>        )<br><br>    def max_draw_down(self):<br>        cum_returns = self.returns.add(1).cumprod()<br>        draw_down = cum_returns.div(cum_returns.cummax()).sub(1)<br>        max_draw_down = draw_down.min()*100<br>        return round( max_draw_down, 2 )<br><br>    def cumulative_rets(self):<br>        return (self.returns+1).prod()<br><br>    def annualized_returns(self):<br>        num_years = len(self.returns) / self.frequency<br>        ending_value = self.cumulative_rets()<br>        return (ending_value **(1 / num_years) - 1)*100<br><br>    def calmar_ratio(self):<br>        return round(self.annualized_returns()/abs(self.max_draw_down()), 2)<br><br>    def downside_risk(self):<br>        df = pd.DataFrame()<br>        df[&#39;Returns&#39;] = self.returns<br>        df[&#39;downside_returns&#39;] = 0<br>        df.loc[df[&#39;Returns&#39;] &lt; 0, &#39;downside_returns&#39;] = df[&#39;Returns&#39;]**2<br>        down_risk = np.sqrt(df[&#39;downside_returns&#39;].mean())*np.sqrt(self.frequency)<br>        return down_risk<br><br>    def sortino_ratio(self):<br>        average_annual_return = np.nanmean(self.returns, axis=0) * self.frequency<br>        return round(<br>            (average_annual_return - self.risk_free_rate)/self.downside_risk(), 2<br>        )<br><br><br>    def beta(self,index):<br>        return round(self.returns.cov(index)/index.var(), 2)<br><br>if __name__ == &#39;__main__&#39;:<br><br>    stock = &#39;LMT.US&#39;<br>    index = &#39;^DJI&#39;<br><br>    start = &#39;2022-01-01&#39;<br>    end = &#39;2023-01-01&#39;<br><br>    lmt = web.DataReader(stock, &#39;stooq&#39;)<br>    lmt = lmt.sort_values(by=&#39;Date&#39;)<br>    lmt = lmt[&#39;Close&#39;].pct_change()<br>    mask = ( start &lt;= lmt.index ) &amp; ( lmt.index &lt;= end )<br>    lmt = lmt[mask]<br><br>    dji = web.DataReader(index, &#39;stooq&#39;)<br>    dji = dji.sort_values(by=&#39;Date&#39;)<br>    dji = dji[&#39;Close&#39;].pct_change()<br>    mask = ( start &lt;= dji.index ) &amp; ( dji.index &lt;= end )<br>    dji = dji[mask]<br><br><br>    metrics = Metrics(lmt)<br><br>    print(&#39;Sharpe Ratio &#39;, metrics.annualized_sharpe_ratio())<br>    print(&#39;Calmar Ratio &#39;, metrics.calmar_ratio())<br>    print(&#39;Sortino Ratio &#39;, metrics.sortino_ratio())<br>    print(&#39;Max Draw Down &#39;, metrics.max_draw_down())<br>    print(&#39;Volatility &#39;, metrics.annualized_volatility())<br>    print(&#39;Beta &#39;, metrics.beta(dji))</pre><p>The code can be also found <a href="https://gist.github.com/mlblogger/fb9552b434e9e2217caf961d01f61321">here</a>.</p><p>The output is as follows</p><pre>Sharpe Ratio  1.35<br>Calmar Ratio  2.4<br>Sortino Ratio  2.21<br>Max Draw Down  -16.94<br>Volatility  26.43<br>Beta  0.41</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=659b4b80890b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating an OpenAI gym for applying Reinforcement Learning to the stock trading problem]]></title>
            <link>https://medium.com/@mlblogging.k/creating-an-openai-gym-for-applying-reinforcement-learning-to-the-stock-trading-problem-61b4506de608?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/61b4506de608</guid>
            <category><![CDATA[algorithmic-trading]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Thu, 19 Jan 2023 11:40:35 GMT</pubDate>
            <atom:updated>2023-01-19T11:40:35.881Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is OpenAI gym ?</h3><p><a href="https://github.com/openai/gym">Gym</a> is an open source Python library for developing and comparing reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments. Recently the gym is no longer maintained by OpenAI but by another team and the latest versions can be found <a href="https://github.com/Farama-Foundation/Gymnasium">here</a>.</p><p>Gym has in built environments for getting started with reinforcement learning like CartPole, LunarLander and various Atari games but it is not limited to these as gym offers apis for customization and creating application specific environments. We will now look at one such application of gym to the stock trading problem.</p><p>We will only discuss the implementation of the reinforcement learning environment and not training any agents as the training can be done with available libraries such as stable baselines.</p><h3>Implementation</h3><h4>Understanding the gym.Env class</h4><p>All the environments created in OpenAI gym should inherit from the gym.Env class. The gym.Env class defines the api needed for the environment. The apis we need to focus on are as follows</p><ul><li><strong>__init__ </strong>: This has the initializations to be done while instantiating the environment.</li><li><strong>reset</strong> : This function resets the environment to its starting state. This is invoked when ever the environment reaches a terminal state or a state which is invalid. It returns the initial state.</li><li><strong>render</strong> : This for rendering the data into visual form. We will not be using this function.</li><li><strong>step</strong> : This is the core of the environment which implements what happens when the agent ‘acts’ on the environment. The step function returns a tuple of four variables — ( observation, reward, done, info )</li></ul><h4>Required imports</h4><pre>import gym<br>from gym.envs.registration import register<br>from gym import error, spaces, utils<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import pandas as pd<br>import pandas_datareader.data as web<br>import arrow<br>import random<br>import sys</pre><h4>Implementing __init__</h4><p>In our example the __init__ function takes in the</p><ul><li>start_date &amp; end_date : these are the start and end dates to use for the data in the environment</li><li>tc &amp; ticker : these represent the transaction cost &amp; the ticker for which we will load the data. its set to ‘^DJI’ i.e Dow Jones index as default.</li></ul><p>Apart from the parameters we also initialize <em>self.action_space &amp; self.observation_space. </em>These variables capture the shape/type of action and the range/size of the observation space. These are required if you are going to use the environment with off the shelf modules for training.</p><p>The next few lines load the dataset and create the features which we will use in our state space and reward. We also initialize <em>self.curr_index</em> and <em>self.action</em> to track the current date active in the environment and the previous action.</p><pre>class TradingEnv(gym.Env):<br>    def __init__(self, start_date, end_date, tc=0.05/100, ticker=&#39;^DJI&#39;):<br>        self.start = start_date<br>        self.end = end_date<br>        self.tc = tc<br>        self.ticker = ticker<br><br>        self.action_space = spaces.Discrete(1)<br>        self.observation_space = spaces.Box(low=-1,high=1,dtype=np.float32)<br>        returns = self.load_dataset()<br>        self.data_df = self.create_features(returns)<br>        self.curr_index = 0<br>        self.data_len = self.data_df.shape[0]<br>        self.action = 0</pre><h4>Implementing load_dataset &amp; create_features</h4><p><strong>load_dataset </strong>: This function loads the data from stooq.com using pandas datareader and the appropriate time slice is selected and returns are computed from the stock price. This function can be modified to use any datasource such YahooFinance or any other custom datasources as well ( including loading data from a file )</p><p><strong>create_features</strong> : This function takes in the returns and computes the returns features as the lagged returns of upto 5 days. It also computes the label return which is the next day return for day. We will use the label return to compute the reward in our environment.</p><pre>def load_dataset(self):<br>    df = web.DataReader(self.ticker, &#39;stooq&#39;)<br>    mask = ( self.start &lt;= df.index ) &amp; ( df.index &lt;= self.end )<br>    df = df[mask]<br>    df = df.sort_values(by=&#39;Date&#39;)<br>    returns = df[&#39;Close&#39;].pct_change()<br>    return returns<br><br>def create_features(self, returns):<br>    dfs = []<br>    for i in range(5):<br>        dfs.append(returns.shift(i).rename(&#39;r%d&#39;%i))<br>    dfs.append(returns.shift(-1).rename(&#39;Y&#39;))<br>    df_net = pd.concat(dfs, axis=1)<br>    df_net = df_net.dropna()<br>    return df_net</pre><h4>Implementing reset</h4><pre>def reset(self):<br>    self.curr_index = 0<br>    return self.extract_state(self.data_df, self.curr_index).values</pre><h4>Implementing step</h4><p>This function is the core logic of the environment. The logic here is as follows:</p><ul><li>get action from agent and compute change in position ( i.e the amount of stock the agent is holding )</li><li>use the change in position to compute the actual cost of transacting ( we need to do this as transacting in stock always incurs a cost which can be either the commission paid for transaction or cost incurred in terms of slippage )</li><li>compute reward as function of action and cost</li><li>update counter and fetch next observation</li><li>return the values to the agent</li></ul><pre>def step(self, action):<br><br>    done = False<br>    stock_return = self.extract_return(self.data_df, self.curr_index)<br>    change_in_position = np.abs(self.action - action)<br>    cost = change_in_position * self.tc<br>    reward = action*stock_return - cost<br><br>    if self.curr_index == self.data_len - 2:<br>        done = True<br>    self.curr_index += 1<br>    self.action = action<br><br>    obs = self.extract_state(self.data_df, self.curr_index).values<br><br>    info = { &#39;date&#39; : self.data_df.index[self.curr_index], &#39;return&#39; : stock_return }<br><br>    return obs, reward, done, info</pre><h4>Registering the env with gym</h4><p>The environment has to be registered with the gym framework for it to be instantiated with a name. The __init__ params are passed as kwargs to the register function.</p><pre>ENV_NAME = &#39;TradingEnv-v0&#39;<br><br>reg = register(<br>    id=ENV_NAME,<br>    entry_point=&#39;__main__:TradingEnv&#39;,<br>    kwargs={<br>        &#39;start_date&#39; : &#39;2019-01-01&#39;,<br>        &#39;end_date&#39; : &#39;2022-01-10&#39;<br>    }<br>)</pre><h4>Testing the env with a random agent</h4><p>To test out the agent we use a simple random agent which uniformly selects an action between -1 , 1 and acts on the environment.</p><pre>if __name__ == &#39;__main__&#39;:<br><br>    env = gym.make(ENV_NAME)<br>    obs = env.reset()<br>    done = False<br><br>    reward_returns = []<br>    while not done:<br>        action = random.uniform(-1,1)<br>        obs, reward, done, info = env.step(action)<br>        reward_returns.append( (reward,info[&#39;return&#39;]) )<br><br>    i = list(range(len(reward_returns)))<br>    agent_returns = np.cumprod([ 1 + x[0] for x in reward_returns ])<br>    stock_returns = np.cumprod([ 1 + x[1] for x in reward_returns ])<br><br>    plt.plot(i, agent_returns, label=&#39;random_agent_return&#39;)<br>    plt.plot(i, stock_returns, label=&#39;actual_stock_return&#39;)<br>    plt.legend()<br>    plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLTZdlLYVg0PXcRnTpISGA.png" /><figcaption>Plot of the random agent simulation compared to the stock return</figcaption></figure><p>The environment is now ready and fully functional and can be used with any reinforcement learning library to train the agents. It can be used with <a href="https://github.com/DLR-RM/stable-baselines3">stable baselines</a> for example to train agents.</p><p>There are lot of enhancement which can be done to the environment specially regarding the dataset and the features. The example environment I have coded up just uses the stock price as the dataset and creates simple returns based features but this can be improved by incorporating much richer datasets and creating complex features ( e.g using technical indicators ).</p><p>The complete code can be found <a href="https://gist.github.com/mlblogger/8a2099859cd542ba634d2c64b405685e">here</a>.</p><h3>References</h3><ul><li><a href="https://github.com/openai/gym">https://github.com/openai/gym</a></li><li><a href="https://www.gymlibrary.dev/">https://www.gymlibrary.dev/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61b4506de608" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simulating Brownian Motion ( and Stock Prices ) using Python]]></title>
            <link>https://medium.com/@mlblogging.k/simulating-brownian-motion-and-stock-prices-using-python-17b6b4bd2a1?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/17b6b4bd2a1</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sun, 15 Jan 2023 09:06:39 GMT</pubDate>
            <atom:updated>2023-01-15T12:56:52.185Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Brownian Motion ?</h3><p>Brownian motion is named after Robert Brown, who was one of the first to comment on the phenomenon when observing through a microscope the seemingly random and continuous movement of pollen grains suspended in water.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FR5t-oA796to%3Fstart%3D11%26feature%3Doembed%26start%3D11&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DR5t-oA796to&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FR5t-oA796to%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0278ead9000e4c72fa169491d1eb22a/href">https://medium.com/media/d0278ead9000e4c72fa169491d1eb22a/href</a></iframe><p>The mathematical model of Brownian motion was given by Albert Einstein in 1905 ( his miraculous year in which he also published papers in relativity ). He modelled the motion of the pollen particles as being constantly bombarded by water molecules resulting in random movement.</p><h3>Random Walk</h3><p>The basis of Brownian Motion is a Random Walk. A random walk in 1-D is defined as follows :</p><ul><li>A simple random walk in 1-D is when a step forward (+d distance ) has probability p and step back ( -d distance ) has probability q = 1 — p</li><li>This can be extended to multiple dimensions quite easily using the same logic as in the 1-D case</li></ul><p>Random walks are an example of Markov processes, in which future behaviour is independent of past history</p><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_1d_rw(nsteps=1000, p=0.5, stepsize=1):<br>    steps = [ 1*stepsize if random.random() &lt; p else -1*stepsize for i in range(nsteps) ]<br>    y = np.cumsum(steps)<br>    x = list(range(len(y)))<br><br>    return x, list(y)<br><br>simulation_data = {}<br>nsims = 5<br>for i in range(nsims):<br>    x, y = simulate_1d_rw()<br>    simulation_data[&#39;x&#39;] = x<br>    simulation_data[&#39;y{col}&#39;.format(col=i)] = y<br><br>ycols = [ &#39;y{col}&#39;.format(col=i) for i in range(nsims) ]<br>fig = px.line(simulation_data, x=&#39;x&#39;, y=ycols)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MSdkFAo8KnEiNpv8MvKtbA.png" /><figcaption>Multiple simulations of 1-D random walk</figcaption></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_2d(nsteps=10000, stepsize=1):<br><br>    deltas = [ (0,-1*stepsize), (-1*stepsize,0), (0,1*stepsize), (1*stepsize,0) ]<br><br>    steps = [ list(random.choice(deltas)) for i in range(nsteps) ]<br>    steps = np.array(steps)<br>    steps = np.cumsum(steps,axis=0)<br>    y = list(steps[:,1])<br>    x = list(steps[:,0])<br><br>    return x, y<br><br>x, y = simulate_2d()<br>fig = px.line({ &#39;x&#39; : x, &#39;y&#39; : y }, x=&#39;x&#39;, y=&#39;y&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*i9WQXvsCNvaqx5H7I-hu-A.png" /><figcaption>Simulation of 2-D random walk for 10K steps</figcaption></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_3d_rw(nsteps=10000, stepsize=1):<br><br>    deltas = [ (0,0,-1*stepsize), (0,-1*stepsize,0), (-1*stepsize,0,0),\<br>               (0,0,1*stepsize), (0,1*stepsize,0), (1*stepsize,0,0) ]<br><br>    steps = [ list(random.choice(deltas)) for i in range(nsteps) ]<br>    steps = np.array(steps)<br>    steps = np.cumsum(steps,axis=0)<br>    z = list(steps[:,2])<br>    y = list(steps[:,1])<br>    x = list(steps[:,0])<br><br>    return x, y, z<br><br>x, y, z = simulate_3d_rw()<br>fig = px.line_3d({ &#39;x&#39; : x, &#39;y&#39; : y, &#39;z&#39; : z }, x=&#39;x&#39;, y=&#39;y&#39;, z=&#39;z&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NvvjsGAYf5flb8WRbZeU0w.png" /><figcaption>Simulation of 3-D random walk for 10K steps</figcaption></figure><h3>Brownian Motion</h3><p>The Wiener Process is the underlying random process which generates Brownian Motion. The properties of a Wiener process is as follows :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*3JqFCYN_kx5jEA9GXuSuYQ.png" /></figure><p>Here X(t) is the path travelled by Brownian Motion, D(t) is the ‘steps’ in the Brownian motion. Usually σ is constant in the standard Brownian motion.</p><p>Brownian motion can be derived as a limit of a random walk as the step size decreases to zero and time becomes continuous</p><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_1d_bm(nsteps=1000, t=0.01):<br>    steps = [ np.random.randn()*np.sqrt(t) for i in range(nsteps) ]<br>    y = np.cumsum(steps)<br>    x = [ t*i for i in range(nsteps) ]<br>    return x, y<br><br>nsims = 5<br>simulation_data = {}<br>for i in range(nsims):<br>    x, y = simulate_1d_bm()<br>    simulation_data[&#39;y{col}&#39;.format(col=i)] = y<br>    simulation_data[&#39;x&#39;] = x<br><br>ycols = [ &#39;y{col}&#39;.format(col=i) for i in range(nsims) ]<br>fig = px.line(simulation_data, x=&#39;x&#39;, y=ycols)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CjOy5VcmAvfN_piTGIzgfQ.png" /><figcaption>Multiple simulations of 1-D Brownian Motion</figcaption></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_2d_bm(nsteps=1000, t=0.01):<br>    x = np.cumsum([ np.random.randn()*np.sqrt(t) for i in range(nsteps) ])<br>    y = np.cumsum([ np.random.randn()*np.sqrt(t) for i in range(nsteps) ])<br>    return list(x), list(y)<br><br>x, y = simulate_2d_bm()<br>fig = px.line({ &#39;x&#39; : x, &#39;y&#39; : y }, x=&#39;x&#39;, y=&#39;y&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mubLQz4nC_1HXuaab4w0HQ.png" /><figcaption>Simulation of a 2-D Brownian Motion</figcaption></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_3d_bm(nsteps=10000, t=0.01):<br>    x = np.cumsum([ np.random.randn()*np.sqrt(t) for i in range(nsteps) ])<br>    y = np.cumsum([ np.random.randn()*np.sqrt(t) for i in range(nsteps) ])<br>    z = np.cumsum([ np.random.randn()*np.sqrt(t) for i in range(nsteps) ])<br>    return list(x), list(y), list(z)<br><br>x, y, z = simulate_3d_bm()<br>fig = px.line_3d({ &#39;x&#39; : x, &#39;y&#39; : y, &#39;z&#39; : z }, x=&#39;x&#39;, y=&#39;y&#39;, z=&#39;z&#39;)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DOM2lWF6-f3kHYifBXqeGA.png" /><figcaption>Simulation of 3-D Brownian Motion for 10K steps</figcaption></figure><h4>Brownian Motion with drift</h4><p>This is variation of Brownian motion where there is drift component along with the usual random component.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/170/1*rd7QCfX9Q_0Ie9_4VK9XWA.png" /><figcaption>along with X(t) with random noise is μ*t which is drift</figcaption></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_1d_bm_with_drift(nsteps=1000, t=0.01, mu=0.5):<br>    steps = [ mu*0.01 + np.random.randn()*np.sqrt(t) for i in range(nsteps) ]<br>    y = np.cumsum(steps)<br>    x = [ t*i for i in range(nsteps) ]<br>    return x, y<br><br>nsims = 5<br>simulation_data = {}<br>for i in range(nsims):<br>    x, y = simulate_1d_bm_with_drift()<br>    simulation_data[&#39;y{col}&#39;.format(col=i)] = y<br>    simulation_data[&#39;x&#39;] = x<br><br>ycols = [ &#39;y{col}&#39;.format(col=i) for i in range(nsims) ]<br>fig = px.line(simulation_data, x=&#39;x&#39;, y=ycols)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l5kpv1-8ZZ_SPq8qMnzgQw.png" /><figcaption>Multiple simulations of 1-D Brownian Motion with drift</figcaption></figure><h3>Geometric Brownian Motion</h3><p>This process is often used to model financial stock prices or population growth, or in other situations where measurements cannot be negative. In finance it forms the basis for the Black–Scholes equation where the log returns of stock prices is modelled: the main interest is in the option pricing of derivatives.</p><p>It is defined as follows :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/367/1*NUBTqsW4mRKtB23fJJTamw.png" /></figure><p>G(t) is not a brownian motion but log[G(t)] is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*ZFItyBqFla7BkDr6M1W_MA.png" /></figure><pre>import random<br>import numpy as np<br>import plotly.express as px<br><br>def simulate_1d_gbm(nsteps=1000, t=1, mu=0.0001, sigma=0.02):<br>    steps = [ (mu - (sigma**2)/2) + np.random.randn()*sigma for i in range(nsteps) ]<br>    y = np.exp(np.cumsum(steps))<br>    x = [ t*i for i in range(nsteps) ]<br>    return x, y<br><br>nsims = 5<br>simulation_data = {}<br>for i in range(nsims):<br>    x, y = simulate_1d_gbm()<br>    simulation_data[&#39;y{col}&#39;.format(col=i)] = y<br>    simulation_data[&#39;x&#39;] = x<br><br>ycols = [ &#39;y{col}&#39;.format(col=i) for i in range(nsims) ]<br>fig = px.line(simulation_data, x=&#39;x&#39;, y=ycols)<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gchTb0KwPSjGfZZ800Bedg.png" /><figcaption>Multiple simulations of Geometric Brownian Motion</figcaption></figure><h4>Simulating Stock Price using Geometric Brownian Motion</h4><p>As seen the above definition we can use actual stock price data to estimate μ &amp; σ and use the parameters to simulate the stock price. In the following example we will look at the Dow Jones index between 2022–01–01 and 2022–12–31 i.e for the year of 2022. We will estimate the parameters of the Geometric Brownian Motion and use it to simulate the price of the index.</p><pre>import pandas_datareader.data as web<br>import numpy as np<br>import plotly.express as px<br>import random<br><br>def simulate_1d_gbm(nsteps=1000, t=1, mu=0.0001, sigma=0.02, start=1):<br>    steps = [ (mu - (sigma**2)/2) + np.random.randn()*sigma for i in range(nsteps) ]<br>    y = start*np.exp(np.cumsum(steps))<br>    x = [ t*i for i in range(nsteps) ]<br>    return x, y<br><br>df = web.DataReader(&#39;^DJI&#39;, &#39;stooq&#39;)<br>mask = ( &#39;2022-01-01&#39; &lt;= df.index ) &amp; ( df.index &lt;= &#39;2022-12-31&#39; )<br>df = df[mask]<br>prices = np.flip(df[&#39;Close&#39;].values)<br><br>logprices = np.log(prices)<br>logreturns = logprices[1:] - logprices[:-1]<br>mu = np.mean(logreturns)<br>sigma = np.std(logreturns)<br>nsteps = logprices.shape[0]<br><br>x, y = simulate_1d_gbm(nsteps=nsteps, mu=mu, sigma=sigma, start=prices[0])<br><br>data = {}<br>data[&#39;x&#39;] = x<br>data[&#39;Simulation&#39;] = y<br>data[&#39;DowJonesIndex&#39;] = prices<br><br>fig = px.line(data,x=&#39;x&#39;, y=[&#39;DowJonesIndex&#39;, &#39;Simulation&#39;])<br>fig.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MF0D1drsyarWIm0UXJFfTg.png" /><figcaption>Comparison of real the Dow Jones index with its simulation. x-axis is time in days</figcaption></figure><h3>References</h3><ul><li><a href="https://en.wikipedia.org/wiki/Brownian_motion">https://en.wikipedia.org/wiki/Brownian_motion</a></li><li><a href="https://en.wikipedia.org/wiki/Geometric_Brownian_motion">https://en.wikipedia.org/wiki/Geometric_Brownian_motion</a></li><li><a href="https://en.wikipedia.org/wiki/Random_walk">https://en.wikipedia.org/wiki/Random_walk</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17b6b4bd2a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[7 must read books for Reinforcement Learning]]></title>
            <link>https://medium.com/@mlblogging.k/7-must-read-books-for-reinforcement-learning-2847198c2163?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/2847198c2163</guid>
            <category><![CDATA[dynamic-programming]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[books-recommendation]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sun, 08 Jan 2023 17:08:58 GMT</pubDate>
            <atom:updated>2023-01-08T17:08:58.626Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Reinforcement Learning ?</h3><p>Reinforcement learning is a branch of machine learning which deals with sequential decision making. Reinforcement learning differs from supervised learning as there are no labels present but learning happens with the help of a reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*W5xFV5TujIK9vZzS9OsxIQ.png" /></figure><p>There is an agent which interacts with an environment and accumulates reward as a result of its action. The objective of the agent is to maximise its cumulative reward and the agent learns this by trial and error.</p><p>Reinforcement learning as field has existed for quite some time but it really gained mainstream popularity after DeepMind’s break through application of Q-learning to play Atari games. Since then reinforcement learning has had major breakthroughs in the recent years and gain in popularity. Some of innovative applications of reinforcement learning in the recent years can be seen in this <a href="https://medium.com/@mlblogging.k/9-awesome-applications-of-reinforcement-learning-e1306ed25c09">blog</a> and some of the interesting environments created can be found <a href="https://medium.com/@mlblogging.k/15-awesome-reinforcement-learning-environments-you-must-know-a38fb75867f2">here</a>.</p><p>The following are 7 must read books on Reinforcement learning which will give you the necessary background to use RL in your work or pursue further research in this field.</p><h4>1. <a href="https://www.amazon.com/Reinforcement-Learning-Introduction-Adaptive-Computation/dp/0262039249/">Reinforcement Learning : An Introduction</a> by Sutton &amp; Barto</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*dOzvbWg5vw7JG85BOXRwFA.jpeg" /></figure><p>Sutton’s book is a classic. Richard Sutton is one of pioneers in the field of reinforcement learning. This book covers all the basics of reinforcement learning from scratch and no prior background is necessary apart from basic calculus and linear algebra.</p><h4>2. <a href="https://www.amazon.com/Reinforcement-Learning-Stochastic-Optimization-Sequential/dp/1119815037/">Reinforcement Learning and Stochastic Optimization: A Unified Framework for Sequential Decisions</a> by Warren B. Powell</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*-81veJHQuGddgmmFJ6G0vA.jpeg" /></figure><p>This is in-depth book for reinforcement learning and covers many topics like stochastic optimization, online learning and reinforcement learning algorithms.</p><h4>3. <a href="https://www.amazon.com/Algorithms-Reinforcement-Synthesis-Artificial-Intelligence/dp/1608454924/">Algorithms for Reinforcement Learning</a> by Csaba Szepesvári</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*vmXOtw-67xckykhZ1iknEw.jpeg" /></figure><p>Csaba Szepesvári is a research scientist at DeepMind and is a prominent researcher in reinforcement learning. This book is a no nonsense mathematical introduction to reinforcement learning. It doesn’t talk about applications or any implementation details but follows only a rigorous mathematical treatment of reinforcement learning algorithms.</p><h4>4. <a href="https://www.amazon.com/Bandit-Algorithms-Tor-Lattimore/dp/1108486827/">Bandit Algorithms</a> by Tor Lattimore, Csaba Szepesvári</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*BXZjY4n5NqGAAnxjfBtdPA.jpeg" /></figure><p>Bandit is subset of reinforcement learning which deals with a special case where there is only one state and the agent tries to learn the reward distribution for each action. This book is a mathematical introduction to the topic of Bandits very much like the RL book by the same author.</p><h4>5. <a href="https://www.amazon.com/Reinforcement-Learning-Optimal-Control-Bertsekas/dp/1886529396/">Reinforcement Learning and Optimal Control</a> by Dimitri Bertsekas</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*ZKYk9sd49RZSRx1nSpouFw.jpeg" /></figure><p>Dimitri Bertsekas is well known researcher in machine learning. He has written various books on dynamic programming, neural networks &amp; reinforcement learning. This book follows a mathematical approach using dynamic programming.</p><h4>6. <a href="https://www.amazon.com/Grokking-Reinforcement-Learning-Miguel-Morales/dp/1617295450/">Grokking Deep Reinforcement Learning</a> by Miguel Morales</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*dGGAvbuJz_QLFISCsitt0Q.jpeg" /></figure><p>Grokking Deep RL is a book which makes the latest reinforcement learning algorithms easily accessible to everyone. The book covers the latest algorithms in Deep RL.</p><h4>7. <a href="https://www.amazon.com/Deep-Reinforcement-Learning-Hands-optimization/dp/1838826998/">Deep Reinforcement Learning Hands-On</a> by Maxim Lapan</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*JrE2BS86x0Jga-fszSt-Cw.jpeg" /></figure><p>As the name suggests this book takes a hands on approach to Deep RL. The book explores the usage of OpenAI gym and related libraries. Various example gym environments and Deep RL agents are discussed. Code for all the examples are also given.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2847198c2163" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 awesome books for Quantitative Trading]]></title>
            <link>https://medium.com/@mlblogging.k/10-awesome-books-for-quantitative-trading-fc0d6aa7e6d8?source=rss-566f32fb7dde------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc0d6aa7e6d8</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <category><![CDATA[algorithmic-trading]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[trading-system]]></category>
            <dc:creator><![CDATA[ML Blogger]]></dc:creator>
            <pubDate>Sat, 07 Jan 2023 18:31:00 GMT</pubDate>
            <atom:updated>2023-01-08T17:23:12.610Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Quantitative Trading ?</h3><p>Quantitative trading is the usage of mathematical models or algorithms to create trading strategies and trade them. Quant trading is usually employed by large institutional traders or hedge funds who employ large teams of PhDs and engineers. Historically the quantitative trading field has been very secretive and ideas which work tend to be guarded very closely by the firms but in the last few years the growth of openly available datasets and access to compute i.e ( in the form of GPUs and cloud ) has made quant trading accessible to a larger audience.</p><h3>Overview of Quantitative Trading</h3><p>Any quantitative trading system consists of the following steps</p><ul><li>Identifying / Creating a trading strategy</li><li>Backtesting the strategy</li><li>Designing a execution setup / system for your strategy</li><li>Managing risk</li></ul><p>Each of the above steps involve lot of research and trial and error to get right.</p><p>Quant trading is a complex field and requires careful and detailed study to be successful. The following are 10 such books which can help one get started on their Quant journey.</p><h4>1 . <a href="https://www.amazon.com/Quantitative-Trading-Build-Algorithmic-Business/dp/1119800064/">Quantitative Trading: How to Build Your Own Algorithmic Trading Business</a> by Ernest Chan.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*dEicFcSRJ_wYv7ksRO21kg.jpeg" /></figure><p>Ernest P. Chan is an award winning quantitative hedge fund manager. He was an researcher in machine learning at IBM. He now runs his own firm and is a well known author who has written multiple books for beginners in quantitative trading. You can find his blogs <a href="http://epchan.blogspot.com/">here</a>.</p><p>The above book he covers the basics of quantitative trading for a complete beginner. The objective of the book is to introduce all the main areas of focus involved in quant trading. It talks about only basic and beginner strategies.</p><h4>2. <a href="https://www.amazon.com/Machine-Trading-Deploying-Computer-Algorithms/dp/1119219604/">Machine Trading: Deploying Computer Algorithms to Conquer the Markets</a> by Ernest Chan.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*FWt_OipXA5FqcaLWG8CRqA.jpeg" /></figure><p>This book too is by Ernest P. Chan. It is a slightly older book from 2017 but the content is still relevant. The focus is not on the end to end quantitative trading process but on intro to strategies in domains such as</p><ul><li>Using Factor Models</li><li>AI for creating strategies</li><li>Options Strategies</li><li>Time series analysis</li><li>Intraday trading</li></ul><h4>3. <a href="https://www.amazon.com/Finding-Alphas-Quantitative-Approach-Strategies/dp/1119571219/">Finding Alphas: A Quantitative Approach to Building Trading Strategies</a> by Igor Tulchinsky.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*mCwGyUGkdptpgzKdgJ-V3A.jpeg" /></figure><p>This book is a classic and is a must read for any one seriously exploring to enter the quantitative trading space. Igor Tulchinsky is the founder and CEO of WorldQuant. WorldQuant is one of the most successful quantitative hedge funds. The aim of the book is not to discuss strategies but rather discuss the process of finding strategies. This book is about ‘alpha’ ( this is the jargon for trading strategies ) research and what are the steps and process you should follow to come up with new ideas. Each of the chapters in the book is written by a quant from WorldQuant.</p><h4>4. <a href="https://www.amazon.com/Advances-Financial-Machine-Learning-Marcos/dp/1119482089/">Advances in Financial Machine Learning</a> by Marcos Lopez de Prado</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*NBEnQDiEEYienyo8obVrCA.jpeg" /></figure><p>This book is by Marcos Lopez de Prado who is also known as “Master of the Robots”. He is currently Professor of Practice at Cornell University’s College of Engineering and before that he was Head of ML at AQR capital ( which is large hedge fund with a focus on using factor models for investing ). This book specifically talks about the challenges and opportunities of applying ML/AI to create trading strategies. This book was one of the first books to talk about all the nuances and challenges in applying ML/AI in quant space such as model overfitting, featurizing and selecting important features, backtesting and evaluating models.</p><h4>5. <a href="https://www.amazon.com/Machine-Learning-Managers-Elements-Quantitative/dp/1108792898/">Machine Learning for Asset Managers</a> by Marcos Lopez de Prado</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*4Lp0_IgL8Qwetiu7GGKgNA.jpeg" /></figure><p>This is another book by Marcos Lopez de Prado. The book talks about AI/ML methods in quant with more focus on portfolio construction, feature selection and identifying overfit models.</p><h4>6. <a href="https://www.amazon.com/Advances-Active-Portfolio-Management-Econometrics/dp/1260453715/">Advances in Active Portfolio Management</a> by Richard Grinold, Ronald Kahn</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*EM8r4F4_LqmkjOisQojEbw.jpeg" /></figure><p>This book was recommended to me by a friend who works in the quant industry. The focus of this book is not creating strategies but portfolio management. Portfolio management is the process of combining various signals and strategies into a single portfolio with the aim of reducing risk. The book talks about approaches to construct portfolios, optimize cost of trading and minimize risk.</p><h4>7. <a href="https://www.amazon.com/151-Trading-Strategies-Zura-Kakushadze/dp/3030027910/">151 Trading Strategies</a> by Zura Kakushadze, Juan Andres Serur</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*bG0tSXPvhfxGx4fvT4CrWA.jpeg" /></figure><p>Zura Kakushadze is a researcher in quant finance who has earlier also collaborated with WorldQuant. This is the latest book published by the author. It is a compilation of various strategies across asset classes and it is a source for advanced strategies. It is not a AI/ML focussed book but many of these strategies can be remodelled to use AI/ML. Some of the types of strategies discussed are</p><ul><li>Earnings Strategies</li><li>Options Strategies</li><li>Volatility Strategies</li><li>Momentum / Mean Reversion Strategies</li><li>Index and Forex Strategies.</li></ul><h4>8. <a href="https://www.amazon.com/Machine-Learning-Asset-Management-Developments/dp/1786305445/">Machine Learning for Asset Management</a> by Emmanuel Jurczenko</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*JeZ_Xwz8ZLpRM3e6fMQR8w.jpeg" /></figure><p>This book is focussed on applying ML in the trading space. The initial content is about an intro to ML focussing on the quant trading related concepts. The book talks about</p><ul><li>Creating long/short strategies</li><li>Using News / Sentiment for trades</li><li>Predicting returns using Machine Learning</li><li>Portfolio Optimization using Machine Learning.</li></ul><h4>9. <a href="https://www.amazon.com/Python-Algorithmic-Trading-Cloud-Deployment/dp/149205335X/">Python for Algorithmic Trading: From Idea to Cloud Deployment</a> by Yves Hilpisch</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*UT5WHyXEn0gdua4V7bazLg.jpeg" /></figure><p>This book focusses more on the engineering aspect of things. It mentions briefly about trading strategies. It talks in depth about how to implement all the required processes in python. The topics discussed are</p><ul><li>Handling financial datasets and reading them in python</li><li>How to setup cloud instances and docker for deployment</li><li>How to work with apis and databases</li><li>How to work with conda and virtual environments</li></ul><h4>10. <a href="https://www.amazon.com/Machine-Learning-Algorithmic-Trading-alternative/dp/1839217715/">Machine Learning for Algorithmic Trading</a> by Stefan Jansen</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*1HlIGLvhjFaYV-yi7lN9Nw.jpeg" /></figure><p>This book takes a hands on approach. It is very extensive in what it covers. Some of the important topics covered are</p><ul><li>Long / Short strategies using Random Forest / NN</li><li>Using News and NLP to enhance strategies</li><li>Using advanced models like CNN/RNNs</li><li>Explore how RL can be used for trading strategies.</li></ul><p>Apart from the above the book also talks about using technical indicators, linear models and libraries like Zipline, Alphalens, pyfolio for analysing the outcome of the strategies.</p><p>Apart from the above books another important book which is a prerequisite to understand many strategies is</p><h4><a href="https://www.amazon.com/Options-Futures-Other-Derivatives-11th/dp/B0B9JS99C2/">Options, Futures, and Other Derivatives</a> by John Hull &amp; S. Basu</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*rkfQygPI8HwSUH7Greqyug.jpeg" /></figure><p>John Hull’s book on Options and derivatives is classic introduction to understanding derivative instruments and no quant’s library is incomplete without this book.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc0d6aa7e6d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>