<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Carlo Parodi on Medium]]></title>
        <description><![CDATA[Stories by Carlo Parodi on Medium]]></description>
        <link>https://medium.com/@carloparodi?source=rss-d0b03247291d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*d-wNvjDUmqzK8TNttcReDg.jpeg</url>
            <title>Stories by Carlo Parodi on Medium</title>
            <link>https://medium.com/@carloparodi?source=rss-d0b03247291d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 08:24:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@carloparodi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Maybe your ML model is not that bad — A Business-driven approach to model evaluation]]></title>
            <link>https://medium.com/@carloparodi/maybe-your-ml-model-is-not-that-bad-a-business-driven-approach-to-model-evaluation-ce9cc4ba878e?source=rss-d0b03247291d------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce9cc4ba878e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[consulting]]></category>
            <category><![CDATA[data-science-training]]></category>
            <dc:creator><![CDATA[Carlo Parodi]]></dc:creator>
            <pubDate>Sun, 04 Feb 2024 17:22:59 GMT</pubDate>
            <atom:updated>2024-02-04T17:22:59.334Z</atom:updated>
            <content:encoded><![CDATA[<h3>Maybe your ML model is not that bad — A Business-driven approach to model evaluation</h3><p>You cleaned your data. You over/under-sampled it. You mined new features and you optimized the hyperparameters of the AI model. In short, you have been going through all the complex, sometimes mechanical, steps of <em>the Good Data Scientist</em>. At the end of the day, you get a brand-new model: it’s disastrous!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*K45O9FUFy253Xw7qAPMmMw.png" /></figure><p><em>Well done, yeah, thank you and goodbye!</em> you would say. Wouldn’t you?</p><p>But you shouldn’t.</p><p>The question here is: <em>how did I evaluate my model?</em> Probably — let’s say your exercise was a classification problem — you chose a list of metrics — let’s say, accuracy, AUC, F1 score, sensitivity, etc. — and you carefully evaluated them. Then, since your AUC is 0.64 and your F1 score doesn’t dare to go beyond 0.59, you have decided to abort your mission. This way you overlooked the very, sole important thing: <em>the benefits your model can provide to the business case it would be eventually applied to</em>.</p><figure><img alt="Actual footage of you going to school when they’d tell you R-squared should be .75 to be ok." src="https://cdn-images-1.medium.com/max/328/1*ubk3rbFfGM5af3RJ9kn6QQ.png" /><figcaption>Actual footage of you going to school when they’d tell you R-squared should be .75 to be ok.</figcaption></figure><p>You see, people tend to look at data scientists as lab geeks, with the head on numbers, abstract models and unconceivable formulas. Sometimes, just sometimes, they’re right. For instance, we’re still in the fabulous world of theory when we consider an R-squared as appreciable only if it exceeds 0.75. You should’ve left in the classroom some things your professors taught you a long time ago!</p><p>The new paradigm I’m going to suggest leverages on one important question: <strong>Is my ML model economically convenient? </strong>Yes — because <em>your goal is to provide an advantage</em> to the client/the company you’re working for, and a performance over/under a certain threshold does not mean anything. I give you an example.</p><p>You’re on a regression model to forecast the number of people entering a door in the next hour. You reach a R-squared of 0.9, and a MAE of 0.2. Great, no? No. Because the client has a model whose R-squared is higher and has a smaller MAE. You can keep your model, they won’t buy it.</p><p>Once again. You’re training a classification model to predict whether a person is male or female. Your best result is an accuracy of 0.55, and the other indexes are pretty much the same. Such a bad result, right? But let’s say that each person misclassified costs, for instance, 1.000 $ to the company, and that the model this company has adopted since to classify people is a coin-throw. Your very stupid AI would make spare 5.000 $ to your company every one-hundred people!</p><p>Of course, these examples are not realistic, but the concept to grasp is clear now, and applicable to a huge number of business cases you’ll stumble upon during your career as a DS. So here’s a first general statement to bear in mind:</p><p><strong>When evaluating a model, the focus should go to the benefits it can provide to the business compared to that provided by the currently solution.</strong></p><ul><li><em>Wait Carlo! You made us come all this way for this banal conclusion!?</em></li><li><em>No folks! Please stop! Just a little more of patience and I’ll get to the point!</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*WSN9_FG4aZn6qBmunBqD5w.png" /></figure><p>Phew! What seems banal to you now, is actually an important step for a further one: in fact, if we all agree that a model’s performance should be evaluated on the benefits provided to the business, then we are saying that our metrics vary from one industry to another: <em>we’re talking about KPIs</em>. Moreover, if we all agree that the new AI model should be compared to the status-quo solution, then we should provide some realistic ways to go through this. Let’s find out how.</p><h3>Advantages of the paradigm</h3><p>Any industry makes use of some specific KPIs. Using them, instead of a statistical index, to evaluate your performance helps in many ways. Here three main advantages of the approach:</p><ul><li>Evaluability. You can effectively evaluate what’s the convenience of adopting a model. You understand if a performance is good or not, and not because someone told you some statistical measure should be higher than X, but because you directly observe the economic advantage.</li><li>Result-driven models. You are not only evaluating a model on the economic result, but you’re also tuning it on a new loss function: the specific KPI(s) selected for your business case. This can make your ML pipeline longer, but also make your model more effective. In fact, domain related KPIs and statistical measures can tell you a very different story. For instance, a False Negative and a False Positive not always have the same importance: in a fraud detection exercise, you’d prefer to have some more false positives but reach a very tiny percentage of false negatives (i.e., non-spotted fraudulent people).</li><li>Explainability. Yes, not so many people will understand you when saying “RMSE has diminished with our new model!”, but if you talk about a smaller churn rate to insurance managers you can be sure you’re going to get their attention. No need to explain here that an efficient communication with stakeholders turns into higher success probability for your project.</li></ul><h3>Application of the paradigm</h3><p>The paradigm here shown can be summarized in a four-points approach:</p><ul><li><strong>Find the KPIs.</strong> First question: how many KPIs should I rely on? Not one, not even two, the correct answer is: as many as necessary! The first thing here is find a shared set of indices with the stakeholder. These indices shall be your metrics, not only those of the stakeholder. What if no KPIs are provided? In this case, it’s up to you to provide a clear definition of KPI to the client (should be something you can measure), accordingly to what seems his priorities. Find an agreed, clear definition of the KPI.</li><li><strong>Systematically translate models’ performance in terms of KPIs.</strong> Put in mathematical words, a substantial part of the problem consists in finding the relationship between your loss function and the chosen KPI to evaluate the advantage the ML model can bring. You not only want to measure models’ performance in terms of KPIs, but also adjust them to improve the KPIs, putting aside traditional measures.</li><li><strong>Get a Benchmark. </strong>You may want to compare your models to some status-quo situation. What if no data is available to measure the KPIs of the current solution? In this case, you need to find additional KPIs, or use a proxy. What’s a proxy? You can either find a proxy performance of the actual solution, by assuming some type of behaviors (agreed with the client), or, in case you have no tips to base your assumtions on, build a basic model to make a comparison. In the case of the model classifing gender’s people, for instance, compare your model to a random one, which says with P=50% a person is a male or female.</li></ul><h3>Conclusions</h3><p>Data Science is still a high-pace evolving domain. Over the last years, data scientists have become more and more aware of the industry and they’re adopting techniques of production similar to those seen in Software Industry. DevOps principles are now translated in MLOps tools, because we understand that a more robust, engineered pipeline is what market needs, and what Machine Learning Industry needs to survive and thrive.</p><p>This mindset is what we need at production level. At a strategic level, though, we need to be more aware of the mission of AI, we need to exit the lab and adopt a new focus on the challenges we face. In short, we need to restructure our approach to the needs we are called to meet. If the production logic of ML Industry has points in common to that of the Software Industry, the design of solutions must strengthen a consulting-driven attitude.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce9cc4ba878e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An AR(1) model estimation with Metropolis Hastings algorithm]]></title>
            <link>https://medium.com/analytics-vidhya/an-ar-1-model-estimation-with-metropolis-hastings-algorithm-a5a0fcc0675f?source=rss-d0b03247291d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5a0fcc0675f</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <dc:creator><![CDATA[Carlo Parodi]]></dc:creator>
            <pubDate>Thu, 26 Mar 2020 00:13:37 GMT</pubDate>
            <atom:updated>2020-03-30T08:46:44.743Z</atom:updated>
            <content:encoded><![CDATA[<p>Here we go, welcome to my second piece! Going through this article, you’ll learn how to build an optimization algorithm by using the Markov Chain Monte Carlo method (MCMC). I’m going to use a practical example to better show you the results by estimating an AR(1) process, but the rules here applied have several useful applications! I will write and show you the code in Python and the outputs!</p><p>A summary of the main topics covered:</p><p>- ARMA processes</p><p>- Transformation theorem for random variables</p><p>- Bayesian Statistics, short overview</p><p>- An application of MCMC: Metropolis-Hastings algorithm</p><p>So let’s see what’s the goal: we want to estimate by scratch the parameters α and β of the following process: yₜ = α +β yₜ₋₁ + εₜ</p><p>Where <em>y </em>is an observation at the moment <em>t </em>of a stochastic process called AR (Autoregressive process), belonging to the family of the ARMA processes. Each observation of such a process depends from a constant, namely α, and its past value, whose influence is proportional to β. There is moreover ε, which is a random variable independent from its previous values and identically distributed (i.i.d) as a Normal r.v., with μ = 0 and σ = 1.</p><p>Let’s suppose that we have N observations and we know that they derive from a process described as above. We need to know the value of α and β in order to forecast the values to come!</p><p>This problem finds a solution in two steps:</p><p>1) Find the objective function of y to maximize</p><p>2) Write a code to maximize the objective function</p><p>To accomplish the first step, we need to use an important result of the <strong>Transformation Theorem</strong>:</p><p>Given a random variable <em>x </em>(in our case, ε) distributed with density function fₓ(x) and a variable y = g(x): ℝ→ℝ (still <em>y</em> in this case), we have that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*NwdX20TGp2xMXb071ZmasA.png" /></figure><p>Bearing in mind that ε ∼ N(0,1), and that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/278/1*-LS_9jHoF94EFds9M_F50A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*fYcLe_KqsywuAVS0PzWxgg.png" /></figure><p>We find that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*6zFNG9w-03tulHkcra5bIw.png" /></figure><p>At this point we need to add a further consideration: we don’t need f(y), but the joint density function f(yₜ, yₜ₋₁, …, y₁). I’m not going to explain step by step the results here shown, but we can consider that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*oiTEcCTsCPNTI1MFgaFlkw.png" /></figure><p>Where f(yᵢ|Iᵢ₋₁) has been previously described, whereas f(y₁) is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*eypOvWTj_Rvmksnklo1JbQ.png" /></figure><p>Now that we have all the elements to build our objective function, we can consider some adjustments in order to speed up the algorithm we are going to write. At first we can use the logarithm of this function, in order to have sums were now we got a product and simplify the whole function. Moreover, we don’t need anything outside the exponential function, because there are only multiplicative constants independent from <em>y</em>.</p><p>What we get with these two adjustments is <strong>a function proportional to the log-likelihood function of our time series</strong>, whose shape is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*k3Ctk5HlGpT2Nh8vAavczQ.png" /></figure><p>The next step is to write an optimization algorithm to compute the maximization of the function above! Now, if we were to use an algorithm derived from the frequentist statistics, I’d have told you that the last function corresponds to the objective function. But we want to implement an algorithm derived from the bayesian approach, then things are going to be a bit different! What’s the frequentist statistic? Or the bayesian approach? Nice questions! In the next section we’ll shed light about these and other questions.</p><h4><strong>Bayesian Inference - a short intro</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*WpcUrYejwjpNkb66pav3FA.png" /><figcaption>Sheldon Cooper in Big Bang Theory while explaining the Bayes’ formula on a whiteboard</figcaption></figure><p>When talking about inference, statistics splits in two different schools of thought: the most famous and commonly adopted is the Frequentist, on the other hand there’s the Bayesian one. The two different approaches can be summarised as follows:</p><p>- <strong>The Frequentist approach</strong> considers a sample as the output of a random variable, whose we don’t know the parameters nor the shape, even though they exist per se. We can know the probability of an event only if we have enough data to base our estimations on. Reasonable, isn’t it?</p><p>- <strong>The Bayesian approach</strong> considers single parameters as a matter of belief. What I mean is that you, the observer of the sample, give an a priori probability to the parameters’ value (therefore not based on any previous evidence), and based on that you start your estimation. Of course you are going to base your estimates on data, and the more you have the better it is, but there’s also a subjective attribution of probability to a given event that is not possible with the first method.</p><p>I reckon an easier way to get the point is to look at the Bayes’ formula, which gives the name to the approach we are about to adopt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*rEm210V4COe1HGPpycSGmA.png" /></figure><p>Which we can translate straight away in our current problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/226/1*tjLagACwwJ5zzibc7l8X8g.png" /></figure><p>Where:</p><p>- <em>Θ</em> is the parameter we are interested to know</p><p>- <em>Ӯ</em> is the sample we analyse</p><p>- <em>P(Ӯ| Θ)</em> is the <strong>likelihood function</strong> we found in the previous section</p><p>- <em>P(Θ)</em> is called <strong>priori distribution</strong>, i.e. the probability we attribute to a value of Θ a priori</p><p>- <em>P(Ӯ)</em> is called <strong>evidence</strong></p><p>Notice we are seeking for the probability distribution function of a parameter! It depends on these three density functions. We already know the likelihood, the priori can be anything you want, because depends on your belief about the value of the parameter before seeing the sample. You don’t need to know any further about the evidence, since it is not going to play any role, as we will see later on. We pass now to introduce the Metropolis Hastings Algorithm.</p><h4><strong>Metropolis Hastings Algorithm</strong></h4><p>MH is a method of the Markov Chain Monte Carlo family (MCMC) used to simulate pseudo-random processes. You’ll wonder: what do we need it for? Well, the last formula we saw describes how a parameter is distributed on a given space in ℝ. Now <strong>we need a sample derived from that distribution</strong>, so that we can find the most likely value of that parameter, and <strong>MH allows us to create a pseudo-random samples from that distribution</strong>! The algorithm consists of the following steps:</p><p>1) We generate a starting value x₁. A proposal, say x₂, is generated from a <strong>proposal distribution</strong> which can be, for instance, a Uniform r.v., or in this case a Normal with μ = x₁ and σ that can be whatever value you want. I’ll give you some tips about the choice of σ later on, since it is quite a key point of the search.</p><p>2) For both x₁ and x₂ we attribute a probability given by the formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/1*m8jQqB4gS1C77MOEy87W-A.png" /></figure><p>3) Now that we have P(x₁|y) and P(x₂|y) we compute the <strong>ratio</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*F7bSs-WFk0wx4SLoDUw8xg.png" /></figure><p>4) The probability to accept the proposal x₂ is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*WsB1qKetl0fYNPbfnHrikA.png" /></figure><p>This means that, if P(x₂|θ) &gt; P(x₁|θ), x₂ shall be always accepted, whereas if it is not bigger, the probability for its acceptance shall be given by the ratio <strong><em>r </em></strong>described as above. In this last case, we generate a Uniform r.v. within the interval (0, 1) so that:</p><p>if : u&gt; r → <strong>reject x₂</strong></p><p>else: → <strong>accept x₂</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_3USoHFhkcIeOXBJ4lQcWg.png" /><figcaption>This schema summarises the acceptance criterion of the proposal value</figcaption></figure><p>5) Now you have two observations in your sample, then generate another proposal x₃ from the proposal distribution introduced at the step 1), with μ = to the last value of the process (i.e. x₂, if it has been accepted as the second value, or still x₁ if x₂ has been rejected).</p><p>Great! We have all the elements to write our code. Before starting and look at the code, just beware: I translated all of the formulas into logarithms, way easier to handle, so that the ratio has become a difference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gUMqS8siURxFEqVURrXc5w.png" /></figure><p>The logarithm must be applied also when comparing the value of xᵢ₊₁ to the uniform r.v., so that:</p><p>if : log(u) &gt; r → <strong>reject x₂</strong></p><p>else: → <strong>accept x₂</strong></p><h4><strong>Python code and results</strong></h4><p>We can finally write down the algorithm as discussed! You find below mine, so feel free to download or copy-paste it, but there are several ways to reach the same result!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/623f2bc0c8c772e412aab09165dbb772/href">https://medium.com/media/623f2bc0c8c772e412aab09165dbb772/href</a></iframe><p>Once you import it, you can follow some easy steps to get graphs and results of your estimate:</p><ul><li>We generate a pseudo random process:</li></ul><pre>y = [0]<br>for k in range (1, 10000):<br>    u = 0.3 + y[k-1]*0.7 + np.random.normal(loc = 0, scale = 1)<br>    y.append(u)</pre><p>This is generating 10.000 values from an AR process, with ε distributed as a Normal(0, 1), α = 0.3 and β = 0.7.</p><ul><li>Now feed the function “create_mcmc()” with the data just generated:</li></ul><pre>starting_points = (2, -1)</pre><pre>priori_alpha_average = 0.5<br>priori_alpha_stdev = 2<br>priori_beta_average = 0<br>priori_beta_stdev = 0.5</pre><pre>priori_params = priori_alpha_average, priori_alpha_stdev, priori_beta_average, priori_beta_stdev</pre><pre>new_values = MH_simulation.create_mcmc(y, 1000, starting_points, priori_params)</pre><p>Here I decided to set the starting points to 2 and -1, and these shall be the first two observations of respectively α and β. The input “priori_params” refers to the average and standard deviation of the Gaussian distribution that I decided to be my priori distribution for both α and β. With the function I invoked I’m generating 1000 values of α and β. Now we can see some results:</p><pre>MH_simulation.create_hist_4_param(new_values, &#39;alpha&#39;, 100)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/963/1*nmpAgkVdxbd3YfOsv1rq8Q.png" /><figcaption>This is the distribution of the alpha observations. Notice that the starting point is 2, on the right, the observations between it and the spike next to the true value can be considered as the “pattern of the process” towards its true average value.</figcaption></figure><pre>MH_simulation.create_hist_4_param(new_values, &#39;beta&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*1td3YTBJ4K2-4bgfTMjTFA.png" /><figcaption>Here as well you can see the steps the process took from the starting point (on the left) towards its average</figcaption></figure><p>As you can see, just considering the value with the highest frequency of observations can take us very close to the “true” value of the parameter: α estimated is -0.1869 with its true value set at -0.20. β estimated falls even closer, hitting exactly the true value (rounded to the third decimal!).</p><pre>MH_simulation.show_pattern(new_values)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*kDJI0UaXoq3Lcd_iZB5aqw.png" /></figure><p>The last graph shows the proposals the algorithm produced (in yellow and in green) and the current values of the Markov chains (in blue and red). These two last series are those we have seen in the histograms above. This graph explains quite well some important concepts: before arriving around the “true value” and settling around it, the Markov chain takes some steps (in this case, about 100). The number of steps needed to approach the average value depend on two things:</p><ul><li>The <strong>starting point you choose</strong>: the farther it is, the more are the steps required to allow the process to settle around its average.</li><li>The <strong>proposal distribution, and particularly its standard deviation </strong>(here I set it to 0.08). Why is the standard deviation that important? A too large value means that new proposals are bound to fall far from the current value, and this in turn means that — especially when the process is close to its average - there are few candidates likely to be accepted vs. the current value (basically, you won’t improve any further). On the other hand, a too small value means a lot of time to reach the proximity to the average, so you’re wasting observation that could be used otherwise! Moreover, you run the risk that you will never find the true value, since with a small change you could end up stuck on a tricky point!</li></ul><p>This is why it’s key to choose carefully your initial settings!</p><p>Finally, we have the joint histogram:</p><pre>MH_simulation.joint_hist(new_values, granularity = 50)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*4R2uyVGz5PLncjhourpdUg.png" /><figcaption>On the y-axis you find the values of β, on the x-axis the values of α</figcaption></figure><p>With the input “granularity”, which is set by default at 100, you can decide how detailed you want the image above.</p><p>Is that all, folks? Nope!!</p><p>We have generated two samples, one for alpha and one for beta, we have the histograms, but we haven’t decided how to estimate the true value. In the histograms above I implied I was taking as good the “most frequent value”-i.e. the modal value — but there is at least another statistics we could base our evaluation on: the average value! If you decide to use the average, just make attention: you do need to discard the first N steps we discussed above, which are not useful because represents the walk towards the average of the Markov chain from its starting point! How many steps do we need to discard? It depends: you’ll need to check the output and my graphs above, and make some trials. But I think at this point I gave you enough tips, so good job!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5a0fcc0675f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/an-ar-1-model-estimation-with-metropolis-hastings-algorithm-a5a0fcc0675f">An AR(1) model estimation with Metropolis Hastings algorithm</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A new Python module for Monte Carlo Simulations]]></title>
            <link>https://medium.com/analytics-vidhya/a-new-python-module-for-monte-carlo-simulations-7d0f1c0eaf96?source=rss-d0b03247291d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d0f1c0eaf96</guid>
            <category><![CDATA[garch]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <category><![CDATA[monte-carlo-simulation]]></category>
            <dc:creator><![CDATA[Carlo Parodi]]></dc:creator>
            <pubDate>Fri, 13 Mar 2020 10:19:45 GMT</pubDate>
            <atom:updated>2020-04-13T10:43:49.586Z</atom:updated>
            <content:encoded><![CDATA[<p>Once a friend of mine asked me: “Carlo, I need your help, I need a Monte Carlo simulation for some financial data, could you help me with the coding in Python?”.</p><p>First thing I thought to do was to google some key-worlds like “python-monte-carlo-module” and find a quick solution to work this problem out. I was a bit surprised when I didn’t manage to find anything satisfying, so I decided to start up a new module dedicated to Monte Carlo simulations on time series.</p><p>This module comprehends three classes:</p><ul><li><strong>AverageModel</strong>: this class is designed to find the best-fitting average model between nine different ARCH Models, based on the Akaike Information Criterion (AIC).</li><li><strong>VolatilityModel</strong>: this class helps you find the best-fitting volatility model the same way the previous one does, between six different GARCH Models.</li><li><strong>MonteCarloSimulation</strong>: this is a class allowing you to make simulations of future scenarios, letting you decide the number of simulations and time horizon. It also shows best and worst scenarios and the worst and best scenarios accordingly to a given confidence interval.</li></ul><p>Here we discuss in particular how to use this powerful and very quick module to run your data and get a great output in the blink of an eye!</p><p>If you want to go straight to a more synthetic documentation you find it here:</p><p><a href="https://www.slideshare.net/CarloParodi1/mcsimulation-documentation">https://www.slideshare.net/CarloParodi1/mcsimulation-documentation</a></p><p>You should first copy-paste this on your prompt to install the module:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/308/1*0QXMmzbUq3MNyh3iPLGwyw.png" /></figure><p>After installing the library, you can go to your editor and import the classes we briefly discussed as follows:</p><pre>from mc_simulation import AverageModel as am<br>from mc_simulation import VolatilityModel as vm<br>from mc_simulation import MonteCarloSimulation as mcs</pre><p>Now we should discuss about your data! It can be in xls or csv format: all the module needs here is the name of the column you are interested to analyse! By default, this column is named ‘Close’, because usually, when modeling financial time series, we refer to the closing price, and not to the Low, High, or whatever. But you can specify otherwise in the input. So let’s define a variable called ‘average_model’ this way:</p><pre>average_model = am(link = r&#39;\THE\PATH\TO\YOUR\FILE\ACEA.csv&#39;, file_type = &#39;csv&#39;, column_name = &#39;Close&#39;, p_value = 0.98)</pre><p>In bold you find the parmeters’ names, and all of them but the first one are optional. This means that, if not specified, AverageModel will assume the document you’re indicating is a csv file, and the column name is, as aforementioned, ‘Close’.</p><p>The p_value is, by default, 0.95, and indicates the region of rejection for the t-test you will run on each parameter of each model you are going to analyse in a while.</p><p>But let’s proceed with order! Now you have an instance of AverageModel, but nothing happened, so let’s type:</p><pre>average_model.fit()</pre><p>The output is not printed in the console, but now we have some interesting fields initialised:</p><ul><li>average_model.<strong>MeanModelsDict</strong>: this field is a dictionary containing, for each ARMA model computed, the Akaike Information Criterion value, the parameters’values, their t-test results, the covariance matrix (in this exact order).</li><li>average_model.<strong>bestModel</strong>: this is a string specifying the best fitting model, based on the Akaike Information Criterion (AIC).</li><li>average_model.<strong>MeanParamsDict</strong>: this field contains the values of the parameters of the best-fitting model.</li><li>average_model.<strong>t_test_values</strong>: this field contains the t-test results of the best-fitting model’s parameters. What’s the p-value of the test? By default 0.95, but you can modify it in input as previously shown!</li></ul><p>average_model.<strong>adf_result</strong>: this is the value of the Augmented Dikey-Fuller test. If you are arrived until this point, it means the test has been accepted: before going forth, in fact, the time series you provided needs to result covariance stationary, if not, the process of model evaluation gets interrupted!</p><p>Now that we have our average model, we can make a further step into the analysis of the data by modeling its volatility! To do that, we use the class VolatilityModel:</p><pre>volatility_model = vm(average_model = average_model, errors_distribution = &#39;Normal&#39;, p_value = 0.98)</pre><p>Again, we underlined the parameters to provide, but only the first one is mandatory to be provided in order to create an instance of VolatilityModel. Put another way, if you want to get a volatility model, you first have to get an average model! You can specify the errors distribution, which is by default set to ‘Normal’, by typing ‘t’ for the t-Student, ‘ged’ for the generalized normal distribution, or ‘skewt’ for the Skewed distribution. The p_value parameter allows you to decide the rejection region for the t-test run on each parameter of the GARCH models.</p><p>Now let’s find the best model:</p><pre>volatility_model.fit()</pre><p>As before, we can’t see any result on the editor, but we have some interesting fields to watch out for:</p><ul><li>volatility_model.<strong>VarianceModelsDict</strong>: this field is a dictionary containing, for each GARCH model computed, the Akaike Information Criterion value, the parameters’ values and their t-test results (in this order).</li><li>volatility_model.<strong>bestVolatilityModel</strong>: this field is a string specifying what is the best model, based on the AIC criterion.</li><li>volatility_model.<strong>VolatilityParamsDict</strong>: here are contained the parameters’ values of the best volatility model.</li><li>volatility_model.<strong>t_test_values</strong>: here you find the t-test results of the parameters of the best volatility model.</li></ul><h4><strong>And finally, we can create our Monte Carlo simulations!</strong></h4><p>To run them, we needed a volatility model — that we have just created — and to have it, in turn, we needed an average model, which is the first thing we computed. Let’s get started:</p><pre>mc_sim = mcs(volatility_model = volatility_model, time_horizon = 750, number_simulations=150)</pre><p>The only mandatory parameter is the first one (our volatility model), whereas time_horizon and number_simulations are set by default at the value, respectively, of 100 and 50 (only integer numbers are allowed). With “time horizon” we mean how far in time we want to go with our simulation. For instance, if we leave the default value of “100”, and data are made up by daily observations as in this case, we are saying we want to see in the next 100 days to come what’s going to be possible to happen.</p><p>With “number of simulations” we mean to set the number of simulations we want to compute on this time series. Keeping in mind the default values, if we keep them unchanged, the method we are about to run will compute 50 different scenarios, each of them made up by 100 observations.</p><p>Let’s see what happens:</p><pre>mc_sim.get_simulation()</pre><p>output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*vT1XBmTPELavTLtHgbShkA.png" /></figure><p>Amazing! Let’s say we’d like to see the average of all of these scenarios. This class provides the perfect method:</p><pre>mc_sim.get_average_scenario()</pre><p>output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*XMUAj2ZeBkWTTVBWzro8sw.png" /></figure><p>As you can notice, we also have the last observation value. How about if we want to get to see the extreme scenarios? To do that, we have another powerful method:</p><pre>mc_sim.get_extreme_scenarios(criterion = &#39;returns_average&#39;)</pre><p>The parameter ‘criterion’ is optional, and by default is set on ‘returns_average’. This means that the 150 scenarios we created are ranked by their average return:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*vpX5jbV04h9o2ww9gJYSdQ.png" /><figcaption>The average is always displayed in order to provide a benchmark</figcaption></figure><p>But you can set the criterion to ‘last_obs’, in this case, scenarios shall be ranked by their last observation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*7jjiIXpInTl8xpSP_Q_w2Q.png" /></figure><p>The last method I want to talk about, and perhaps the most relevant, is as follows:</p><pre>mc_sim.get_confidence_scenarios(conf_interval = 0.95, criterion = &#39;returns_average&#39;)</pre><p>The first parameter is mandatory, it allows us to see the scenario which ranks at the 95th percentile and the opposite one, which ranks at the 5th percentile. You can set this parameter to any floating number between 1 and 0.5 (excluded). This can be extremely useful to computer the Value at Risk of a financial time series at a given day in the future. The second parameter is exactly as discussed before, so by default is set to rank scenarios by their average return:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*qV-7WxFUy8_crt7LkTj4mg.png" /></figure><p>And here’s the result by setting: <strong>criterion </strong>= ‘last_obs’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*MhjjqkY9Sicuk5uj_ve1eg.png" /></figure><p>Great! How many information and graphs in a few lines of code!</p><p>If you have any doubt, or would like to see implemented some new features in the next version, don’t hesitate to contact me. Good job!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d0f1c0eaf96" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/a-new-python-module-for-monte-carlo-simulations-7d0f1c0eaf96">A new Python module for Monte Carlo Simulations</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>