<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gaurav Kumar on Medium]]></title>
        <description><![CDATA[Stories by Gaurav Kumar on Medium]]></description>
        <link>https://medium.com/@gauravkr927?source=rss-3255973ef5a6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*kmcSqtVhKfWoYyaXiKSu8w.jpeg</url>
            <title>Stories by Gaurav Kumar on Medium</title>
            <link>https://medium.com/@gauravkr927?source=rss-3255973ef5a6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 20:45:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gauravkr927/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deep Dive into Bayesian Optimization]]></title>
            <link>https://medium.com/@gauravkr927/bayesian-optimization-a655d75a679?source=rss-3255973ef5a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/a655d75a679</guid>
            <category><![CDATA[bayesian-machine-learning]]></category>
            <category><![CDATA[bayesian-inference]]></category>
            <category><![CDATA[bayesian-optimization]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <category><![CDATA[bayesian]]></category>
            <dc:creator><![CDATA[Gaurav Kumar]]></dc:creator>
            <pubDate>Mon, 19 Dec 2022 05:10:56 GMT</pubDate>
            <atom:updated>2022-12-20T11:20:47.197Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uMMo_1oA7Aluwy3f" /><figcaption>Photo by <a href="https://unsplash.com/@v2osk?utm_source=medium&amp;utm_medium=referral">v2osk</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>Optimization is core to Machine Learning, all parametric models are built on the optimization of an objective function. Roughly there are two types of optimization algorithms <br>1). Where the exact functional form of the objective function is known to us, in such cases we can apply different numerical methods such as Newton Raphson, the Midpoint method, or even a gradient-based approach, most of the Machine learning algorithm&#39;s objective function falls into this category where we formalize our objective function.<br> 2). The second one is where the exact functional form is not known, for that we generally use sample-based techniques to get an estimate of optimal points.(Genetic Algorithm, Particle Swarm Optimization, Bayesian Optimization falls into this category). Hyperparameter tuning for a deep neural network falls into this where we don’t have an exact functional form (parameter vs performance metrics). Even the sample calculation at different points are very heavy computation process. In such cases, Bayesian Optimization comes into the picture.</p><h3>When to use Bayesian Optimization?</h3><blockquote>For any Optimization problem if we are facing the following problems, we recommend to use Bayesian Optimization.<br>1. No exact functional form of the f(x) or f`(x)<br>2. Even for a single sample, f(x) is very expensive to evaluate (eg Metrics evaluation for a deep neural network for a particular sample of hyperparameter)<br>3. We have some observations of f(x), it may be noisy estimates or actual value<br>4. Want x* such that, x* = argmax(f(x))</blockquote><p>If our problem does not satisfy the above-mentioned criteria, we should move to alternative optimization algorithms. <br>Suppose the <strong>sample evaluation is easy to calculate</strong>, we might switch to the evolutionary algorithm, we can find the global optimum using that. <br>If the <strong>exact form is known</strong>, it’s much better to use the fast gradient-based technique or other numerical methods.</p><p>Let’s take an example of a company that is trying to replicate the taste of coke, since the original company Coca-Cola has already filed a patent over Coke, we don’t have legal rights to do the replication process. But since we know bayesian optimization we are going to use it to find a product which is much similar to coke. We are going to take the help of a taster who is going to evaluate the different compositions with respect to the original coke. Assuming this whole process is a very expensive process, we want to do it in the least number of trials.</p><h3>Terminology</h3><p>There are a few terminologies frequently used in Bayesian Optimization</p><p><strong>Surrogate function: </strong>It is used to estimate the exact functional form of f(x). We generally use Gaussian Process for this purpose. You can refer to Gaussian Process for complete details (<a href="https://medium.com/@gauravkr927/gaussian-process-ab50994b54a5">link)</a>. The main advantage of using GP over other functions is that we can use our existing knowledge about the behaviour of f(x). Since it is a bayesian based method, we can easily update the posterior distribution using likelihood and prior.</p><p>In our Coke example, the taster is acting as a surrogate function. He is evaluating the different compositions of drinks and scores the composition on a scale of 1 to 100. 100 being most similar, 1 being most dissimilar.</p><p><strong>Acquisition function</strong>: Since our black box function is very expensive to evaluate, we can’t evaluate at all points we need some sort of strategy to find which points to evaluate, the core purpose of the Acquisition function is used to find those points where to evaluate black box function given our previous evaluation. A good acquisition function balances exploitation(using existing information) and exploration (querying where we might find x* in an unexplored region).</p><p>In our coke example, which composition of ingredients should be tried next is coming from the acquisition function given all the historical composition and its corresponding score.</p><p><em>Upper bound function: It is an acquisition function defined as <br>a(x) = </em>𝛍(x) + k<strong>σ(x). <br></strong>There are many options available for<strong> the </strong>acquisition function.<strong><br></strong>Let us try to understand this equation intuitively 𝛍(x) is actually the expected value of the surrogate function, also we are trying to approximate the expensive function, <strong>σ(x) </strong>measures uncertainty from the expected value.</p><h3>Algorithms</h3><p>Let us define a few nomenclatures before going to the formal algorithm.<br><strong>f(x)</strong>: unknown black box function which is very expensive to evaluate, we also want to find global maxima for f(x)<br><strong>s(x)</strong>: surrogate function which is acting as an approximation to f(x)<br><strong>a(x)</strong>: acquisition function which is giving optimal point where to evaluate f(x)</p><p>Let&#39;s start with some Φ set of examples where we have few x and f(x) pairs. Let the sample be randomly distributed points in the bounds where we are trying to find optimal maximal, the cardinality of samples is very large.</p><blockquote>FOR iteration epochs:<br> 1. fit surrogate function on Φ<br> 2. Evaluate surrogate function on samples<br> 3. Evaluate acquisition function on samples, store it in acquisition values<br> 4. Select x from samples for which acquisition values is the largest<br> 5. Evaluate f(x) (expensive step)<br> 6. Update Φ = Φ U (x, f(x))<br>END FOR</blockquote><h3>Visualization</h3><p>Let’s visualize how s(x) is changing with each iteration, initially we are starting with just 2 data points at 0 and 1. Epoch 0 is representing the initial conditions. With the progress of each epoch, the acquisition function is selecting new data points, this new data point is selected for evaluation by the original expensive black-box function. At the end of epoch 1, we will build a surrogate function using 3 data points and so on, this process will continue until we reach the end of the total epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*O_s2hektu1OY0tqBXrNBQQ.png" /><figcaption>Epoch 0: This is the initial stage where we started with just 2 known data points, Note Surrogate function has minimum deviation at available values, surrogate function is built just using 2 data points</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*f73IwIMDuaTvutK2OKr4ww.png" /><figcaption>Epoch 1: Since we are giving a large weightage to exploration, the corner point got selected by the acquisition function, we again built the surrogate function using 3 points, with one additional point coming from the acquisition function.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*O0BNwwKaXK_0wzTXY8XhdQ.png" /><figcaption>Epoch 2: Once again a new data point got selected by the acquisition function, surrogate function was built using 4 data points.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*zPyQqxlyulxoG_ZBroBQ1Q.png" /><figcaption>Epoch 2: Since we have given huge weightage to exploration, a new point is selected from the region where the deviation is the largest.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*c5_q_6FdNbYzXdArnrbwsA.png" /><figcaption>Epoch 10: We are very close to the global maximum in just 10 epochs</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*z30t4dl2CzunktiShzKqvA.png" /><figcaption>Epoch 14: This image is a little bit zoomed version of earlier images</figcaption></figure><p>We can see that within a few epochs we are able to find the optimal points</p><h3>Implementation from Scratch in Python</h3><pre># importing relevant python libraries<br>import matplotlib.pyplot as plt<br>import matplotlib.patches as mpatches<br>import numpy as np<br>from scipy.spatial.distance import cdist</pre><pre>def kernel_function(X1, X2, length_scale=1):<br>    pairwise_distance = cdist(X1, X2, metric=&#39;sqeuclidean&#39;)<br>    return np.exp(-pairwise_distance/(2*length_scale**2))</pre><pre>def surrogate_function(X_train,y_train, X_test):<br>    sigma_11 = kernel_function(X_train, X_train)<br>    sigma_12 = kernel_function(X_train, X_test)<br>    sigma_22 = kernel_function(X_test, X_test)<br>    sigma_21 = kernel_function(X_test, X_train)<br>    <br>    mean_1 = np.mean(X_train, axis=1).reshape(-1, 1)<br>    mean_2 = np.mean(X_test, axis=1).reshape(-1, 1)<br>    <br>    mean = mean_2 + sigma_21@(np.dot(np.linalg.pinv(sigma_11), y_train - mean_1))<br>    sigma = sigma_22 - sigma_21@(np.linalg.pinv(sigma_11))@sigma_12<br>    <br>    # Only for numerical stability the below two operation is done, earlier<br>    # it was throwing nan values resulting computation issues<br>    sigma[np.isnan(sigma)] = 0<br>    sigma[sigma&lt;0.0001] = 0<br>    std = np.sqrt(np.diag(sigma))<br>    return mean.flatten(), std</pre><pre>def functional_form(x):<br>    return (x**2)*np.cos(x/3) + np.sin(4*x) + 10</pre><pre># Below is acquisition_function where we can control exploitation <br># vs exploration using kappa, exploration is directly proportional to kappa<br>def acquisition_function(mean, std, kappa=1):<br>    return mean + kappa*std<br><br>def next_poins(acquisition_values, X_test):<br>    return X_test[np.argmax(acquisition_values)]<br><br><br><br># Just for sake of visualization, we are trying to find optimal values for<br># a known function so that at each iteration we can compare with our original<br># function<br>x = np.linspace(start=-4, stop=4, num=150)<br>x = x.reshape(-1, 1)<br>y = functional_form(x)</pre><pre>def plot_utils(X_train, y_train, X_test, title=&quot;&quot;):<br>    <br>    mean, sigma = surrogate_function(X_train,y_train, X_test)<br>    <br>    plt.plot(x, y, &#39;.&#39;, color=&#39;blue&#39;)<br>    plt.plot(X_train, y_train, &#39;or&#39;)<br>    plt.plot(X_test, mean, &#39;-&#39;, color=&#39;orange&#39;)<br>    plt.fill_between(X_test.flatten(), mean - 2*sigma, mean + 2*sigma,<br>                         color=&#39;gray&#39;, alpha=0.2)<br>    <br>    <br>    pop_a = mpatches.Patch(color=&#39;blue&#39;, label=&#39;original black box function&#39;)<br>    pop_c = mpatches.Patch(color=&#39;orange&#39;, label=&#39;surrogate function&#39;)<br>    pop_b = mpatches.Patch(color=&#39;gray&#39;, label=&#39;95% confidence interval&#39;)<br>    plt.legend(handles=[pop_a, pop_b, pop_c], loc=&#39;lower right&#39;)<br>    plt.xlabel(&quot;x&quot;)<br>    plt.ylabel(&quot;y&quot;)<br>    plt.xlim(-4, 4)<br>    plt.title(title)<br>    <br>    plt.show()</pre><pre># Note we have only two known values of black box function at 0 and 1<br>np.random.seed(10)<br>X_intial = np.array([0, 1]).reshape(-1, 1)<br>y_initial = np.array([10, 10.18])<br><br>X_test = x.copy()<br>X_train = X_intial.copy()<br>y_train = y_initial.reshape(-1, 1).copy()</pre><pre>def bayes_optimization(X_intial, y_intial, x, epochs=15, kappa=50):<br>    X_test = x.copy()<br>    X_train = X_intial.copy()<br>    y_train = y_initial.reshape(-1, 1).copy()<br>    <br>    for epoch in range(epochs):<br>        plot_utils(X_train, y_train, X_test, title=f&quot;epoch = {epoch}&quot;)<br>        # calculate surrogate function which is cheap to evaluate<br>        mean, sigma = surrogate_function(X_train,y_train, X_test)<br>        # based on surrogate evaluation, find where to evaluate our expensive<br>        # function, the below two step is just doing that, note kappa is 50 means<br>        # high exploiration<br>        values = acquisition_function(mean, sigma, kappa=kappa)<br>        # define where to evaluate expensive f(x)<br>        x_next = next_poins(values, X_test)<br>        <br>        X_train = np.append(X_train, [x_next], axis=0)<br>        y_train = np.append(y_train, [functional_form(x_next)], axis=0)<br><br>bayes_optimization(X_train, y_train, X_test)</pre><h3>Conclusion</h3><p>In this article, we discussed the details of Bayesian Optimization, when to use it and how to use it. We also implemented Bayesian Optimization from scratch in Python. Bayesian Optimization is used a lot in Hyperparameter tuning in different boosting algorithms, Deep Neural Network Hyperparameter tuning.</p><h3>References</h3><p><a href="http://krasserm.github.io/2018/03/21/bayesian-optimization/">http://krasserm.github.io/2018/03/21/bayesian-optimization/</a><br><a href="https://bayesoptbook.com/">https://bayesoptbook.com/</a><br><a href="https://arxiv.org/abs/1807.02811">https://arxiv.org/abs/1807.02811</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a655d75a679" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gaussian Process]]></title>
            <link>https://medium.com/@gauravkr927/gaussian-process-ab50994b54a5?source=rss-3255973ef5a6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab50994b54a5</guid>
            <category><![CDATA[bayesian-modeling]]></category>
            <category><![CDATA[bayesian-machine-learning]]></category>
            <category><![CDATA[bayesian-inference]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <dc:creator><![CDATA[Gaurav Kumar]]></dc:creator>
            <pubDate>Thu, 15 Dec 2022 05:46:18 GMT</pubDate>
            <atom:updated>2022-12-20T05:31:59.000Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MKDDeDKxnXzDMpXX" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>Let’s start with some interesting properties of Normal distribution, these properties will later help us in deriving certain Gaussian Process equations. Assuming X and Y are two random variables that follow a normal distribution, then the following also holds true</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/235/1*swCbvUXO3pY1-8hbYmphdQ.gif" /></figure><blockquote>Then following are also a normally distributed.<br>1. X+Y<br>2. Joint distribution P(X=x, Y=y)<br>3. Conditional P(X=x| Y=y)<br>4. Marginalization<br>5. Multiplication of X and Y</blockquote><p>These properties are extensively used in many supervised algorithms for parameters estimation using MLE (Most Likelihood Estimate) and MAP (Maximum A Posteriori), let&#39;s look at the difference between them, In MLE we are trying to find parameters that are most likely to generate the observed data (P(D|w)), whereas in MAP we also take care of the prior distribution of parameters in addition to MLE (P(w|D)). MAP seems to be more reasonable than MLE since we are giving some weightage to our prior belief.</p><p>MLE: P(D| w) <br>MAP: P(w|D) ∝ P(D|w) x P(w)<br>where D represents training data (x, y) pairs and w are parameters</p><h3>Intuition</h3><p>Gaussian Process is a bayesian based supervised algorithm that gives us predictions with some confidence interval, It is also a non-parametric method. In GP we try to learn the distribution of functions P(f(x)) where x is any unobserved data or test data.</p><p>Let us assume x is a test point, y be corresponding unknown prediction and D be training dataset, then we are trying to get an estimate of y which is P(y| x, D). This can be rewritten as the marginalization of P(y, w| x, D)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/1*MSiwhY14q88HSuD8lbZcFg.gif" /><figcaption>Fig 1: Rewriting P(y|x, D) as a marginalization of P(y, w|x, D)</figcaption></figure><p>We can see P(y| x, D) is the marginalization of P(y, w| x, D) with respect to w, therefore the prediction value at x is weighted prediction over all possible vectors w. Let us try to understand equation 3 intuitively. <br>The first part P(y|x, w) is representing the distribution of y given weights(w) and features(x). The second part is referring to the prediction weightage given the training dataset and how probable are the weights. <br>In summary, we are going over all the weights possibilities, getting their corresponding influence on our test data (P(w|D)), and their prediction (P(y|x, w)). P(y|x, D) follows a normal distribution, this can be proved using product properties mentioned in the Introduction section.</p><blockquote><strong>P(y| x, D) follows Normal Distribution</strong><br>This can be proved using equation 3, P(y| x, w) follows normal distribution (see fig 2 for reference, for simplicity we have considered a linear model)<br>P(w| D) also follows Normal distribution, this can be also proved as P(D|w) is also normal distribution and if we assume P(w) as normally distributed, P(y|x, D) follows Normal distribution</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iKbgkZ2GF8AlgmweGHrVkQ.jpeg" /><figcaption>Fig 2 At each point, It follows a normal distribution with the same variance, this is the same as <strong>P(y| x, w)</strong></figcaption></figure><p>Look at the below figure (1), each individual curve is representing a different function based on some weights, each passing through the observed training data(black dots).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hVNARIt_HC0JtIv1Tq6nkg.png" /><figcaption>Fig 3: The separate line indicates the different functions, and black dots represent actual data points, for getting inference at any point test data x, and different models and their corresponding weights can be used to generate confidence intervals.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*VX0ySfeNmR4RbtKDTdbgww.png" /><figcaption>Fig 4: The confidence interval gets wider as we move away from observation data points, for any test data point the near points play important role in the prediction</figcaption></figure><h3>Mathematical Representation</h3><p>Let us assume y2 is prediction at x2, y1, and x1 are observation points.</p><p><strong>The joint distribution P(y1, y2) is given by below equation</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/351/1*ZaqmzSVoAbIZBTuvrNKmlA.png" /><figcaption>Fig 4: Joint Distribution</figcaption></figure><p><strong>Conditional Distribution of P(y2 | y1, x1, x2)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*n9ETO01z6fZ3rFmjhrsuSA.png" /><figcaption>Fig 5: Conditional Distribution</figcaption></figure><p>We use the conditional distribution mentioned in Fig 5, for inference on test data. There are several options to select the Kernel function for measuring covariances.</p><h3>Kernel Function</h3><p>The Kernel function is used to define covariance between X and X*, As we know covariance is the measure of the relationship between two random variables, a +ve covariance means the random variables tend to move in the same direction while -ve covariance means the random variables tend to move in the opposite direction. Any function which can satisfy some properties can be a kernel function. You can find these properties <a href="https://people.eecs.berkeley.edu/~jordan/kernels/0521813972c03_p47-84.pdf">here</a>.</p><blockquote>Suppose k is a kernel function then it should satisfy following conditions<br><em>1. k(x, y) = f(x) . f(y) <br>2. k is also symmetric from property 1, k(y, x) = f(y) . f(x)<br>3. k is kernel function if and only if Kernel matrix (K) should be +ve semidefnite matrix</em></blockquote><p>Some commonly used kernel functions are the Rational Quadratic Kernel, and Radial Basis Function.</p><h3>Python Implementation from Scratch</h3><pre># importing relevant python libraries<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import matplotlib.patches as mpatches<br>from scipy.spatial.distance import cdist</pre><pre>def model(x):<br>    return x * np.sin(x)<br><br># simulating some data points from<br>X_train = np.array([1, 3, 5, 6, 8]).reshape(-1, 1)<br>y_train = model(X_train)<br>X_test = np.linspace(0, 10, 100).reshape(-1, 1)</pre><pre># Defining rbf kernel function<br>def kernel_function(X1, X2, length_scale=1):<br>    pairwise_distance = cdist(X1, X2, metric=&#39;sqeuclidean&#39;)<br>    return np.exp(-pairwise_distance/(2*length_scale**2))<br><br># generating kernel matrix(covariance matrix) for training &amp; test data<br>sigma_11 = kernel_function(X_train, X_train)<br>sigma_12 = kernel_function(X_train, X_test)<br>sigma_22 = kernel_function(X_test, X_test)<br>sigma_21 = kernel_function(X_test, X_train)<br><br># mean vector<br>mean_1 = np.mean(X_train, axis=1).reshape(-1, 1)<br>mean_2 = np.mean(X_test, axis=1).reshape(-1, 1)</pre><pre># mean vector &amp; covariance matrix for test data<br>mean = mean_2.flatten() + \<br>sigma_21@np.dot(np.linalg.inv(sigma_11), (y_train - mean_1)).flatten()<br><br>sigma = sigma_22 - sigma_21@(np.linalg.inv(sigma_11))@sigma_12<br>sigma = np.sqrt(np.diag(sigma))</pre><pre># plotting test &amp; train data<br>plt.plot(X_train, y_train, &#39;or&#39;)<br>plt.plot(X_test, mean, &#39;-&#39;, color=&#39;orange&#39;)<br>plt.plot(X_test, model(X_test), &#39;-&#39;, color=&#39;blue&#39;)<br>plt.fill_between(X_test.flatten(), mean - 2*sigma, mean + 2*sigma,<br>                 color=&#39;gray&#39;, alpha=0.2)<br><br>pop_a = mpatches.Patch(color=&#39;red&#39;, label=&#39;training dataset&#39;)<br>pop_b = mpatches.Patch(color=&#39;blue&#39;, label=&#39;test label&#39;)<br>pop_c = mpatches.Patch(color=&#39;orange&#39;, label=&#39;test data prediction&#39;)<br>pop_d = mpatches.Patch(color=&#39;gray&#39;, label=&#39;confidence interval&#39;)<br>plt.legend(handles=[pop_a, pop_b, pop_c, pop_d], loc=&#39;upper left&#39;)<br><br>plt.xlabel(&quot;x&quot;)<br>plt.ylabel(&quot;y&quot;)<br>plt.xlim(0, 10)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*-3fhW7kqRd4a5nOm-fWtAw.png" /><figcaption>Fig 6: Plotting test data predictions &amp; confidence interval</figcaption></figure><h3>Conclusion</h3><p>This article introduced the mathematical intuition &amp; derivation of the core equations working behind the Gaussian Process, we also implemented a naive version of GP from scratch in Python. Gaussian Process can have huge applications where confidence intervals are very crucial, want to replicate behaviours of training data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab50994b54a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>